Apache Spark Core and Structured Streaming 3.0 In-Depth
In-Depth, Hands-On driven exposure to the features and concepts of Spark Core with tips on tuning its performance
Big Data,Data & Analytics,Data Science,PySpark,Hadoop
Lectures -50
Quizzes -8
Duration -13.5 hours
30-days Money-Back Guarantee
Get your team access to 10000+ top Tutorials Point courses anytime, anywhere.
Course Description
Apache Spark has turned out to be the most sought-after skill for any big data engineer. An evolution of MapReduce programming paradigm, Spark provides unified data processing from writing SQL to performing graph processing to implementing Machine Learning algorithms. It effectively uses cluster nodes and better memory management to spread the load across cluster of nodes to get faster results. Apache Spark drives the mission of data-driven-decision-making in thousands of organizations.
In order to fairly appreciate the benefits of the libraries of Apache Spark, it is essential to know the foundations right. This course aims exactly at that part. It starts from the beginner level and gradually explains all the complex concepts in an easy to reflect manner. It gives a profound description of the features and working of the framework through 5 different use cases with detailed hands on implementations. In fact, some hands-on sessions and solutions to the use-cases are explained in a full classroom mode with videos extending over 40 mins. After taking this course, you will gain the expertise on Spark Core and usage of further libraries like Spark SQL, Structured Streaming, Spark ML and GraphX will be much easier to visualize, implement and optimize.
This illustrative course will build your foundational knowledge. You will learn the differences between batch & stream processing, programming model, the APIs and the challenges specific to stream processing. Quickly we'll move to understand the concepts of stream processing with wide varieties of examples & hands-on, dealing with inner working and taking a use case towards the end. All of this activity will be on cloud using Spark 3.0.
Goals
- Strong focus on the practicality by getting into hands-on mode with plentiful of examples
- Develop in-depth understanding of the underlying concepts the core of Apache Spark
- Know the ways to get the best performance from Spark in production
- Avoid the common pitfalls when writing Spark applications
- In Depth exploration of Spark Structured Streaming 3.0 using Python API
- Get introduced to Apache Kafka on a high level in the process
- Understand the nuances of Stream Processing in Apache Spark
- Discover various features Spark provides out of the box for Stream Processing
Prerequisites
We'll be using Python API in Spark Programming. However, we'll explain all the programs in details but fundamental knowledge of Python will be beneficial.
![Apache Spark Core and Structured Streaming 3.0 In-Depth Apache Spark Core and Structured Streaming 3.0 In-Depth](https://d3mxt5v3yxgcsr.cloudfront.net/courses/18276/course_18276_image.jpeg)
Curriculum
Check out the detailed breakdown of what’s inside the course
Distributed Data Processing
2 Lectures
-
Expectations from Data Processing Framework 11:41 11:41
-
Introduction to Map Reduce 22:09 22:09
Resources: Spark Core
1 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Concepts of Apache Spark
3 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Starting HandsOn with Spark Applications
3 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Deep Dive into Spark Concepts
3 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Exploring operations on RDDs
5 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Advanced Concepts of Apache Spark
9 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Running Spark Application in a Cluster
1 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Tuning Spark Applications
5 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Interview with Spark Expert
1 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
First Steps with Spark Structured Streaming
5 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Resources: Spark Structured Streaming
1 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Deep Dive into Spark Structured Streaming
4 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Integrating Spark Structured Streaming with Kafka
3 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Applying Structured Streaming in Production and Road to Expertise
4 Lectures
![Tutorialspoint](/market/public/assets/images/loader.gif)
Instructor Details
![Amit Ranjan](https://www.tutorialspoint.com/assets/profiles/919194/profile/200_1454863-1708934589.jpg)
Amit Ranjan
Course Certificate
Use your certificate to make a career change or to advance in your current career.
![sample Tutorialspoint certificate](/market/public/assets/newDesign/img/primePacks/V-certificate.png)
Our students work
with the Best
![adobe logo](/market/public/assets/newDesign/img/adobe.png)
![adp logo](/market/public/assets/newDesign/img/adp.png)
![apple logo](/market/public/assets/newDesign/img/apple.png)
![armani logo](/market/public/assets/newDesign/img/armani.jpg)
![bosch logo](/market/public/assets/newDesign/img/bosch.png)
![capegemini logo](/market/public/assets/newDesign/img/capegemini.png)
![cisco logo](/market/public/assets/newDesign/img/cisco.png)
![hdfc bank logo](/market/public/assets/newDesign/img/hdfcbank.png)
![ibm logo](/market/public/assets/newDesign/img/ibm.png)
![intel logo](/market/public/assets/newDesign/img/intel.png)
![mastercard logo](/market/public/assets/newDesign/img/mastercard.png)
![netflix logo](/market/public/assets/newDesign/img/netflix.png)
![qualcomm logo](/market/public/assets/newDesign/img/qualcomm.png)
![samsung logo](/market/public/assets/newDesign/img/samsung.png)
![sony logo](/market/public/assets/newDesign/img/sony.png)
![unilever logo](/market/public/assets/newDesign/img/unilever.png)
![walmart logo](/market/public/assets/newDesign/img/walmart.png)
![adobe logo](/market/public/assets/newDesign/img/adobe.png)
![adp logo](/market/public/assets/newDesign/img/adp.png)
![apple logo](/market/public/assets/newDesign/img/apple.png)
![armani logo](/market/public/assets/newDesign/img/armani.jpg)
![bosch logo](/market/public/assets/newDesign/img/bosch.png)
![capegemini logo](/market/public/assets/newDesign/img/capegemini.png)
![cisco logo](/market/public/assets/newDesign/img/cisco.png)
![hdfcbank logo](/market/public/assets/newDesign/img/hdfcbank.png)
![ibm logo](/market/public/assets/newDesign/img/ibm.png)
![intel logo](/market/public/assets/newDesign/img/intel.png)
![mastercard logo](/market/public/assets/newDesign/img/mastercard.png)
![netflix logo](/market/public/assets/newDesign/img/netflix.png)
![qualcomm logo](/market/public/assets/newDesign/img/qualcomm.png)
![samsung logo](/market/public/assets/newDesign/img/samsung.png)
![sony logo](/market/public/assets/newDesign/img/sony.png)
![unilever logo](/market/public/assets/newDesign/img/unilever.png)
![walmart logo](/market/public/assets/newDesign/img/walmart.png)
Related Video Courses
View MoreAnnual Membership
Become a valued member of Tutorials Point and enjoy unlimited access to our vast library of top-rated Video Courses
Subscribe now![Annual Membership Annual Membership](/market/public/assets/newDesign/img/membership-large-small.jpg)
Online Certifications
Master prominent technologies at full length and become a valued certified professional.
Explore Now![Online Certifications Online Certifications](/market/public/assets/newDesign/img/traning-for-a-team.png)