Save for later
Exploring the Apache Spark Structured Streaming API for Processing Streaming Data
Stream processing applications work with continuously updated data and react to changes in real-time. In this course, Exploring the Apache Spark Structured Streaming API for Processing Streaming Data, you'll focus on using the tabular data frame API as well as Spark SQL to work with streaming, unbounded datasets using the same APIs that work with bounded batch data. First, you’ll explore Spark’s support for different data sources and data sinks, understand the use case for each and also understand the fault-tolerance semantics that they offer. You’ll write data out to the console and file sinks, and customize your write logic with the foreach and foreachBatch sinks. Next, you'll see how you can transform streaming data using operations such as selections, projections, grouping, and aggregations using both the DataFrame API as well as Spark SQL. You'll also learn how to perform windowing operations on streams using tumbling and sliding windows. You'll then explore relational join operations between streaming and batch sources and also learn the limitations on streaming joins in Spark. Finally, you'll explore Spark’s support for managing and monitoring streaming queries using the Spark Web UI and the Spark History server. When you're finished with this course, you'll have the skills and knowledge to work with different sources and sinks for your streaming data, apply a range of processing operations on input streams, and perform windowing and join operations on streams.
Get a Reminder
Rating | Not enough ratings |
---|---|
Length | 2.8 hours |
Starts | On Demand (Start anytime) |
Cost | $35/month (Access to entire library- free trial available) |
From | Pluralsight |
Instructor | Janani Ravi |
Download Videos | On Windows, MacOS, iOS, and Android Pluralsight app |
Language | English |
Subjects | IT & Networking |
Tags | Data Professional |
Get a Reminder
Similar Courses
Careers
An overview of related careers and their average salaries in the US. Bars indicate income percentile.
Data 1 2 $50k
Streaming Data Developer/Architect $59k
Big Data Developer (Streaming Data) $77k
Streaming Support Producer $84k
Associate Senior Webcast Engineer / Streaming Media Producer $95k
Live Streaming Engineer $100k
PYTHON/DJANGO Developer (Video Streaming Exp) REMOTE WORK $109k
Project Manager, Streaming Media $119k
Data Base Designer and Data Administrator $124k
Streaming Content Producer $131k
Android Streaming Engineer $135k
Senior Streaming Content Producer $174k
Write a review
Your opinion matters. Tell us what you think.
Please login to leave a review
Rating | Not enough ratings |
---|---|
Length | 2.8 hours |
Starts | On Demand (Start anytime) |
Cost | $35/month (Access to entire library- free trial available) |
From | Pluralsight |
Instructor | Janani Ravi |
Download Videos | On Windows, MacOS, iOS, and Android Pluralsight app |
Language | English |
Subjects | IT & Networking |
Tags | Data Professional |
Similar Courses
Sorted by relevance
Like this course?
Here's what to do next:
- Save this course for later
- Get more details from the course provider
- Enroll in this course