Given the rise of IoT and other real-time sources and businesses’ desire to draw fast insights, there is a growing imperative for data professionals to build streaming data pipelines. Given the plethora of different tools and frameworks in the big data community, it is challenging to architect such pipelines correctly that achieve the desired performance and data quality. Hence, it is important to properly understand the business and technical requirements of the pipelines and accordingly select the right tools to build them.
By combining the scalability, streaming, and access to advanced analytics of Apache Spark with the performance and ACID compliance of a data warehouse, Delta Lake helps solve many of the pain points of building a streaming system to analyze data in real-time.
In this webinar, we are going to show how Structured Streaming and Delta Lake together make it super-easy to write end-to-end pipelines. Specifically, we will cover the following:
Sreeram Nudurupati, Senior Solutions Architect, Databricks
Thank you for registering. Please check your inbox for the login details. We look forward to your attendance.