Running Streaming Jobs Once a Day For 10x Cost SavingsMay 22, 2017 by Burak Yavuz and Tyson Condie in Engineering Blog This is the sixth post in a multi-part series about how you can perform complex streaming analytics using Apache Spark. Traditionally, when people...
Processing Data in Apache Kafka with Structured Streaming in Apache Spark 2.2April 26, 2017 by Kunal Khamar, Tyson Condie and Michael Armbrust in Engineering Blog This is the third post in a multi-part series about how you can perform complex streaming analytics using Apache Spark. In this blog...
Working with Complex Data Formats with Structured Streaming in Apache Spark 2.1February 23, 2017 by Burak Yavuz, Michael Armbrust, Tathagata Das and Tyson Condie in Engineering Blog In part 1 of this series on Structured Streaming blog posts, we demonstrated how easy it is to write an end-to-end streaming ETL...
Real-time Streaming ETL with Structured Streaming in Apache Spark 2.1January 19, 2017 by Tathagata Das, Michael Armbrust and Tyson Condie in Engineering Blog Explore why lakehouses are the data architecture of the future with the father of the data warehouse, Bill Inmon. Try this notebook in...