Taking Apache Spark’s Structured Streaming to Production
This is the fifth post in a multi-part series about how you can perform complex streaming analytics using Apache Spark. At Databricks, we’ve...
Detecting Abuse at Scale: Locality Sensitive Hashing at Uber Engineering
This is a cross blog post effort between Databricks and Uber Engineering. Yun Ni is a software engineer on Uber’s Machine Learning Platform...
Event-time Aggregation and Watermarking in Apache Spark’s Structured Streaming
This is the fourth post in a multi-part series about how you can perform complex streaming analytics using Apache Spark. Continuous applications often...
Processing Data in Apache Kafka with Structured Streaming in Apache Spark 2.2
This is the third post in a multi-part series about how you can perform complex streaming analytics using Apache Spark. In this blog...
Query Watchdog: Handling Disruptive Queries in Spark SQL
Read Rise of the Data Lakehouse to explore why lakehouses are the data architecture of the future with the father of the data...
Real-Time End-to-End Integration with Apache Kafka in Apache Spark’s Structured Streaming
View the Notebook in Databricks Community Edition Structured Streaming APIs enable building end-to-end streaming applications called continuous applications in a consistent, fault-tolerant manner...
Take Reports From Concept to Production with PySpark and Databricks
Introduction: What is MediaMath? MediaMath is a demand-side media buying and data management platform. This means that brands and ad agencies can use...
Next Generation Physical Planning in Apache Spark
Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway. — Andrew Tanenbaum, 1981 magine a cold, windy...
Delivering a Personalized Shopping Experience with Apache Spark on Databricks
This is a guest blog from our friends at Dollar Shave Club. Dollar Shave Club (DSC) is a men's lifestyle brand and e-commerce...
The Tenth Spark Summit with a Terrific Agenda for All
The number 10 is often used as a measuring yardstick to denote achievement, attainment or accomplishment: the 10th anniversary; a perfect score of...