Build a Mobile Gaming Events Data Pipeline with Databricks Delta
How to build an end-to-end data pipeline with Structured Streaming The world of mobile gaming is fast paced and requires the ability to scale quickly. With millions of users around the world generating millions of events per second by means of game play, you will need to calculate key metrics (score adjustments, in-game purchases, in-game...
Announcing RStudio and Databricks Integration
At Databricks, we are thrilled to announce the integration of RStudio with the Databricks Unified Analytics Platform. You can try it out now with this RMarkdown notebook (Rmd | HTML) or visit us at www.databricks.com/rstudio. For R practitioners looking at scaling out R-based advanced analytics to big data, Databricks provides a Unified Analytics Platform that...
Introducing Getting Started with Apache Spark on Databricks
We are proud to introduce the Getting Started with Apache Spark on Databricks Guide. This step-by-step guide illustrates how to leverage the Databricks’ platform to work with Apache Spark. Our just-in-time data platform simplifies common challenges when working with Spark: data integration, real-time experimentation, and robust deployment of production applications. Databricks provides a simple, just-in-time...
Apache Spark Key Terms, Explained
As observed in the Fortune article Survey shows huge popularity spike for Apache Spark: "Apache Spark is the Taylor Swift of big data software. The open source technology has been around and popular for a few years. But 2015 was the year Spark went from an ascendant technology to a bona fide superstar." One of...
Genome Sequencing in a Nutshell
This is a guest post from Deborah Siegel from the Northwest Genome Center and the University of Washington with Denny Lee from Databricks on their collaboration on genome variant analysis with ADAM and Spark. This is part 1 of the 3 part series Genome Variant Analysis using K-Means, ADAM, and Apache Spark: 1. Genome Sequencing in...
Parallelizing Genome Variant Analysis
This is a guest post from Deborah Siegel from the Northwest Genome Center and the University of Washington with Denny Lee from Databricks on their collaboration on genome variant analysis with ADAM and Spark. This is part 2 of the 3 part series Genome Variant Analysis using K-Means, ADAM, and Apache Spark: 1. Genome Sequencing in...
Predicting Geographic Population using Genome Variants and K-Means
This is a guest post from Deborah Siegel from the Northwest Genome Center and the University of Washington with Denny Lee from Databricks on their collaboration on genome variant analysis with ADAM and Spark. This is part 3 of the 3 part series Genome Variant Analysis using K-Means, ADAM, and Apache Spark: 1. Genome Sequencing in...
On-Time Flight Performance with GraphFrames for Apache Spark
Introduction Graph structures are a more intuitive approach to many classes of data problems. Whether traversing social networks, restaurant recommendations, or flight paths, it is easier to understand these data problems within the context of graph structures: vertices, edges, and properties. For example, the analysis of flight data is a classic graph problem as airports...
Findify’s Smart Search Gets Smarter with Apache Spark MLlib and Databricks
Spark Summit East is just around the corner! If you haven’t registered yet, you can get tickets here with this promo code for 20% off: Databricks20 We are happy to announce that Findify has deployed Databricks as its machine learning and analytics platform, achieving faster time to complete projects, more efficient operations, and improved collaboration. You can...
Databricks launches Meetup-in-a-box for Apache Spark Meetup Organizers
One of the most important reasons for the growth of Apache Spark is the amazing grassroots community interest and support to share, teach, and learn with each other. At the heart of the Spark community movement are the Spark Meetup organizers who continue to dedicate their time, resources, and effort. This core group of individuals...
