Databricks: From raw data, to insights and data products in an instant!
Enterprises have been collecting ever-larger amounts of data with the goal of extracting insights and creating value. Yet despite a few innovative companies...
Databricks at Strata San Jose
The Strata + Hadoop World Conference in San Jose last week was abuzz with "putting data to work" in keeping with this year's...
Extending MemSQL Analytics with Apache Spark
Read Rise of the Data Lakehouse to explore why lakehouses are the data architecture of the future with the father of the data...
Introducing DataFrames in Apache Spark for Large Scale Data Science
Today, we are excited to announce a new DataFrame API designed to make big data processing even easier for a wider audience. When...
Apache Spark: A review of 2014 and looking ahead to 2015 priorities
2014 has been a year of tremendous growth for Apache Spark. It became the most active open source project in the Big Data...
Automatic Labs Selects Databricks for Primary Real-Time Data Processing
We're really excited to share that Automatic Labs has selected Databricks as its preferred big data processing platform. Automatic Labs needed to run...
"Learning Spark" book available from O'Reilly
Today we are happy to announce that the complete Learning Spark book is available from O’Reilly in e-book form with the print copy...
Apache Spark selected for Infoworld 2015 Technology of the Year Award
Recently Infoworld unveiled the 2015 Technology of the Year Award winners, which range from open source software to stellar consumer technologies like the...
An introduction to JSON support in Spark SQL
Note: Starting Spark 1.3, SchemaRDD will be renamed to DataFrame. In this blog post, we introduce Spark SQL’s JSON support, a feature we have been working on at Databricks to make it dramatically easier to query and create JSON data in Spark. With the prevalence of web and mobile applications, JSON has become the de-facto interchange format for web service API’s as well as long-term storage. With existing tools, users often engineer complex pipelines to read
Introducing Streaming k-means in Apache Spark 1.2
Many real world data are acquired sequentially over time, whether messages from social media users, time series from wearable sensors, or — in...