$1.44 per terabyte: setting a new world record with Apache Spark
We are excited to share with you that a joint effort by Nanjing University, Alibaba Group, and Databricks set a new world record...
GPU Acceleration in Databricks
Databricks is adding support for Apache Spark clusters with Graphics Processing Units (GPUs), ready to accelerate Deep Learning workloads. With Spark deployments tuned...
Databricks Bi-Weekly Apache Spark Digest: 10/4/16
Here’s our recap of what’s transpired with Apache Spark since our previous digest . Databricks Apache Spark Survey 2016 Report published and now...
Voice from CERN: Apache Spark 2.0 Performance Improvements Investigated With Flame Graphs
This is a guest post from CERN, the European Organization for Nuclear Research. In this blog, Luca Canali of CERN investigates performance improvements...
Apache Spark @Scale: A 60 TB+ production use case from Facebook
This is a guest Apache Spark community blog from Facebook Engineering . In this technical blog, Facebook shares their usage of Apache Spark...
Databricks Bi-Weekly Digest: 8/31/16
Here’s our recap of what’s transpired with Apache Spark since our previous digest . Databricks CTO and Co-founder Matei Zaharia presented “Unifying big...
How to use SparkSession in Apache Spark 2.0
Generally, a session is an interaction between two or more entities. In computer parlance, its usage is prominent in the realm of networked...
Databricks Bi-Weekly Digest: 8/8/16
Continuing with our bi-weekly digest series, here’s our recap of what’s transpired over the last two weeks with Apache Spark since our previous...
Spark Structured Streaming
Apache Spark 2.0 adds the first version of a new higher-level API, Structured Streaming, for building continuous applications . The main goal is...
Continuous Applications: Evolving Streaming in Apache Spark 2.0
Since its release, Spark Streaming has become one of the most widely used distributed streaming engines, thanks to its high-level API and exactly-once...