Detecting Abuse at Scale: Locality Sensitive Hashing at Uber Engineering
This is a cross blog post effort between Databricks and Uber Engineering. Yun Ni is a software engineer on Uber’s Machine Learning Platform…
Intel’s BigDL on Databricks
Intel recently released its BigDL project for distributed deep learning on Apache Spark. BigDL has native Spark integration, allowing it to leverage Spark…
Deep Learning on Databricks
We are excited to announce the general availability of Graphic Processing Unit (GPU) and deep learning support on Databricks! This blog post will…
On Demand Webinar and FAQ: Apache Spark MLlib 2.x: Migrating ML Workloads to DataFrames
Last week, we held a live webinar, Apache Spark MLlib 2.x: Migrating ML Workloads to DataFrames, to demonstrate the ease with which you…
GPU Acceleration in Databricks
Databricks is adding support for Apache Spark clusters with Graphics Processing Units (GPUs), ready to accelerate Deep Learning workloads. With Spark deployments tuned…
Apache Spark 2.0 Preview: Machine Learning Model Persistence
Introduction Consider these Machine Learning (ML) use cases: A data scientist produces an ML model and hands it over to an engineering team…
Approximate Algorithms in Apache Spark: HyperLogLog and Quantiles
Introduction Apache Spark is fast, but applications such as preliminary data exploration need to be even faster and are willing to sacrifice some…
On-Time Flight Performance with GraphFrames for Apache Spark
Introduction Graph structures are a more intuitive approach to many classes of data problems. Whether traversing social networks, restaurant recommendations, or flight paths,…
Introducing GraphFrames
We would like to thank Ankur Dave from UC Berkeley AMPLab for his contribution to this blog post. Databricks is excited to announce the…
Auto-scaling scikit-learn with Apache Spark
Data scientists often spend hours or days tuning models to get the highest accuracy. This tuning typically involves running a large number of…
