GPU Acceleration in DatabricksOctober 27, 2016 by Joseph Bradley, Tim Hunter and Yandong Mao in Engineering Blog Databricks is adding support for Apache Spark clusters with Graphics Processing Units (GPUs), ready to accelerate Deep Learning workloads. With Spark deployments tuned...
Databricks Bi-Weekly Digest: 8/8/16August 8, 2016 by Jules Damji in Engineering Blog Continuing with our bi-weekly digest series, here’s our recap of what’s transpired over the last two weeks with Apache Spark since our previous...
Apache Spark 2.0 Preview: Machine Learning Model PersistenceMay 31, 2016 by Joseph Bradley in Engineering Blog Introduction Consider these Machine Learning (ML) use cases: A data scientist produces an ML model and hands it over to an engineering team...
Genome Sequencing in a NutshellMay 24, 2016 by Deborah Siegel in Engineering Blog This is a guest post from Deborah Siegel from the Northwest Genome Center and the University of Washington with Denny Lee from Databricks...
Parallelizing Genome Variant AnalysisMay 24, 2016 by Deborah Siegel in Engineering Blog This is a guest post from Deborah Siegel from the Northwest Genome Center and the University of Washington with Denny Lee from Databricks...
Predicting Geographic Population using Genome Variants and K-MeansMay 24, 2016 by Deborah Siegel in Engineering Blog Spark Summit 2016 will be held in San Francisco on June 6–8. Check out the full agenda and get your ticket This is...
New Content in Databricks Community EditionApril 12, 2016 by Ion Stoica in Engineering Blog At the Spark Summit New York , we announced Databricks Community Edition (CE) beta. CE is a free version of the Databricks service...
The Unreasonable Effectiveness of Deep Learning on Apache SparkApril 1, 2016 by Miles Yucht and Reynold Xin in Engineering Blog Update: this post is an April Fools joke. It is not an actual project we're working on. For the past three years, our...
Auto-scaling scikit-learn with Apache SparkFebruary 8, 2016 by Tim Hunter and Joseph Bradley in Engineering Blog Data scientists often spend hours or days tuning models to get the highest accuracy. This tuning typically involves running a large number of...
Deep Learning with Apache Spark and TensorFlowJanuary 25, 2016 by Tim Hunter in Engineering Blog Neural networks have seen spectacular progress during the last few years and they are now the state of the art in image recognition...