Introducing Databricks AutoML: A Glass Box Approach to Automating Machine Learning Development
Today, we announced Databricks AutoML, a tool that empowers data teams to quickly build and deploy machine learning models by automating the heavy…
Announcing Single-Node Clusters on Databricks
Databricks is used by data teams to solve the world’s toughest problems. This can involve running large-scale data processing jobs to extract, transform,…
Announcing GPU-aware scheduling and enhanced deep learning capabilities
Databricks is pleased to announce the release of Databricks Runtime 7.0 for Machine Learning (Runtime 7.0 ML) which provides preconfigured GPU-aware scheduling and…
Simplify Data Conversion from Apache Spark to TensorFlow and PyTorch
Petastorm is a popular open-source library from Uber that enables single machine or distributed training and evaluation of deep learning models from datasets…
Deep Learning Tutorial Demonstrates How to Simplify Distributed Deep Learning Model Inference Using Delta Lake and Apache Spark™
On October 10th, our team hosted a live webinar—Simple Distributed Deep Learning Model Inference—with Xiangrui Meng, Software Engineer at Databricks. Model inference, unlike…
Introducing Databricks Runtime 5.1 for Machine Learning
Last week, we released Databricks Runtime 5.1 Beta for Machine Learning. As part of our commitment to provide developers with the latest deep…
Introducing HorovodRunner for Distributed Deep Learning Training
Today, we are excited to introduce HorovodRunner in our Databricks Runtime 5.0 ML! HorovodRunner provides a simple way to scale up your deep…
Access Control for Databricks Jobs
Secure your production workloads end-to-end with Databricks’ comprehensive access control system Databricks offers role-based access control for clusters and workspace to secure infrastructure…
Accelerating R Workflows on Databricks
At Databricks we strive to make our Unified Analytics Platform the best place to run big data analytics. For big data, Apache Spark…
Entropy-based Log Redaction for Apache Spark on Databricks
This blog post is part of our series of internal engineering blogs on Databricks platform, infrastructure management, tooling, monitoring, and provisioning. We love…
