Accelerating Machine Learning on Databricks: On-Demand Webinar and FAQ Now Available!
On January 15th, we hosted a live webinar—Accelerating Machine Learning on Databricks—with Adam Conway, VP of Product Management, Machine Learning, at Databricks and Hossein Falaki, Software Development Engineer and Data Scientist at Databricks. In this webinar, we covered some of the latest innovations brought into the Databricks Unified Analytics Platform for Machine Learning. In particular,...
Databricks Runtime 5.2 ML Features Multi-GPU Workflow, Pregel API, and Performant GraphFrames
We are excited to announce the release of Databricks Runtime 5.2 for Machine Learning. This release includes several new features and performance improvements to help developers easily use machine learning on the Databricks Unified Analytics Platform. Continuing our efforts to make developers’ lives easy to build deep learning applications, this release includes the following features...
New videos from Databricks Academy: Introduction to Machine Learning Series and the Apache Spark™ Cost-Based Optimizer
Databricks' commitment to education is at the center of the work we do. Through Instructor-Led Training, Certification, and Self-Paced Training, Databricks Academy provides strong pathways for users to learn Apache Spark and Databricks, and to push their knowledge to the next level. In that spirit, we are pleased to present some great new free content....
5 Reasons to Become an Apache Spark Expert
Apache Spark™ has fast become the most popular unified analytics engine for big data and machine learning. It was originally developed at UC Berkeley in 2009 by the team who later founded Databricks. Since its release, Apache Spark has seen rapid adoption. Today’s most cutting-edge companies such as Apple, Netflix, Facebook, and Uber have deployed Spark...
Apparate: Managing Libraries in Databricks with CI/CD
This is a guest blog from Hanna Torrence, Data Scientist at ShopRunner. Introduction As leveraging data becomes a more vital component of organizations' tech stacks, it becomes increasingly important for data teams to make use of software engineering best-practices. The Databricks platform provides excellent tools for exploratory Apache Spark workflows in notebooks as well as...
Kicking Off 2019 with an MLflow User Survey
It’s been six months since we launched MLflow, an open source platform to manage the machine learning lifecycle, and the project has been moving quickly since then. MLflow fills a role that hasn’t been served well in the open source community so far: managing the development lifecycle for ML, including tracking experiments and metrics, building...
Introducing Databricks Library Utilities for Notebooks
Databricks has introduced a new feature, Library Utilities for Notebooks, as part of Databricks Runtime version 5.1. It allows you to install and manage Python dependencies from within a notebook. This provides several important benefits: Install libraries when and where they're needed, from within a notebook. This eliminates the need to globally install libraries on...
Announcing Databricks Runtime 5.1
We are excited to announce the release of Databricks Runtime 5.1. This release includes several new features, performance and quality improvements. We highly recommend upgrading to Databricks Runtime 5.1 to benefit from all of these. Release highlights: Azure Active Directory Credential Passthrough to ADLS Gen1 (Preview) 7% performance gain overall for Databricks Azure New in...
Introducing Databricks Runtime 5.1 for Machine Learning
Last week, we released Databricks Runtime 5.1 Beta for Machine Learning. As part of our commitment to provide developers with the latest deep learning frameworks, this release includes the best of these libraries. In particular, our PyTorch addition makes it simple for a developer to simply import the appropriate Python torch modules and start coding,...
MLflow v0.8.1 Features Faster Experiment UI and Enhanced Python Model
MLflow v0.8.1 was released this week. It introduces several UI enhancements, including faster load times for thousands of runs and improved responsiveness when navigating runs with many metrics and parameters. Additionally, it expands support for evaluating Python models as Apache Spark UDFs and automatically captures model dependencies as Conda environments. Now available on PyPI and...
