Managed MLflow on Databricks now in public preview
Building production machine learning applications is challenging because there is no standard way to record experiments, ensure reproducible runs, and manage and deploy models. To address these challenges, last June we introduced MLflow, an open source platform to manage the ML lifecycle that works with any machine learning library and environment. The project has grown...
Kicking Off 2019 with an MLflow User Survey
It’s been six months since we launched MLflow, an open source platform to manage the machine learning lifecycle, and the project has been moving quickly since then. MLflow fills a role that hasn’t been served well in the open source community so far: managing the development lifecycle for ML, including tracking experiments and metrics, building...
MLflow On-Demand Webinar and FAQ Now Available!
On August 30th, our team hosted a live webinar—Introducing MLflow: Infrastructure for a complete Machine Learning lifecycle—with Matei Zaharia, Co-Founder and Chief Technologist at Databricks. In this webinar, we walked you through MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library...
MLflow 0.2 Released
At this year’s Spark+AI Summit, we introduced MLflow, an open source platform to simplify the machine learning lifecycle. In the 3 weeks since the release, we’ve already seen a lot of interest from data scientists and engineers in using and contributing to MLflow. MLFlow’s GitHub repository already has 180 forks, and over a dozen contributors...
Introducing MLflow: an Open Source Machine Learning Platform
View the MLflow Spark+AI Summit keynote Everyone who has tried to do machine learning development knows that it is complex. Beyond the usual concerns in the software development, machine learning (ML) development comes with multiple new challenges. At Databricks, we work with hundreds of companies using ML, and we have repeatedly heard the same concerns:...
Matei Zaharia’s 5 predictions about big data and AI in 2018
Over the past few years, the demand for artificial intelligence (AI) and machine learning capabilities has surged with innovations in natural language processing, task automation, and predictions. From autonomous cars to a more personalized shopping experience, big data and artificial intelligence is at the forefront of new solutions that are delighting customers, improving business operations...
Spark Summit is Becoming the Spark + AI Summit
We’re excited to announce that Spark Summit is expanding its coverage in 2018 to include in-depth content on artificial intelligence. We are also renaming the conference Spark + AI Summit. AI has always been one of the most exciting applications of big data and Apache Spark, so with this change, we are planning to bring...
A Technical Overview of Azure Databricks
This is a joint blog post from Matei Zaharia, Chief Technologist at Databricks and Peter Carlin, Distinguished Engineer at Microsoft. Today at Microsoft Connect(); we introduced Azure Databricks, an exciting new service in preview that brings together the best of the Apache Spark analytics platform and Azure cloud. As a close partnership between Databricks and...
Databricks Delta: A Unified Data Management System for Real-time Big Data
Combining the best of data warehouses, data lakes and streaming For an in-depth look and demo, join the webinar. Today we are proud to introduce Databricks Delta, a unified data management system to simplify large-scale data management. Currently, organizations build their big data architectures using a mix of systems, including data warehouses, data lakes and...
Sharing Knowledge with the Community in a Preview of Apache Spark: The Definitive Guide
Apache Spark has seen immense growth over the past several years. The size and scale of this Spark Summit is a true reflection of innovation after innovation that has made itself into the Apache Spark project. Hundreds of contributors working collectively have made Spark an amazing piece of the technology powering thousands of organizations, and...
