Introducing Glow: an open-source toolkit for large-scale genomic analysis
The key to solving some of today’s most challenging medical problems lies in the analysis of genomics data. Understanding the impact of the minor changes in an individual’s genome on their overall health is fundamentally a data driven challenge that requires integration across hundreds of thousands of individuals. By analyzing genomes across large cohorts, researchers...
Introducing the MLflow Model Registry
At today’s Spark + AI Summit in Amsterdam, we announced the availability of the MLflow Model Registry, a new component in the MLflow open source ML platform. Since we introduced MLflow at Spark+AI Summit 2018, the project has gained more than 140 contributors and 800,000 monthly downloads on PyPI, making MLflow one of the fastest...
Managed MLflow Now Available on Databricks Community Edition
In February 2016, we introduced Databricks Community Edition, a free edition for big data developers to learn and get started quickly with Apache Spark. Since then our commitment to foster a community of developers remains steadfast: to date, we have over 150K registered Community Edition users; we have trained thousands of people at meetups and...
Delta Lake Now Hosted by the Linux Foundation to Become the Open Standard for Data Lakes
At today’s Spark + AI Summit Europe in Amsterdam, we announced that Delta Lake is becoming a Linux Foundation project. Together with the community, the project aims to establish an open standard for managing large amounts of data in data lakes. The Apache 2.0 software license remains unchanged. Delta Lake focuses on improving the reliability...
A Guide to MLflow Talks at Spark + AI Summit 2019 Europe
We are thrilled to see how well MLflow has been welcomed by the community since we launched it last summer. With now over 800K monthly downloads, 130 code contributors and dozens of contributing organizations including RStudio and Microsoft, it is one of the fastest growing open source projects in the field of Machine Learning, confirming...
A Guide to Developer, Deep Dive, and Apache Spark Tutorial Talks at Spark + AI Summit, Europe
You might have heard the famous saying, “Why software is eating the world.” But if software is eating the world, you may ask, where does software come from? Naturally, Developers! Some software developers advocate that the “Developers are eating the world.” A research report by Stripe indicates that “developers have the ability to raise global...
Exciting Keynotes at Spark + AI Summit Europe 2019
Spark + AI Summit is the premier global event for the data and machine learning community to discuss the latest advances in open-source technologies such as Apache Spark™, Delta Lake, MLflow, Koalas and TensorFlow as well as best practices for deploying AI in the real world. In addition to over 100 exciting breakout sessions, this...
AutoML on Databricks: Augmenting Data Science from Data Prep to Operationalization
Thousands of data science jobs are going unfilled today as global demand for the talent greatly outstrips supply. Every day, businesses pay the price of the data scientist shortage in missed opportunities and slow innovation. For organizations to realize the full potential of machine learning, data teams have to build hundreds of predictive models a...
New cost savings option for Azure Databricks with DBU pre-purchase
The rapid adoption of Azure Databricks through our strategic partnership with Microsoft has been remarkable, and it’s proven to be a compelling service for our customers' big data, analytics and machine learning initiatives. To further help our customers save costs and improve budgeting for Azure Databricks, we are pleased to share a new pricing option...
Announcing the MLflow 1.1 Release
We’re excited to announce today the release of MLflow 1.1. In this release, we’ve focused on fleshing out the tracking component of MLflow and improving visualization components in the UI. Some of the major features include: Automatic logging from TensorFlow and Keras Parallel coordinate plots in the tracking UI Pandas DataFrame based search API Java...
