Introducing Databricks Unity Catalog: Fine-grained Governance for Data and AI on the Lakehouse
Data lake systems such as S3, ADLS, and GCS store the majority of data in today’s enterprises thanks to their scalability, low cost, and open interfaces. Over time, these systems have also become an attractive place to process data thanks to lakehouse technologies such as Delta Lake that enable ACID transactions and fast queries. However,...
Introducing Delta Sharing: An Open Protocol for Secure Data Sharing
Data sharing has become critical in the modern economy as enterprises look to securely exchange data with their customers, suppliers and partners. For example, a retailer may want to publish sales data to its suppliers in real time, or a supplier may want to share real-time inventory. But so far, data sharing has been severely...
New Features to Accelerate the Path to Production With the Next Generation Data Science Workspace
Today, at the Data + AI Summit Europe 2020, we shared some exciting updates on the next generation Data Science Workspace - a collaborative environment for modern data teams - originally unveiled at Spark + AI Summit 2020. The power of data and artificial intelligence is already disrupting many industries, yet we've only scratched the...
A Guide to MLflow Talks at Data + AI Summit Europe 2020
In the last two years since its release, MLflow has seen a rapid adoption among enterprises and the data science community. With over 2M downloads, 260 contributors, and 100+ organizations contributing, the momentum seems to grow each year. We have put together a short list of picks–keynotes, tutorials, and sessions–for MLflow: how the community and...
Introducing the Next-Generation Data Science Workspace
At today’s Spark + AI Summit 2020, we unveiled the next generation of the Databricks Data Science Workspace: An open and unified experience for modern data teams. Existing solutions make data teams choose from three bad options. Giving data scientists the freedom to use any open-source tools on their laptops doesn’t provide a clear path...
MLflow Joins the Linux Foundation to Become the Open Standard for Machine Learning Platforms
Watch Spark + AI Summit Keynotes here At today's Spark + AI Summit 2020, we announced that MLflow is becoming a Linux Foundation project. Two years ago, we launched MLflow, an open source machine learning platform to let teams reliably build and productionize ML applications. Since then, we have been humbled and excited by the...
A Guide to MLflow Talks at Spark + AI Summit 2020
It's been 2 years since we originally launched MLflow, an open source platform for the full machine learning lifecycle, and we are thrilled and humbled by the adoption and impact it has gained in the data science and data engineering community. With now over 2M+ monthly downloads, 200 code contributors and over a 100 contributing...
Developing Shiny Applications in Databricks
Join our live webinar hosted by Data Science Central on March 12 to learn more We are excited to announce that you can now develop and test Shiny applications in Databricks! Inside the RStudio Server hosted on Databricks clusters, you can now import the Shiny package and interactively develop Shiny applications. Once completed, you can...
Databricks Named A Leader in Gartner Magic Quadrant for Data Science and Machine Learning Platforms
Gartner has released its 2020 Data Science and Machine Learning Platforms Magic Quadrant, and we are excited to announce that Databricks has been recognized as a Leader. Gartner evaluated 17 vendors for their completeness of vision and ability to execute. We are confident the following attributes contributed to the company's success: Our unique ability to...
Deep Learning Tutorial Demonstrates How to Simplify Distributed Deep Learning Model Inference Using Delta Lake and Apache Spark™
On October 10th, our team hosted a live webinar—Simple Distributed Deep Learning Model Inference—with Xiangrui Meng, Software Engineer at Databricks. Model inference, unlike model training, is usually embarrassingly parallel and hence simple to distribute. However, in practice, complex data scenarios and compute infrastructure often make this "simple" task hard to do from data source to...
