Engineering Blog | Databricks Blog

Page 31

Learn How Disney+ Built Their Streaming Data Analytics Platform With Databricks and AWS to Improve the Customer Experience

December 14, 2020 by Hector Leano in Data Streaming

https://youtu.be/WAOrqsHpJuM Martin Zapletal, Software Engineering Director at Disney+, is presenting at re:Invent 2020 with the session "How Disney+ uses fast data ubiquity to...

ACID Transactions on Data Lakes Tech Talks: Getting Started with Delta Lake

November 23, 2020 by Ryan Boyd in Platform Blog

Get an early preview of O'Reilly's new ebook for the step-by-step guidance you need to start using Delta Lake. As part of our...

Delta vs. Lambda: Why Simplicity Trumps Complexity for Data Pipelines

November 20, 2020 by Hector Leano in Company Blog

“Everything should be as simple as it can be, but not simpler” - Albert Einstein Generally, a simple data architecture is preferable to...

MLflow Model Registry on Databricks Simplifies MLOps With CI/CD Features

November 19, 2020 by Sue Ann Hong, Ankit Mathur, Jules Damji and Mani Parkhe in Engineering Blog

MLflow helps organizations manage the ML lifecycle through the ability to track experiment metrics, parameters, and artifacts, as well as deploy models to...

How to Train XGBoost With Spark

November 16, 2020 by Stephen Offer in Data Science and ML

XGBoost is currently one of the most popular machine learning libraries and distributed training is becoming more frequently required to accommodate the rapidly...

MLflow 1.12 Features Extended PyTorch Integration

November 13, 2020 by Jules Damji, Siddharth Murching and Harutaka Kawamura in Engineering Blog

MLflow 1.12 features include extended PyTorch integration, SHAP model explainability, autologging MLflow entities for supported model flavors , and a number of UI...

Announcing the Launch of Databricks SQL

November 12, 2020 by Joel Minnick in Announcements

Databricks SQL is now generally available on AWS and Azure. Today, we announced the new Databricks SQL service to provide Databricks customers with...

Leveraging ESG Data to Operationalize Sustainability

November 11, 2020 by Antoine Amend in Engineering Blog

The benefits of Environmental, Social and Governance (ESG) are well understood across the financial services industry. In our previous blog post , we...

Improving the Spark Exclusion Mechanism in Databricks

November 6, 2020 by Tianhan Hu, Xingbo Jiang and Xiao Li in Engineering Blog

Ed Note: This article contains references to the term blacklist, a term that the Spark community is actively working to remove from Spark...

Quickly Deploy, Test, and Manage ML Models as REST Endpoints with MLflow Model Serving on Databricks

November 2, 2020 by Aaron Davidson, Tomas Nykodym and Clemens Mewald in Data Science and ML

MLflow Model Registry now provides turnkey model serving for dashboarding and real-time inference, including code snippets for tests, controls, and automation. MLflow Model...