Company Blog

The transaction log is key to understanding Delta Lake because it is the common thread that runs through many of its most important features, including ACID transactions, scalable metadata handling, time travel, and more. In this article, we’ll explore what the transaction log is, how it works at the file level, and how it offers...

Thousands of data science jobs are going unfilled today as global demand for the talent greatly outstrips supply. Every day, businesses pay the price of the data scientist shortage in missed opportunities and slow innovation. For organizations to realize the full potential of machine learning, data teams have to build hundreds of predictive models a...

This summer, I interned on the ML Platform team. I worked on MLflow, an open-source machine learning management framework. This blog post details the projects I worked on, and my experience at Databricks overall. The automatic logging feature I developed makes it easier for data scientists to track their training sessions, without having to change...

Introduction: The Problem Deep learning sometimes seems like sorcery. Its state-of-the-art applications are at times delightful and at times disturbing. The tools that achieve these results are, amazingly, mostly open source, and can work their magic on powerful hardware available to rent by the hour in the cloud. It's no wonder that companies are eager...

Try out this notebook series in Databricks - part 1 (Delta Lake), part 2 (Delta Lake + ML) For many data scientists, the process of building and tuning machine learning models is only a small portion of the work they do every day. The vast majority of their time is spent doing the less-than-glamorous (but...

On June 26th, we hosted a live webinar — Deep Learning on Medical Images at Population-scale— with members of the data science and engineering teams from Human Longevity Inc (HLI), a leader in medical imaging and genomics. During the webinar, HLI shared how they use MRI images, whole-genome sequencing data, and other clinical data sets...

At Databricks, we’re committed to learning and development at every level, so it’s important to our teams that we recruit and develop our next generation of Databricks leaders. Our interns are encouraged to live out one of our core values, “be an owner” and they play an integral role in developing our platform during their...

The rapid adoption of Azure Databricks through our strategic partnership with Microsoft has been remarkable, and it’s proven to be a compelling service for our customers' big data, analytics and machine learning initiatives. To further help our customers save costs and improve budgeting for Azure Databricks, we are pleased to share a new pricing option...

At Databricks, we are proud to celebrate diversity. We believe that the core values of Databricks are found in Bricksters of all backgrounds: Team work makes the dream work, Be customer obsessed, Let the data decide and Own it!   Team Databricks celebrating Pride 2019 Databricks was a proud sponsor of Amsterdam Pride 2019, one...

On July 18th, we hosted a live webinar —Using ML and Azure to improve customer lifetime value - with Rob Saker, Industry Leader - Retail Industry, Colby Ford,  Associate Faulty - School of Data Science, UNC Charlotte and Navin Albert,  Solutions Marketing Manager at Databricks. This blog has a recording of the webinar and some of...