All posts

During the Data + AI Summit (fka Spark + AI Summit), Databricks CEO Ali Ghodsi recognized four exceptional data teams for how they have used data, machine learning, and AI to solve some of the world’s toughest problems. The second annual Databricks Data Team Awards brought a diverse set of submissions, representing many industries, use...

Today, we announced the launch of Databricks Machine Learning, the first enterprise ML solution that is data-native, collaborative, and supports the full ML lifecycle. This launch introduces a new purpose-built product surface in Databricks specifically for Machine Learning (ML) that brings together existing capabilities, such as managed MLflow, and introduces new components, such as AutoML...

Today, we announced Databricks AutoML, a tool that empowers data teams to quickly build and deploy machine learning models by automating the heavy lifting of preprocessing, feature engineering and model training/tuning. With this launch, data teams can select a dataset, configure training, and deploy models entirely through a UI. We also provide an advanced experience...

Today, we announced the launch of the Databricks Feature Store, the first of its kind that has been co-designed with Delta Lake and MLflow to accelerate ML deployments. It inherits all of the benefits from Delta Lake, most importantly: data stored in an open format, built-in versioning and automated lineage tracking to facilitate feature discovery....

As the amount of data, data sources and data types at organizations grow, building and maintaining reliable data pipelines has become a key enabler for analytics, data science and machine learning (ML). Prioritizing these initiatives puts increasing pressure on data engineering teams because processing the raw, messy data into clean, fresh, reliable data is a...

Data lake systems such as S3, ADLS, and GCS store the majority of data in today’s enterprises thanks to their scalability, low cost, and open interfaces. Over time, these systems have also become an attractive place to process data thanks to lakehouse technologies such as Delta Lake that enable ACID transactions and fast queries. However,...

Data sharing has become critical in the modern economy as enterprises look to securely exchange data with their customers, suppliers and partners. For example, a retailer may want to publish sales data to its suppliers in real time, or a supplier may want to share real-time inventory. But so far, data sharing has been severely...

This is a guest co-authored post. We thank Connor Gervin, partner engineering lead, KX, for his contributions. KX recently announced a partnership with Databricks making it possible to cover all the use cases for high-speed time-series data analytics. Today we’re going to explain the integration options available between both platforms for both streaming and batch...

The Databricks Partner Ecosystem of over 450 partners globally are critical to building and delivering the best data and AI solutions in the world. We are proud to collaborate with our partners and recognize that success of our joint customers is the result of mutual commitment and investments in training, solution development, and our ongoing...

This is a guest authored post by Nick Lee, partnership integration lead, at Labelbox Large data lakes typically house a combination of structured and unstructured data. Data teams often use Apache Spark™ to analyze structured data, but may struggle to apply the same analysis to unstructured, unlabeled data (specifically in the form of images, video,...