Google Datastream Integration With Delta Lake for Change Data Capture
This is a collaborative post between the data teams as Badal, Google and Databricks. We thank Eugene Miretsky, Partner, and Steven Deutscher-Kobayashi, Senior...
Scaling SHAP Calculations With PySpark and Pandas UDF
Motivation With the proliferation of applications of Machine Learning (ML) and especially Deep Learning (DL) models in decision making, it is becoming more...
Streamline MLOps With MLflow Model Registry Webhooks
As machine learning becomes more widely adopted, businesses need to deploy models at speed and scale to achieve maximum value. Today, we are...
Make Your Data Lakehouse Run, Faster With Delta Lake 1.1
Delta Lake 1.1 improves performance for merge operations, adds the support for generated columns and improves nested field resolution With the tremendous contributions...
The Ubiquity of Delta Standalone: Java, Scala, Hive, Presto, Trino, Power BI, and More!
The Delta Standalone library is a single-node Java library that can be used to read from and write to Delta tables. Specifically, this...
Orchestrating Databricks Workloads on AWS With Managed Workflows for Apache Airflow
In this blog, we explore how to leverage Databricks’ powerful jobs API with Amazon Managed Apache Airflow (MWAA) and integrate with Cloudwatch to...
Investing in TickSmith: Enabling an E-Commerce Data Experience With Open Data Exchange
We are excited to announce Databricks Ventures’ investment in TickSmith, a leading SaaS platform that simplifies the online data shopping experience. The investment...
Creating a Faster TAR Extractor
Tarballs are used industry-wide for packaging and distributing files, and this is no different at Databricks. Every day we launch millions of VMs...
Building Data Applications on the Lakehouse With the Databricks SQL Connector for Python
We are excited to announce General Availability of the Databricks SQL Connector for Python . This follows the recent General Availability of Databricks...
Hunters and Databricks Ventures Partner for Advanced Security on the Lakehouse
Modern security teams must quickly detect, investigate and respond to threats to minimize their impact and better mitigate the risk to the organization...