Big Book of MLOps Updated for Generative AI
Last year, we published the Big Book of MLOps, outlining guiding principles, design considerations, and reference architectures for Machine Learning Operations (MLOps). Since...
Llama 2 Foundation Models Available in Databricks Lakehouse AI
We’re excited to announce that Meta AI’s Llama 2 foundation chat models are available in the Databricks Marketplace for you to fine-tune and...
A Pattern for the Lightweight Deployment of Distributed XGBoost and LightGBM Models
A common challenge data scientists encounter when developing machine learning solutions is training a model on a dataset that is too large to...
Announcing the Public Preview of AI/BI Dashboards!
We are excited to announce the public preview of the next generation of Databricks SQL dashboards, dubbed AI/BI Dashboards . Available today, this...
Deploy Private LLMs using Databricks Model Serving
We are excited to announce public preview of GPU and LLM optimization support for Databricks Model Serving! With this launch, you can deploy...
Making Spark Accessible: My Databricks Summer Internship
My summer internship on the PySpark team was a whirlwind of exciting events. The PySpark team develops the Python APIs of the open...
Apache Spark ❤️ Apache DataSketches: New Sketch-Based Approximate Distinct Counting
Introduction In this blog post, we'll explore a set of advanced SQL functions available within Apache Spark that leverage the HyperLogLog algorithm, enabling...
Introducing the Support of Lateral Column Alias
We are thrilled to introduce the support of a new SQL feature in Apache Spark and Databricks: Lateral Column Alias (LCA). This feature...
Introducing Apache Spark™ 3.5
Today, we are happy to announce the availability of Apache Spark™ 3.5 on Databricks as part of Databricks Runtime 14.0. We extend our...
Introducing MLflow 2.7 with new LLMOps capabilities
As part of MLflow 2’s support for LLMOps, we are excited to introduce the latest updates to support prompt engineering in MLflow 2.7...