Announcing public preview of Databricks Asset Bundles: Apply software development best practices with ease

We are delighted to announce that Databricks Asset Bundles are now in public preview. Bundles, for short, facilitate the adoption of software engineering best practices, including source control, code review, testing and continuous integration and delivery (CI/CD). Bundles enable data engineers, data scientists and ML engineers to express data, analytics and AI projects as source files. These source files provide an end-to-end definition of a project, including how it should be tested and deployed to the Lakehouse. This definition can easily be edited, tested and deployed.

Automate testing and deployment

CI/CD is essential in modern software development, helping to automate tests and deployments, thus speeding up release cycles and reducing errors. You can configure bundles to describe how to deploy and test projects. Bundles can automatically deploy across multiple workspaces, regions and clouds. You can use your existing CI/CD system to automate testing and deployment of bundles.

Develop, test, and deploy with IDEs

Bundles are typically used with a local IDE and work with the Databricks extension for VS Code - you can configure your IDE for bundle configuration auto-completion so it is easier to author your bundles. You can iteratively develop using a personal copy without affecting collaborators and execute resources such as jobs or DLT pipelines before they are deployed to production.

Standardize development with templates

Use default templates or create your own custom templates to set dev, staging and prod workspaces, permissions, default libraries, service principals, default Apache Spark configs and more. You can use bundles to set these configurations once and reuse them to streamline developing and deploying code.

Get started

Begin using Databricks Asset Bundles in only a few short steps:

1. Install the latest CLI
   When using Homebrew: brew tap databricks/tap; brew install databricks
   Note: We are adding support for other software package managers soon. See our detailed instructions for installation without Homebrew.
2. Authenticate to Databricks
   databricks configure
3. Generate and customize your first bundle
   databricks bundle init
   This generates a bundle with a README that works out of the box. You can customize the default configuration by updating any defaults you like.
4. Deploy your project to your development workspace
   databricks bundle deploy

How do Databricks Asset Bundles relate to Terraform and dbx?

Bundles are recommended for applying CI/CD to developing data, analytics and AI projects. Bundles and the Databricks Terraform provider work well together: bundles can be used to define lakehouse assets, while Terraform can be used for infrastructure, such as workspaces, service principals, and cloud assets. The Databricks Labs project dbx has been in an "experimental" release state since its launch, and we encourage migrating to bundles instead.

What's next

Databricks Asset Bundles are now in public preview, which means they are ready for production usage. In the coming months, you can look forward to additional features such as support for including dashboards as source files. Bundles already work with Databricks Workflows jobs, Delta Live Tables pipelines, ML Experiments, ML Registered Models and Model Serving endpoints. ML Ops Stack (currently in private preview) also uses bundle templates for productionizing your ML Projects. Learn more about bundles in our docs pages and let us know your feedback.