Data Engineering Light: Expanding Choice for Production Workloads

Adding a Lower Tier, Lower Price Option

Since its inception a guiding principle for Databricks has been the unification of data science and engineering helping to bring together silos and raising the level of collaboration and productivity among the analytics professionals. And our customers have been reaping the benefits of this to solve some of their most challenging problems. While our customers see a lot of value in our Unified Analytics Platform, we recognized the need for a lower-priced option suitable for running simple production jobs at scale.

We are proud today to announce the general availability, of Databricks Data Engineering Light (formerly known as Databricks Basic, priced currently at $0.07/DBU), a new addition within our Unified Analytics Platform, aimed at running production jobs on Databricks’ equivalent of open source Apache Spark™.  Using Data Engineering Light is easy - you simply select the “Light” cluster runtime version when scheduling your job.

Data Engineering Light provides a runtime option for jobs that don’t need the advanced performance, reliability, or autoscaling benefits provided by the more capable Databricks Data Engineering offering. You can select Databricks Light only when you create a cluster to run a JAR job, spark-submit job, or Python job (via API); you cannot select this option for clusters on which you run interactive or notebook job workloads.

Beyond the capabilities offered in Databricks Data Engineering Light, the Unified Analytics Platform offers other more advanced offerings with capabilities such as:

• Databricks Delta Lake - for adding data reliability and performance to data lakes
• Runtime for ML - for working with ML frameworks of your choice
• Workspace - enabling highly effective collaboration among data professionals
• Interactive clusters - supporting high concurrency use for multiple users
• Ecosystem integrations - for being able to work with a range of third-party BI tools

Customers now have increased choice and flexibility and improved cost-effectiveness when it comes to running their analytics production workloads on Databricks’ platform.

If you are not already using Databricks, I invite you to try Databricks today.