Cloud-based Relational Database Management Systems at Databricks
Databricks and Microsoft have jointly developed a new cloud service called Microsoft Azure Databricks, which makes Apache Spark analytics fast, easy, and collaborative on the Azure cloud. Not only does this new service allow data scientists and data engineers to be more productive and work collaboratively with their respective teams, but it also gives them...
Declarative Infrastructure with the Jsonnet Templating Language
This blog post is part of our series of internal engineering blogs on Databricks platform, infrastructure management, integration, tooling, monitoring, and provisioning. At Databricks engineering, we are avid fans of Kubernetes. Much of our platform infrastructure runs within Kubernetes, whether in AWS cloud or more regulated environments. However, we have found that Kubernetes alone is...
Next Generation Physical Planning in Apache Spark
Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway. — Andrew Tanenbaum, 1981 magine a cold, windy day at Lake Tahoe in California. When a group of Bricksters who were cross-country skiing stopped for lunch, one found a 4TB hard drive in their backpack. One brickster remarked that this...
