Announcing the Launch of Delta Live Tables on Google Cloud
Today, we are excited to announce the availability of Delta Live Tables (DLT) on Google Cloud. With this launch, enterprises can now use DLT to easily build and deploy SQL and Python pipelines and run ETL workloads directly on their lakehouse on Google Cloud. In order to power analytics, data science and machine learning, data...
Infrastructure Design for Real-time Machine Learning Inference
This is a guest authored post by Yu Chen, Senior Software Engineer, Headspace. Headspace’s core products are iOS, Android and web-based apps that focus on improving the health and happiness of its users through mindfulness, meditation, sleep, exercise and focus content. Machine learning (ML) models are core to our user experiences by offering recommendations...
How Incremental ETL Makes Life Simpler With Data Lakes
Incremental ETL (Extract, Transform and Load) in a conventional data warehouse has become commonplace with CDC (change data capture) sources, but scale, cost, accounting for state and the lack of machine learning access make it less than ideal. In contrast, incremental ETL in a data lake hasn’t been possible due to factors such as the...
Frequently Asked Questions About the Data Lakehouse
Question Index What is a Data Lakehouse? How is a Data Lakehouse different from a Data Warehouse? How is the Data Lakehouse different from a Data Lake? How easy is it for data analysts to use a Data Lakehouse? How do Data Lakehouse systems compare in performance and cost to data warehouses? What data governance...
Announcing Databricks Serverless SQL
Databricks SQL already provides a first-class user experience for BI and SQL directly on the data lake, and today, we are excited to announce another step in making data and AI simple with Databricks Serverless SQL. This new capability for Databricks SQL provides instant compute to users for their BI and SQL workloads, with minimal...
Announcing Databricks Autologging for Automated ML Experiment Tracking
Machine learning teams require the ability to reproduce and explain their results--whether for regulatory, debugging or other purposes. This means every production model must have a record of its lineage and performance characteristics. While some ML practitioners diligently version their source code, hyperparameters and performance metrics, others find it cumbersome or distracting from their rapid...
How to Manage End-to-end Deep Learning Pipelines with Databricks
Deep Learning (DL) models are being applied to use cases across all industries -- fraud detection in financial services, personalization in media, image recognition in healthcare and more. With this growing breadth of applications, using DL technology today has become much easier than just a few short years ago. Popular DL frameworks such as Tensorflow...
Improving On-Shelf Availability for Items With AI Out of Stock Modeling
This post was written in collaboration with Databricks partner Tredence. We thank Rich Williams, Vice President Data Engineering, and Morgan Seybert, Chief Business Officer, of Tredence for their contributions. Retailers are missing out on nearly $1 trillion in global sales because they don’t have on-hand what customers want to buy in their stores. Adding...
Solution Accelerator: Multi-touch Attribution
Behind the growth of every consumer-facing product is the acquisition and retention of an engaged user base. When it comes to customer acquisition, the goal is to attract high-quality users as cost effectively as possible. With marketing dollars dispersed across a wide array of different touchpoints -- campaigns, channels, and creatives -- measuring effectiveness is...
Make Your RStudio on Databricks More Durable and Resilient
One of the questions that we often hear from our customers these days is, “Should I develop my solution in Python or R?” There is no right or wrong answer to this question, as it largely depends on the available talent pool, functional requirements, availability of packages that fit the problem domain and many other...
