Reuse Existing Workflows Through Terraform
Part 2 of the blog series on deploying Workflows through Terraform. Here we focus on how to convert inherited Workflows into Terraform IaC...
Building a Cybersecurity Lakehouse for CrowdStrike Falcon Events Part III
In Part I of this series, we walked through the process of setting up a Cybersecurity Lakehouse that allowed us to collect and...
Streaming in Production: Collected Best Practices
Releasing any data pipeline or application into a production state requires planning, testing, monitoring, and maintenance. Streaming pipelines are no different in this...
Best Practices for Super Powering Your dbt Project on Databricks
dbt is a data transformation framework that enables data teams to collaboratively model, test and document data in data warehouses . Getting started...
An Automated Guide to Distributed and Decentralized Management of Unity Catalog
Unity Catalog provides a unified governance solution for all data and AI assets in your lakehouse on any cloud. As customers adopt Unity...
Why We Migrated From Apache Airflow to Databricks Workflows at YipitData
This is a collaborative post from Databricks and YipitData. We thank Engineering Manager Hillevi Crognale at YipitData for her contributions. YipitData is the...
ArcGIS GeoAnalytics Engine in Databricks
This is a collaborative post from Esri and Databricks. We thank Senior Solution Engineer Arif Masrur, Ph.D. at Esri for his contributions. Advances...
Intermittent Demand Forecasting With Nixtla on Databricks
This is a collaborative post from Nixtla and Databricks. We thank Max Mergenthaler Canseco, CEO and Co-founder, and Federico Garza Ramírez, CTO and...
Databricks Workflows Through Terraform
Part 1 of the blog series on deploying Workflows through Terraform. How to create complex jobs / workflows from scratch in Databricks using...
Near-real-time IoT Robust Anomaly Detection Framework
1. Introduction With rapid digitalization of the economy, we experience exponential growth of real-time data being processed by businesses and organizations worldwide. Lots...