Building a Geospatial Lakehouse, Part 1
An open secret of geospatial data is that it contains priceless information on behavior, mobility, business activities, natural resources, points of interest and…
Data Engineering
Ray on Databricks
Ray is an open-source project first developed at RISELab that makes it simple to scale any compute-intensive Python workload. With a rich set…
10 Powerful Features to Simplify Semi-structured Data Management in the Databricks Lakehouse
Hassle Free Data IngestionDiscover how Databricks simplifies semi-structured data ingestion into Delta Lake with detailed use cases, a demo, and live Q&A. WATCH…
GPU-accelerated Sentiment Analysis Using Pytorch and Huggingface on Databricks
Sentiment analysis is commonly used to analyze the sentiment present within a body of text, which could range from a review, an email…
Introducing Apache Spark™ 3.2
We are excited to announce the availability of Apache Spark™ 3.2 on Databricks as part of Databricks Runtime 10.0. We want to thank…
Creating an IP Lookup Table of Activities in a SIEM Architecture
When working with cyber security data, one thing is for sure: there is no shortage of available data sources. If anything, there are…
Developing Databricks’ Runbot CI Solution
Runbot is a bespoke continuous integration (CI) solution developed specifically for Databricks’ needs. Originally developed in 2019, Runbot incrementally replaces our aging Jenkins…
How YipitData Extracts Insights From Alternative Data Using Delta Lake
This is a guest post from YipitData. We thank Anup Segu, Data Engineering Tech Lead, and Bobby Muldoon: Director of Data Engineering, at…
Real-time Point-of-Sale Analytics With a Data Lakehouse
Disruptions in the supply chain – from reduced product supply and diminished warehouse capacity – coupled with rapidly shifting consumer expectations for seamless…
How Incremental ETL Makes Life Simpler With Data Lakes
Incremental ETL (Extract, Transform and Load) in a conventional data warehouse has become commonplace with CDC (change data capture) sources, but scale, cost,…
