Last-Mile Data Delivery: Fast, Federated, and Fully Compliant
Overview
Experience | In Person |
---|---|
Type | Breakout |
Track | Data and AI Governance |
Industry | Enterprise Technology, Public Sector, Financial Services |
Technologies | Delta Sharing, Unity Catalog, Databricks Apps |
Skill Level | Intermediate |
Duration | 40 min |
As organizations scale, evolving privacy requirements and decentralized team ownership often lead to fragmented data lakehouses—where data is produced and accessed across separate workspaces. Mission-critical workflows like AI model training or customer support triage frequently span datasets scattered across teams and systems, each with varying sensitivity and access controls. Engineers spend more time discovering and fetching data than delivering outcomes. They're slowed by increasing compliance hurdles—and yet, direct access still exposes organizations to significant risk. To solve this, we built a secure, scalable serving layer on Databricks that brings the right data to the right users at the right time, for just the right amount of time. Powered by Unity Catalog, Delta Sharing, and Databricks Apps, our solution ensures governed, efficient access across the lakehouse—without compromising speed, security, or user experience.
Session Speakers
Arpan Ghosh
/Engineering Manager
Databricks
Shuting Zhang
/software engineer
Databricks