Session
The Future of DSv2 in Apache Spark™
Overview
| Experience | In Person |
|---|---|
| Type | Breakout |
| Track | Data Lakehouse Architecture and Implementation |
| Industry | Enterprise Technology |
| Technologies | Apache Spark |
| Skill Level | Advanced |
| Duration | 40 min |
DSv2, Spark's next-generation Catalog API, is gaining traction among data source developers. It shifts complexity to Apache Spark™, improves connector reliability and unlocks new functionality such as catalog federation, MERGE operations, storage-partitioned joins, aggregate pushdown, stored procedures and more. This session covers the design of DSv2, current strengths and gaps and its evolving roadmap. It's intended for Spark users and developers working with data sources, whether custom-built or off-the-shelf.
Session Speakers
IMAGE COMING SOON
Anton Okolnychyi
/Databricks