Announcing General Availability of Lakehouse Federation for Google BigQuery and Public Preview for Teradata and Oracle

We’re excited to announce the General Availability of Lakehouse Federation for Google BigQuery and the Public Preview for Oracle and Teradata. Now, you can connect, discover, govern, and query data from these sources through Unity Catalog—without migration or ETL. This makes data access easier while ensuring an open, interoperable lakehouse architecture.

Unify your data and governance across distributed platforms with the Lakehouse Federation

Lakehouse Federation enables users to query and analyze data across disparate databases, data warehouses and catalogs without duplicating or transferring data. By integrating external data sources directly into the Unity Catalog, organizations can:

• Gain a unified view of your entire data estate: Automatically classify and discover both structured and unstructured data in a single platform, empowering your organization to securely access and explore all available data—wherever it resides.
• Query and analyze data seamlessly with a single engine: Enable fast, ad hoc analysis and prototyping across all your data, analytics, and AI workloads without the need for ingestion. A unified engine optimizes query performance with advanced query planning, caching, and cross-source execution, allowing you to efficiently combine data from multiple platforms in a single query.
• Ensure consistent data security and governance: Apply a unified permission model to enforce access rules across all data sources. Implement row- and column-level security, tag-based policies, and centralized auditing while maintaining full visibility into data usage. Meet compliance requirements effortlessly with built-in data lineage and auditability.

Lakehouse Federation supports a diverse set of platforms, including MySQL, PostgreSQL, Teradata, Oracle, Amazon Redshift, Salesforce Data Cloud, Snowflake, Microsoft SQL Server, Azure Synapse (SQL Data Warehouse), Google BigQuery, and Hive metastore, with more connectors coming soon.

General Availability of Lakehouse Federation for BigQuery

With the GA release of Lakehouse Federation for Google BigQuery, users can now connect BigQuery data to Databricks without the need for data movement or duplication. This enables them to run workloads directly on BigQuery data from their Databricks environment on GCP,  Azure or AWS, while leveraging Unity Catalog’s fine-grained governance for secure and efficient data management.

This integration allows for:

• Unified data management: Seamlessly discover and govern BigQuery data alongside other sources within Unity Catalog, ensuring consistent access control and governance.
• Optimized query performance: Utilize BigQuery’s compute power while benefiting from Databricks' advanced query optimization for faster insights.
• Improved collaboration: Empower teams to work across cloud environments efficiently with a unified interface for querying and analyzing data across multiple platforms.

Lakehouse Federation for BigQuery has seamlessly connected our Google Cloud and Azure environments. Without the need for data movement or duplication, we can now leverage Databricks on Azure to run Python and SQL workloads directly on BigQuery data, all while benefiting from Unity Catalog's fine-grained governance. It's a game-changer for our multicloud strategy, streamlining operations and enhancing analytics 

— Lisa Fiege, Data Science Engineer, Taylor Farms

The Google BigQuery connector is now generally available starting with DBR version 16.1 and will soon be available in Databricks SQL.

Public Preview for Teradata and Oracle Connectors

In addition to the GA announcement for BigQuery, Databricks is also introducing Teradata and Oracle connectors into Public Preview. These new connectors will allow users to extend the reach of Lakehouse Federation to even more data sources, further unifying the data estates.

Get Started

Read our documentation to get started with Lakehouse Federation connectors for Google BigQuery, Oracle and Teradata:

• Google BigQuery (AWS, Azure, GCP)
• Teradata (AWS, Azure, GCP)
• Oracle (AWS, Azure, GCP)