What’s new with Data Sharing & Collaboration

Databricks enables organizations to securely share data, AI models, and analytics across teams, partners, and platforms without duplication or vendor lock-in. With Delta Sharing, Databricks Marketplace, and Clean Rooms, businesses can collaborate in real-time while maintaining data privacy and governance.

Databricks continues to push the boundaries of data sharing and collaboration. In this blog, we’ll dive into:

• The General Availability of Databricks Clean Rooms
• New Delta Sharing features and how they improve secure data exchange
• The role of Delta Sharing in our SAP partnership
• Why we integrated Databricks Marketplace with Partner Connect and how the Marketplace ecosystem is expanding

Let’s review each of these updates in the rest of the blog.

Clean Rooms is now generally available for Azure & AWS

Databricks Clean Rooms is powered by Delta Sharing and allows businesses to easily collaborate with their customers and partners on any cloud without compromising privacy or sharing sensitive data. Leading companies such as Mastercard, Intuit and AppsFlyer have already begun using Databricks Clean Rooms for use cases like targeted advertising, fraud detection, lending processes, and clinical trial efficiency.

New capabilities include federated sharing across clouds, support for HIPAA compliance for healthcare use, management APIs for automation, and self-collaboration within single metastores. Read our GA announcement blog.

Driving growth in data sharing with Delta Sharing

We recently launched several new features, including Cross-Platform View Sharing, Secure Open Sharing with OpenID Connect, History sharing to boost table read performance, serverless egress controls and Lakehouse Federation Sharing. With these enhancements, Delta Sharing provides streamlined cross-platform data collaboration for multicloud ecosystems while enforcing strict security protocols.

Cross-Platform View Sharing

We recently launched Public Preview of Cross Platform View Sharing. View sharing has been useful; other vendors do it as well. But until now, it’s mostly been limited to the same platform. You could share views within one platform but not across multiple platforms and clouds. Previously, when a view was shared between Databricks accounts, consumers could query it using only Databricks SQL Serverless.

Databricks solves this problem with cross-platform view sharing and lets you share views seamlessly across different environments. Now, data consumers can leverage any type of Databricks cluster or even utilize open Delta Sharing clients to access and query shared views. This is a game changer because it expands data providers' reach and avoids vendor lock-in for data consumers, making collaboration easier and faster. Take a look at this demo so cross-platform view sharing in action.

Secure Open Sharing with OIDC Token Federation

Secure Open Sharing with OIDC Token Federation is soon going to be in a gated public preview. Open recipients can now authenticate through their preferred Identity Providers (IdPs) using OpenID Connect (OIDC) or OAuth tokens. This reduces exposure risks by eliminating the direct exchange of sensitive information when sharing with non-Databricks recipients (Databricks-to-Open Sharing).

Imagine you’re sharing a locked box of important documents with someone. Instead of giving them a physical key (which could get lost or stolen), you let them unlock the box using their own secure ID card from a trusted system, like their work badge.

This is similar to how Databricks now allows open recipients to use their own trusted Identity Providers (like Google or Microsoft) to securely access shared data, without needing to exchange sensitive keys or passwords.

Lakehouse Federation Sharing
What if you have data that resides outside of Databricks, such as databases, to share? Do you need to find a different sharing solution? Not needed.

Sharing for the Lakehouse Federation is the answer. Customers can now share data directly where it is stored across data platforms, including databases and data warehouses, such as Snowflake or Google BigQuery, without moving or copying the data. This differentiating feature will help customers eliminate costly ETL processes and ensure real-time access to data in its original non-Databricks location.

The Lakehouse Federation Sharing is now in Private Preview.

Faster table read performance with history sharing
Improve table read performance with history sharing is in public preview. This feature improves the performance of reading shared tables between Databricks workspaces when history sharing. It leverages a cloud token-based approach, which uses temporary security credentials from cloud storage to securely share the entire table directory, eliminating the need for pre-signed every file share. These credentials enable faster data retrieval, achieving performance levels comparable to directly accessing the source tables.

Serverless Egress Control
We have introduced Serverless Egress Controls for Delta Sharing, ensuring that recipients’ serverless environments remain secure and isolated from the open internet. This feature allows recipients to access only approved storage locations associated with Delta Shares, enhancing security and reducing the risks of unauthorized data access.

SAP Partnership Powered by Delta Sharing

With the newly announced SAP–Databricks partnership, “SAP Databricks” is now a native component of SAP Business Data Cloud, enabling two-way data sharing between SAP Databricks and enterprise environments via Delta Sharing. For organizations that wish to leverage their existing Databricks accounts for AI and analytics, SAP data can also be integrated using Delta Sharing.

At the core of this partnership, Delta Sharing provides seamless, secure data exchange between SAP Business Data Cloud and Databricks—whether it’s SAP Databricks or a customer’s current Databricks workspace. This approach reduces the risk of data inconsistencies by keeping SAP data in its original location (SAP Business Data Cloud) while allowing organizations to maximize the value of SAP investments by combining it with the rest of their enterprise data for advanced analytics and AI across the Databricks Platform.

Databricks Marketplace Momentum continues

The Databricks Marketplace is on a roll, continuing to grow at an impressive pace. It’s one of the fastest-growing data and AI marketplaces out there. We’re excited to announce the merging of Partner Connect, and the addition of Dun and Bradstreet’s extensive business data.

Partner Connect is now part of Marketplace UI

Partner Connect is now part of the Databricks Marketplace UI, streamlining how customers discover and access ecosystem offerings. All partner solutions, including third-party products and integrations, are now accessible through a single, centralized entry point in the main Databricks navigation menu. Say goodbye to navigating multiple interfaces—everything is now in one place.

Dun & Bradstreet 600+ million business records now available

What if you could access over 600 million business records—updated in near real-time—without the hassle of complex data transfers? Dun & Bradstreet’s rich dataset is now available on the Databricks Marketplace, offering seamless, secure, and scalable access to critical business intelligence.

This data set can help customers get insights like

• Foundational contact information on your most important relationships.
• Understand a company’s liquidity to prevent future risk.
• Identify cross-sell and up-sell opportunities within accounts.
• Understand the most important contacts across your account list.
• And understand the buying readiness of your account list.

“Reliable, trusted and up-to-date data is the backbone of informed decision making. The power of Dun & Bradstreet’s data sets and analytical insights and the openness, scalability and security of the Databricks Marketplace provide a strong foundation for organizations to put the power of data to work for them when and where needed to accelerate their business objectives.” 

— Ginny Gomez, President of Dun & Bradstreet, North America

Consider this hypothetical scenario of how a company could benefit from D&B data. ManuCorp, a global manufacturing company, faces significant supply chain risks due to factors such as financial instability among suppliers, geopolitical tensions, and environmental compliance issues. To address these challenges, the company turns to Dun & Bradstreet (D&B) datasets available through the Databricks Marketplace. By subscribing to these datasets, ManuCorp gains access to real-time information on supplier financial health, risk scores, and ESG ratings. These can readily made available in ManuCorp’s Unity Catalog via Delta Sharing.

With this data, ManuCorp conducts thorough risk assessments of its suppliers, identifying those with high default probabilities or subject to international sanctions. This enables proactive management of supplier relationships and risk mitigation. The company also leverages predictive analytics within Databricks to forecast potential disruptions from geopolitical events or natural disasters, allowing for optimized transportation routes and contingency planning.

The demo below shows how businesses can leverage D&B's tools to enhance their data with reliable business information and maintain up-to-date insights through continuous monitoring. Any updates are shared as "delta shares," providing real-time insights into changes for those businesses. This ensures continuous data accuracy and relevance for decision-making.

Check out Dun & Bradstreet on Databricks Marketplace

These are just the beginning—more innovations are on the way to enhance data sharing. Databricks will soon offer Materialized Views and Streaming Table Sharing, enabling seamless sharing of real-time data and precomputed query results for better performance and cost efficiency. Additionally, the Databricks Marketplace will introduce Databricks Apps and an AI Assistant to simplify data product discovery through conversational prompts.

We encourage you to learn more about Data Sharing and Collaboration. Check out the 3rd edition of A New Approach to Data Sharing