Driving collaboration and innovation with Delta Sharing
Deutsche Börse leverages Databricks to optimize workflows and more effectively reach customers
Faster insights (on average) to extract insight from dataset vs. on-premises solution
Faster publishing (on average) to publish products on Marketplace vs. previous solution
Deutsche Börse Group creates trust in the markets of today and tomorrow. As an international exchange organization and innovative market infrastructure provider, Deutsche Börse ensures capital markets are transparent, reliable and stable. With their wide range of products, services and technologies, the Group organizes safe and efficient markets for sustainable economies — including Deutsche Börse’s leading exchanges like Eurex, EEX and Xetra. The organization deals with a huge amount of data, and delivering data products to market participants is essential to their business. However, fast data processing and efficiently governed data sharing are proving to be a constant challenge. Partnering with Databricks, Deutsche Börse is taking a significant step toward faster time to market, which is essential for ensuring future innovation potential.
Challenges sharing data across businesses and environments
Deutsche Börse’s data offering comprises a range of attractive and widely used data products, yet there are still a lot of untapped data pools and synergies to be explored across businesses. There were several challenges to overcome. Internal data processing and sharing was complex and time-consuming, as data mostly resided on-premises or in a hybrid cloud environment, making the exploration of AI and advanced analytics use cases more complicated. Data was siloed among various data warehouses and applications with individual taxonomies, requiring data engineers and data scientists to spend more time adjusting their workflows anytime a change was made to the underlying sources. To share and process data, the company often had to replicate it repeatedly — a challenge for data governance and ownership.
At the same time, Deutsche Börse is facing a growing appetite from market participants for providing new, high-quality, insightful data products and services to enable sophisticated investment and risk management decision-making. For Deutsche Börse, keeping up with the increasing demand in a scalable way posed a challenge due to limited technological capacity and protracted data product development processes. Data consumers also struggled with integrating and managing the various data sources they rely on, and they looked to Deutsche Börse, among others, to provide a better solution for them. “Opening up and listening to data consumers’ pain points and how to help them get the data where they want it is a high-priority item on our agenda,” Jan Stiebing, Head of Business Strategy and M&A for Market Data and Services at Deutsche Börse, said.
In addition, the financial industry is highly regulated, triggering the need to share data that is well governed, accurate and actionable in a secure multicloud environment that meets the strictest compliance requirements (e.g., GDPR and competition law).
Optimizing workflows and reaching customers where they are
Deutsche Börse turned to Databricks for a solution. Using the Databricks Data Intelligence Platform, the exchange can streamline data processing across multiple cloud environments (Google Cloud, Azure) and store it in one place within the lakehouse architecture, supported by modern data governance standards and meeting data protection requirements. Databricks Workflows allows seamless analysis and standardization of data, which is then streamed into tables, reports and business intelligence dashboards. This can be done in a few clicks, without the replication that can cause inconsistent taxonomies and governance headaches. And with this building block of their technology stack, Deutsche Börse isn’t locked into a single vendor, allowing the enterprise to take advantage of the flexibility offered by a multicloud environment. “We would need multiple solutions and services to replace this ecosystem,” Abed Shareef, Vice President of Big Data and Advanced Analytics at Deutsche Börse, said.
Deutsche Börse is also a data provider on Databricks Marketplace, the open marketplace powered by Delta Sharing. The Group provides data products directly to customers’ Databricks accounts, including sample notebooks with pre-built code to help data analysts and data scientists alike when thinking of how to use third-party data. This makes Deutsche Börse products easier for customers to discover and to access, helping solve their data integration pain points and ultimately speed up time to value. “We’re leveraging the ecosystem and direct reach of Databricks Marketplace to place our analytics solutions closer to where our customers would love to have it to empower insights and action,” Stiebing said.
This opens further opportunities for direct connection with and feedback from consumers, empowering Deutsche Börse to better understand and fulfill customers’ needs with new innovative solutions.
Faster time to market, future opportunities
Databricks provides Deutsche Börse’s data product development teams with a unified platform that extends access to more users and empowers them to collaborate with streamlined data sharing. Processes that used to take several days can now be completed within a few minutes. These time savings add up to faster time to insights, faster time to market and more-innovative products. “Our workflows are becoming more intelligent, integrated and agile, so we can process faster for enhanced insights and decision-making,” Shareef said.
For Deutsche Börse, Databricks capabilities are an important part of the Group’s target enterprise architecture — essential for the company’s Data Mesh plans as well as a springboard for innovation. The Group envisions other uses of Databricks technology, such as using data clean rooms and creating new opportunities for codeveloping groundbreaking new solutions with partners and customers. “Having a platform that allows secure data sharing with fine-grained access controls, the highest security standards and privacy assurance opens up new possibilities,” Stiebing said. “We can now engage in conversations on customized solutions, where in the past, we would have said, ‘Unfortunately, our clients don’t want to share their data and models with us, or we don’t want to share more-granular data or our models for confidentiality reasons.’”
Deutsche Börse will continue to work with Databricks as an important partner in achieving their goals. “We are relying on Databricks technology and cross-cloud data sharing to help us unlock the potential of data, analytics and AI for Deutsche Börse,” Stiebing said.
