How Databricks’ Lakehouse is helping to power a new era for TD Bank Group's Data Transformation

At TD, we know the importance of safeguarding and utilizing trusted data across our organization. Data continues to be integral in helping make TD the "Better Bank," and we are focused on nurturing data to create meaningful insights and impact to help support our customers. Managing, processing, analyzing, and acting on this data has often been complex and time-consuming. In today's environment, real-time decisions and insights are required to keep up with the needs of our customers and ever-changing market demands.

Increasing regulatory pressures and customer demands are forcing all organizations to exert tighter control over the management and accuracy of their data. These days, increasing data availability and improving data quality represent two critical priorities for banks. There are over 100 countries that now have privacy or data protection laws. This will only increase in the coming years as more countries focus on the security and safety of their citizens' data. Our customers' trust in us is core to our innovation philosophy, no matter which set of technologies and capabilities we're exploring. Every day, the privacy of our customers' information and the importance of maintaining their trust while protecting the Bank's systems and keeping our information secure is a fundamental priority.

To change how we work, we have deployed a new operating model and structure across the Bank. The goal was to rethink how many of our colleagues work on a daily basis and how we could work more efficiently. This transformation at TD is called the "Next Evolution of Work" – or NEW. The driving focus is to deliver better and faster customer outcomes, while attracting and retaining top talent. From a data perspective, teams are looking to data to gain deeper insights across our customers' wants and needs while helping to enable safe, secure, and personalized human experiences.

Alongside this organizational transformation, TD chose to collaborate with Databricks and Microsoft to deliver a new high-performing Data Platform strategy (Lakehouse) in the cloud to help increase stability and security. Working with the benefits of a single data and analytics platform powered by Azure (Microsoft) and Databricks, TD established a single source of centralized data. This centralized source brought together all business lines, technological capabilities, and business assets under one agile roof – known as Data as a Service (DaaS).

Our objective for Data as a Service is clear: Provide faster access to secure and trusted data in the most cost-effective manner.

Organization

Through our NEW transformation, we brought the people who know, engineer, support and manage our data together. DaaS is over 2,000 colleagues strong and focused on delivering new data products and capabilities and helping to empower our colleagues. We are a cross-functional team of Data Engineers, Analysts, Product Owners, Data Architects, Agile Leaders, Risk Management Professionals and Domain experts. Training, upskilling and developing our colleagues across all dimensions of their career path is the overarching objective of all our teams.

Our New Organization

We are structured across Product Families where we manage and deliver Data Products and Capabilities across the enterprise.

• Defined scaled agile organization designed by capability and business function
• Integrated Product Owners, Data Engineers with Data Domain expertise across the model
• Standardized data platform, tooling, capabilities and processes
• Integrated Objectives, Key Results and outcomes focused on simplification, speed and cost reduction
• Single unified strategy, roadmap and path forward to lead our data transformation

Establishing the DaaS can help enable the transformation of our data at scale and across the enterprise. Databricks has provided a unified platform for our engineers, analysts and leaders to utilize modern tooling, technology and capabilities with a new pace, focus and drive. The Databricks unified platform underpins how our teams operate, deliver and enable the data foundations for TD.

Lakehouse Platform

We required a high-performing, unified, scalable, and secure platform to establish the data backbone for the enterprise. Additionally, as the adoption of Data and Artificial Intelligence (AI) develops and continues across the financial services industry, we believe this is a critical time to advance our capabilities while progressing forward. The establishment of Databricks Lakehouse can help prepare TD for the next generation of advancements, capabilities, and innovations, all dependent on highly secure and trusted data.

We have deployed Azure Databricks to move, manage, transform and process our data at scale. We needed to simplify our data platform architecture and accelerate time to market – all the while reducing both build and run costs. Incorporating Databricks, in combination with other native services on Azure, has helped us bring that vision to life.

Objectives

We are very focused on the outcomes we are creating as part of this transformation. The five main pillars are:

1. Faster Access to Data
   Data is stored, processed, and transformed in a single open format platform powered by Databricks and Delta Lake with the goal of reducing data processing and delivery times, lowering costs and improving the end-user experience.
2. Trusted Data
   Approved provisioning points based on Data Domains provide a single source of truth for analytics, reporting and insights, all on one unified platform. Data can then be understood, managed and trusted for business usage and adoption.
3. Modernized Platform
   A simplified architecture, backed by scalable operational frameworks, controls and monitoring, helps to simplify adoption, tooling and the need for integration.
4. Operational Excellence, Stability and Security
   Uptime, support and reliability of enterprise data pipelines have increased. Additionally, self-healing operations and processes are now standard for our environment.
5. Attract and Develop Top Talent
   Data Product Owners and Data Engineers collaborate on a unified platform leveraging modern delivery techniques, tools and technologies. New ways of working, coupled with modern technology help attract, develop, and recruit top talent.

Looking Ahead

Making better use of data, grounded in our values, is integral to our future at TD. We believe managing and delivering our data through a unified platform and dedicated organization at scale will help solidify our data.

Establishing a new, secure, stable, and trusted AI platform supported by Databricks has helped accelerate and simplify our data transformation, all while reducing costs and driving simplicity at scale.

This is a new era the financial services industry is entering. Without trusted, understood, and accessible data, leading-edge solutions will be unattainable. TD will continue to deliver data-driven personalized, connected, and human experiences for our customers and colleagues, all at an accelerated pace. We will continue to redefine the way we operate and the markets we serve – all powered by data at scale.

We continue to be excited by what promises to be one of the world's most influential times for data, analytics and AI.

We're looking forward to sharing more of our transformation story at this year's Data + AI Summit from June 26-29. You can also watch the on-demand replay if you cannot attend in-person.