Connecting the global investment ecosystem

The challenge of scaling data for $7 trillion in assets

Addepar is a global technology and data company connecting the investment ecosystem. By delivering transparency and data-driven insights across the entire investment management lifecycle, Addepar empowers investment professionals to provide the most informed, precise guidance for their clients. In just four years, Addepar experienced exponential growth, with managed assets on the platform surging from $2 trillion to $7 trillion. However, this rapid expansion exposed significant limitations in their infrastructure. Built on custom-developed technology, these systems struggled to handle the sheer volume and complexity of data. Moreover, fragmented and siloed datasets — spanning market and portfolio information — created barriers to seamless integration, making it more challenging to deliver the timely, holistic insights their clients rely on. These challenges became even more pronounced as Addepar began scaling to meet the demands of enterprise clients.

As Addepar grew to onboard some of the world’s largest financial institutions, the complexities of managing massive datasets — measured in millions of positions and decades of historical transactions — became increasingly daunting. As enterprise clients onboarded, they significantly expanded the volume of data on the platform — essentially doubling it. This rapid growth presented an opportunity to streamline data integration and enhance connectivity, ensuring a more unified and actionable data ecosystem. Diverse datasets such as client portfolios, market data feeds and custodial feeds were living in separate systems and formats. This fragmentation made it difficult to gain a unified view for actionable insights, particularly given the array of tools that spanned different business intelligence (BI) and visualization stacks. Analysts and engineers struggled to collaborate across these siloed systems, with development workflows for ingesting, transforming and analyzing data taking up to 16 weeks. 

These delays had ripple effects, influencing time to insight and innovation for Addepar’s clients. Clients often required weeks of data onboarding and validation. Jeff Jiang, Director of Data Platform, Engineering, at Addepar, noted, “The biggest pain point wasn’t just data volume — it was also the complexity and variety of the datasets. Mapping these disparate sources into something actionable at scale felt like an impossible challenge without rethinking our platform entirely.”

The Addepar team knew they needed to modernize and reinvent their data infrastructure, shifting from homegrown systems to a unified and scalable data platform. Their goals included improving client onboarding, streamlining data ingestion and integration (e.g., portfolio and market data feeds), centralizing data for large-scale analysis and enabling the creation of new systems and products.

Transforming data silos into scalable solutions

Addepar adopted the Databricks Data Intelligence Platform to transform their data capabilities into a scalable, centralized ecosystem. Through lakehouse architecture, Addepar unified diverse datasets, enabling seamless collaboration across teams and transforming their ability to generate actionable insights. 

Addepar leveraged Delta Lake to bring their client portfolio data, market data and custodial feeds into a single, centralized platform. This eliminated the silos that previously fragmented their operations. Using Delta Sharing, Addepar securely shared data across global regions, enabling clients to comply with diverse regulatory requirements while supporting global operations. “With Databricks, we’ve established a unified data foundation that enables us to scale our platform globally and securely while continuously delivering value to our clients,” Bob Pisani, CTO at Addepar, said. 

Addepar also uses the platform’s interactive Notebooks for cross-functional collaboration. Engineers and financial analysts, who previously struggled to work together due to the complexity of siloed systems, now share a single shared workflow. Databricks Notebooks allow them to collaboratively ingest, analyze and model data in real time. “What previously took weeks to align between engineering and analytics can now be achieved in minutes thanks to Notebooks. This integration has changed how our teams work together,” Jeff noted.

Data pipeline development also saw remarkable improvements with the adoption of the Databricks Data Intelligence Platform. By automating key aspects of data ingestion and processing, Addepar reduced development times, which not only sped up client onboarding but also enabled the team to iterate on new products and data models faster than ever before.

The shift to a serverless architecture brought cost and efficiency gains, particularly in managing Addepar’s burst traffic during nightly data ingestion cycles. Traditional SQL warehouses previously struggled to scale dynamically and cost-effectively under such load. By utilizing serverless jobs, Addepar reduced operational complexity, cut EC2 costs and improved compute efficiency by nearly 2x. 

With sensitive financial data, governance was paramount. Unity Catalog provided Addepar with a centralized mechanism to manage data access, auditing and security. Advanced features like column masking and dynamic views ensured that sensitive client information was accessible only to authorized personnel. This governance model supported regulatory compliance globally, bolstered by auditability and seamless integration with other tools.

Additionally, Databricks’ AI and ML tools drove innovation at Addepar. Mosaic AI and fine-tuned models played a key role in the development of a new chatbot offering for clients and enhanced internal workflows. “The integration of these capabilities continues to elevate our operations, automating manual processes in real time,” Allen Yeong, Head of Architecture at Addepar, shared. Early exploration of features like Genie empowered the team to navigate Addepar’s data catalog — offering a novel approach to discovery when building new data feeds. “The flexibility Databricks gives us has redefined how our teams collaborate and innovate at scale,” Alec Solder, Data Architect at Addepar, explained.