Fueling net-zero innovations toward a sustainable future
UK Power Networks uses the Databricks Data Intelligence Platform to deliver sustainable energy to the masses
Total value created over the next 3 years
Annual value created due to reduced downtime
Providing reliable energy services to a quarter of the UK is core to UK Power Networks’ mission, which is also focused on supporting low-carbon technologies like electric car charging and heat pumps while cost-effectively helping the UK transition to net zero. Burdened with data challenges that drove up operational costs, UK Power Networks shifted to the Databricks Data Intelligence Platform, which equipped them with real-time power outage and voltage change data for quick and accurate decision-making. UK Power Networks not only reduced downtime for customers but also enabled general staff to conduct simple analysis while empowering a predictive machine learning (ML) engine to manage a smarter, data-driven electricity network.
Slow data processing and lack of self-service impede insights
As concerns over climate change continue to rise, the focus on achieving net-zero emissions and providing more sustainable energy solutions is paramount. At UK Power Networks, their priority is to help facilitate the UK net-zero carbon target as quickly as possible, in a cost-effective way for customers. To ensure incredible customer service — even to their most vulnerable customers — and have the ability to widely share data to empower sector innovation, UK Power Networks wanted to leverage smart digital methods such as real-time network monitoring.
Unfortunately, their on-premises solution was holding them back. Data resided across multiple servers to decentralize the storage costs and data-preparation efforts required for different data sources. “Processing real-time data with our existing technology stack often proved cumbersome, impeding timely business insights,” explained Matt Webb, Head of Enterprise Data Management at UK Power Networks.
The on-premises solution posed additional hurdles for UK Power Networks, including licensing issues and limited support from their data center provider. Access control complexities emerged as data volumes surged, requiring robust governance. Business units operated in silos, reliant on a central data warehouse managed by a single team, which inhibited cross-collaboration and self-service data provisioning. Externally, data sharing challenges further impeded the realization of value for innovative projects, hampering rapid experimentation and feasibility assessments.
“Our transition to Databricks was driven by the identification of critical datasets pivotal to multiple projects, initiatives and teams. Rather than adopting a use case–specific approach, we prioritized migrating essential data from our on-premises legacy data warehouse to the cloud,” shared Webb. “Databricks played a key role in the provisioning of real-time smart meter alerts for power outages and voltage changes. This data not only benefited various internal teams but also facilitated new forms of network monitoring, enhancing our understanding of constraints and faults.”
The Databricks Platform transforms real-time collaboration
UK Power Networks has rolled out a Network Load Forecasting and Estimation machine learning model. The Databricks Data Intelligence Platform helps all data teams, including business users like data scientists and analysts, to collaborate more effectively on this ML model and other engineering-heavy tools. “The Databricks Platform caters to the diverse requirements of all teams, encouraging even business data users to work within it,” said Webb. “My team can provide data to any other team in Databricks, where they can use the same tool to build their models and share the outputs. With our previous on-premises solution, this all would have been done in separate technologies.”
Using Databricks Unity Catalog to refine and simplify fine-grained access management, UK Power Networks is simplifying navigation of their data landscape. The data team leverages Delta Sharing to streamline data sharing across the organization, avoiding the silos that happen naturally with emailing or setting up one-off file shares. Webb further added, “Databricks offers a simpler unified access management system with a user-friendly, browser-based notebook interface. This facilitates collaboration between previously siloed teams, fostering seamless sharing of data both internally and externally. With Databricks, both data ingestion and analysis have become clearer and more efficient.”
Empowering innovation for their customers at the lowest possible costs
The improved network visibility provided by the Databricks Platform creates significant cost savings, enabling UK Power Networks to invest in innovative projects such as smart meter alerts and network load forecasting.
“The ML model helps us predict headroom on our network,” said Webb. “This increases the quality of the Network Planning teams’ work, saving them time and helping to avoid unnecessary reinforcement. It also helps encourage the storage and analysis of data in a single unified tool, creating a single source of truth. This avoids point solutions that take separate time, effort and investment to set up and maintain.”
The Databricks Data Intelligence Platform will provide value across UK Power Networks by reducing costs to service network equipment, minimizing customer interruptions from unplanned downtime, lowering costs to deliver network expansion plans and improving data literacy and business performance.
The ability to scale using the Databricks Platform without sacrificing performance has democratized access, allowing UK Power Networks to explore increasingly complex use cases such as predictive models and ML. Now they can innovate and employ smart solutions in managing their network in line with Distribution Service Operator (DSO) priorities to explore nontraditional methods for achieving a net-zero carbon target.
“We’ve modernized our processes with Databricks. We’ve improved the accessibility of quality data, simplified governance and promoted data ownership. All in one single solution,” proclaimed Webb.
