Today’s power grid traces its roots back to the late 1800’s when Pearl Street Station first serviced a handful of consumers. Over the last century, technological progress and societal changes have continually reshaped the electrical landscape. These changes have accelerated recently with the rapid growth of renewable generation, increased regulatory scrutiny, and more frequent extreme weather events. Many modern utility companies are struggling to adapt to these challenges.
Innovative utility companies use data to meet these demands. The most successful companies combine sophisticated sensors, modern data infrastructure, and artificial intelligence to increase efficiency, maintain reliability, and cut operational costs. For example, a high-performing electrical company could use an AI-based model to detect an unusual load pattern that identifies a transformer needing repair. As a result of this early detection, the company proactively repairs the transformer, avoids a costly outage, and provides more reliable service. Utility companies that can use data effectively are well-positioned to successfully navigate the demands of the modern utilities landscape.
Unfortunately, maximizing value from data in this way remains challenging. Data silos exist throughout organizations, with separate systems for customer records, operational historians, trading history, and a myriad of third-party standards. Different teams within a single company often gravitate toward different solutions, creating confusion and inefficiencies. In addition, utility companies remain rightly concerned about cybersecurity and intellectual property. These challenges can prevent companies from successfully capitalizing on their data.
Utility companies have a treasure trove of data siloed in operational systems that can provide unique opportunities to bolster investments in capital projects and lower operational costs. In a traditional technology landscape many of these systems are brought into a legacy data warehouse to reduce silos and enable cross functional reporting and analytics. An immediate problem that surfaces is that not all data can be consolidated. Some data types are not supported such as images and voice recordings. In other instances, the data sets are too large to bring the full history over or the dataset has too high of a velocity to bring at all.
Having cross functional data in a legacy warehouse enables reporting and some advanced analytics, but this creates some insurmountable problems. These legacy warehouses are not meant to directly support machine learning which is crucial for truly unlocking the potential in the data available. This forces initiatives that require advanced techniques to copy data from the warehouse to be used on other machines and technologies that can support this. Utilities are forced to operate their data projects in silos while struggling to unlock cross functional value.
Modern problems require a wide variety of data. This can be in the form of structured data, geospatial types, imagery, notes, and audio. In order to use these cross functionally to enrich each other, a platform that can interoperate on these different data types is required.
Data needed to solve problems is scattered throughout an organization. These critical pieces of information are stored in operational systems, warehouses, and file systems. This creates friction and roadblocks when trying to utilize disparate data which leads to long project timelines, and an inability to experiment, and ultimately failed projects.
In order to bring these siloed sources together, the data needs to be duplicated somewhere else so that they can be joined together. Without centralized governance and a platform that can handle multimodal data, projects will choose bespoke technologies that can solve their individual problems. This leads to data being copied multiple times to different systems and a fragmented model for governance. Not having a centralized way to govern data can lead to data loss, unauthorized access, and increased costs.
The Databricks Data Intelligence Platform helps utility companies solve these challenges. Databricks brings together data from inside and outside their organization in a standard format on a single platform. This is made easier by partnering with solution vendors like AVEVA and data providers like Accuweather to seamlessly enrich proprietary data with external sources. Additionally, Databricks has been designed from the beginning with a vision that every company should own its data and AI. Databricks has the governance and security in place to meet strict cybersecurity and IP protection requirements. Leaders in the industry rely on Databricks to get more from their data by democratizing data as an asset for the organization to use in order to solve complex problems.
The Data Intelligent Utility combines the latest advances in sensors, artificial intelligence, and data infrastructure with its own deep expertise in the industry to thrive in the face of modern challenges. This utility has merged data from its smart meters, trading systems, and operational historians into a single source of truth. Advanced analytics using these rich, proprietary datasets enable the use of accurate weather forecasting to accurately predict storm damage and ensure the right crews are in place to make repairs quickly. Maintenance teams use artificial intelligence fed by computer vision and live sensor data to prioritize urgent work rather than perpetuating broken schedule based maintenance systems. Generative AI trained on the utility’s own proprietary data ensures users have the information they need at their fingertips to more quickly help customers calling into the call center, or find information in manuals for field technicians to do complete installations or fixes. To realize this value, utilities need to break down data silos and have a single source of data that can interoperate together no matter if the data is structured or unstructured with the ability to govern it once.
The Data Intelligent Utility can redefine the industry standard for customer experience, operational efficiency, and environmental sustainability by maximizing the value of its data. Data from siloed systems can be brought together in a single source of truth on an open format that handles all different types of data. Having all of the sources in a single interoperable format allows for a single governance and discoverability model which eliminates the need to duplicate data and the potential for data loss. With this foundation in place, different business units can experiment more effectively on problems that plague their business without the traditional friction and roadblocks. This allows utility companies to bring value to the business rather than focussing on overcoming technical hurdles. Transforming into a Data Intelligent Utility reduces operational costs, eliminates silos and governance challenges and allows teams to solve business problems in a way that was not possible before. Leveraging Databricks as the foundation for data and AI turns the vision of a Data Intelligent Utility into reality.
Visit the Databricks Energy Industry page to learn more.
