Clean energy leader delivers analytics solutions efficiently

Legacy Spark system struggles to process data from 60 million endpoints

As the power grid undergoes a radical transformation, energy providers and their customers are changing the way they think about energy — and about each other. Uplight seeks to understand the beliefs and behaviors of energy customers deeply so that they can enlist their help in reducing the carbon footprint. The company uses proprietary algorithms, AI, analysis and human curiosity to help energy providers create a more sustainable future by motivating the right actions in their customers. These actions vary from implementing consistent behavior change to purchasing energy-efficient devices and enrolling in grid-supportive programs.

“We enable energy customers to make informed choices about their energy-related actions and to ensure those actions align with positive grid outcomes,” Micaela Christopher, Director of Data Science at Uplight, explained.

As Uplight has grown, so has the volume of data it ingests and processes. The company now works with data from more than 60 million endpoints — ranging from smart thermostats and usage meters to electric vehicle (EV) chargers and solar arrays. Uplight’s data infrastructure eventually became a bottleneck.

“Our legacy Spark management system and siloed applications couldn’t keep pace with the increasing demand for our services,” Micaela recalled. “We needed a faster, smarter way of extracting meaningful information from the gigabytes of raw data we import daily. We also wanted to quickly introduce new, high-value energy insights for energy providers and their customers.”

Faster data processing slashes development time for new services

Hoping to derive greater insights from their data, Uplight implemented Databricks on Google Cloud. With this solution, procured through the Google Cloud Marketplace, Uplight ingests and analyzes data from utility systems, sociodemographic sources, OEM partners, weather stations and more. The company can now efficiently unlock actionable analytics. Uplight is also developing new services that help people adjust their energy use in ways that save money and reduce environmental impacts.

“We’re running 30,000 Spark jobs per month on Databricks to ingest, prepare and analyze our data,” Daniel Coll, Senior Data Engineer at Uplight, said. “We’re leveraging BigQuery, BigTable, Pub/Sub and Google Kubernetes Engine (GKE) for peak performance in our data platform. By using Databricks MLflow and Vertex AI, we’ve halved development time for new services and accelerated the rollout of new predictive machine learning (ML) models.”

Developed with Databricks MLflow and deployed with Vertex AI, Uplight’s predictive ML models analyze energy consumption patterns to pull out actionable insights targeted at utility customers or employees. This information helps utility companies perform functions meant to conserve energy and prevent grid overload. For example, companies can proactively alert customers of recommended specific actions such as enrolling in demand response programs.

Creating a greener, more sustainable future

Uplight’s advanced AI models and customer programs helped California utility companies avoid rolling power outages amid a dangerous heat wave in 2022. As temperatures topped 115 degrees, demand for electricity to cool homes and businesses approached an unprecedented 52,000MW. Utility companies called demand-response events and automatically adjusted smart thermostats to help manage this peak load. At scale, across multiple utilities and tens of thousands of homes, the cumulative effect of these small changes cut more than 1,200MW of energy from the grid.

Uplight’s demand-response and time-of-use programs include smart thermostats, water heaters, EV chargers and even EVs. The company has plans to expand to home battery packs and beyond. “We look forward to continuing to collaborate with energy providers to realize our vision of a smart, connected, interactive and decarbonized grid edge,” Micaela said.

Uplight continuously adds new applications and systems to their platform using Databricks on Google Cloud. Uplight’s data engineering team works closely with Databricks to expedite even the most complex implementations and completes most integrations in hours.

“Seamless integration between Databricks and the full stack of Google Cloud solutions and robust security has been central to our success,” Daniel remarked. “Because the data plane and clusters run within our Google Cloud account, we can easily manage identity permissions with BigQuery and quickly configure clusters to securely access protected network resources. This is a major advantage when it comes to networking and security.”

With Databricks on Google Cloud, Uplight cost-effectively scales to support new use cases and to launch additional services that help utility companies — and their customers — significantly reduce energy consumption and costs. As Micaela emphasized, energy providers should make it easy for people to understand how much energy they use while highlighting how they can better conserve and optimize this usage.

“Uplight empowers sustainable change with actionable energy analytics and programs that minimize carbon footprints,” Micaela concluded. “We’re proud to educate, engage and enable people so they can take a step back and say, ‘All right, I’ve got this. I'm doing what I need to do to create a greener future.’”