customer story
Delivering innovative energy solutions for a cleaner world

Read story Watch Video

 

Shell optimizes trillion rows of IoT sensor data by simplifying reliable ETL and Data Lake with Data Live Tables. Learn more →

INDUSTRY: Energy

SOLUTION: Lakehouse

TECHNICAL USE CASE: Data ingest and ETL, data analytics, machine learning

Shell has been at the forefront of creating a cleaner tomorrow by investing in digital technologies to tackle climate change and become a net-zero emissions energy business. Across the business, they are turning to data and AI to improve operational efficiencies, drive customer engagement, and tap into new innovations like renewable energy. Hampered by large volumes of data, Shell chose Databricks to be one of the foundational components of its Shell.ai platform. Today, Databricks empowers hundreds of Shell’s engineers, scientists and analysts to innovate together as part of their ambition to deliver cleaner energy solutions more rapidly and efficiently.

The challenges of extracting insights at scale

Throughout its 100-plus-year history, Shell has generated pioneering ideas that have influenced the way we consume energy.

“We, as an industry, are going through a massive transition,” explained Dan Jeavons, GM of data science at Shell. “Digital technology is absolutely core to making our existing business more effective and efficient. As the industry continues to expand into new areas of energy that are more sustainable and reduce environmental impact, data and digital technology are now table stakes.”

While digital transformation is a primary initiative for every energy company, challenges remain with legacy technology infrastructure, the complexities of an exponential growth in data, and the lack of data engineering and science skills needed to build data-powered solutions.

Shell has met these challenges head-on by creating a Data Science Centre of Excellence (CoE), where teams continually work to identify the highest value use cases across the entire value chain. However, although they were identifying opportunities to innovate with data, Shell had the challenge to scale its data infrastructure for analytics, big data processing and machine learning.

Unifying data and AI across the enterprise to fuel innovation

Shell chose the Databricks Lakehouse Platform as one of the key tools within the Shell.ai Platform. Databricks provides Shell’s data team with a scalable, fully managed platform that unifies all their data, analytics and AI workloads. The interactive workspace has not only democratized access to data but has fostered cross-team collaboration across data engineering, data science and the analyst team.

“Shell has been undergoing a digital transformation as part of our ambition to deliver more and cleaner energy solutions. As part of this, we have been investing heavily in our data lake architecture. Our ambition has been to enable our data teams to rapidly query our massive datasets in the simplest possible way. The ability to execute rapid queries on petabyte scale datasets using standard BI tools is a game changer for us. Our co-innovation approach with Databricks has allowed us to influence the product roadmap and we are excited to see this come to market,” said Dan.

This low barrier of entry has opened up analytics beyond machine learning, including business intelligence and reporting. In fact, Shell’s focus on data and analytics has enabled over 250 data analysts (or citizen data scientists), and 800 citizen data scientists to be more productive with all the data available to them.

Transforming Shell into the energy company of the future

Shell’s CoE is now able to explore and deploy new data-driven solutions focused on improving supply chain operations as well as unlocking high-valued use cases that bring to life differentiated capabilities for their customers and their own businesses.

From an operations perspective, one of the biggest challenges any major industrial company faces is efficiently managing its inventory and supply chain. Shell stocks thousands of spare parts across its global facilities, and its inventory analysts were struggling to understand what level of spare parts they should hold in their warehouses. With Databricks, Shell was able to leverage its full historic data set to run 10,000+ inventory simulations across all its parts and facilities. Shell’s inventory prediction models now run in 45 minutes — down from 48 hours — significantly improving stocking practices and saving a lot of money annually.

Shell has also developed a recommendation engine for its new loyalty program called Go+ used by 1.5 million customers. Running on Azure and Databricks, the AI software can look at the full transaction history of a customer and use the information to tailor the offers and rewards to the preferences of the individual, combining their data with other aggregated data.

Data and AI have also unlocked new opportunities for Shell to engage with customers. Shell Remote Sense is a new initiative focused on optimizing the durability and performance of large-scale engines on ships and cruise liners. Shell processes over 750,000 lubricant samples per annum and delivers customer insights about lube oil quality and how it’s performing. This not only saves customers potentially millions of dollars in the cost of repair or engine downtime, but Shell also saves significantly on time and operational costs.

A data-driven culture that delivers results

Today Shell is redefining its boundaries of the oil and gas industry through data and AI. With Databricks as a key component of the Shell.ai platform, Shell is able to run data analytics and deploy machine learning models that improve operational efficiencies.

Using a common platform has empowered engineers, data scientists and analysts to be more agile, collaborative and data driven. Shell currently has over 160 AI projects running, and it’s only just getting started. In the coming years, Shell aims to make leaps in technological advancements powered by data and AI — from trillions of IoT sensors all generating data to 3-D printed equipment and parts that will disrupt the global supply chain and greatly reduce costs— and Databricks is a key part of the Shell.ai platform that will make this a reality.

  • Millions
    of dollars saved in potential engine repair costs
  • 250
    Data team members supporting 160+ high-value use cases
  • 9x
    Faster – 5 minutes to validate a label reduced from 45 minutes

The usage of Databricks over the years has broadened significantly. We started out using Databricks as a big data and AI platform but the scope has broadened. We have an entirely different class of citizen engineers and data scientists who are using it as a modern business intelligence tool to make smarter business decisions.”

– Daniel Jeavons, General Manager – Advanced Analytics CoE, Shell

Related Content


Technical Talk at Spark + AI Summit EU 2018