What's new in Workflows?
Databricks Workflows is the cornerstone of the Databricks Data Intelligence Platform, serving as the orchestration engine that powers critical data and AI workloads...
How automated workflows are revolutionizing the manufacturing industry
Discover how Corning, Volvo, and Cox Automotive are modernizing their data operations with Databricks Workflows to gain efficiency, real-time insights, and automation
Summary
- How Corning, Volvo, and Cox Automotive are leveraging Databricks Workflows to streamline and automate complex data processes.
- Specific use cases, from improving inventory visibility to enhancing data quality and enabling real-time decision-making.
- The business value realized through Databricks Workflows, including operational efficiency, agility, and stronger customer insights.
For today’s manufacturers, streamlined and automated workflows are crucial for overcoming challenges such as manual data management and equipment downtime. By leveraging automated workflows and enabling predictive maintenance, manufacturers can gain real-time production insights that reduce inefficiencies and waste. The elimination of data silos and the ability to scale analytics empower better decision-making and support the growing volume of operational data. In a data-driven landscape, automated workflows have become essential for business success, allowing data practitioners to shift from reactive problem-solving to proactive innovation.
Databricks Workflows, the unified orchestration tool for data, analytics, and AI, can help meet the growing demands of data teams by easily defining, managing, and monitoring automated workflows for ETL, analytics, and machine learning pipelines. Fully integrated with the Data Intelligence Platform, Workflows offers a simple workflow definition experience, advanced observability capabilities and high reliability. Workflows provides a wide range of task types including notebooks, JAR files, Python scripts, Databricks SQL queries and dashboards, Delta Live Tables pipelines, dbt tasks and more. Workflows also features recent advancements in capabilities—from data-triggered table and file arrival automation to AI-powered cron syntax generation and serverless compute.
In this blog, we review how Databricks Workflows enabled manufacturing leaders like Corning, Volvo, and Cox Automotive to simplify their data pipelines, improve real-time insights, and reduce maintenance overhead. Using Workflows, their data teams have instead shifted their focus to innovation. The results include improved inventory visibility, increased operational efficiency, and better customer experiences, showcasing how Databricks Workflows can benefit the manufacturing sector.
Using Databricks Workflows, these businesses were able to automate the flow of data from IoT devices on the factory floor to predictive maintenance models, generate schedules and alerts with pinpoint accuracy, and scale analytics workloads to handle massive volumes of data from their operations. As a result, they had meaningful changes to their outputs — delivering optimized production schedules, predicting equipment failures before they happened, and having a real-time view of the supply chain to proactively manage disruptions to ensure products arrive on time. Let’s dive into a few manufacturing customer success stories.
Corning’s end-to-end data orchestration with Databricks Workflows drives operational efficiency
Corning has truly revolutionized its data management processes with Databricks Workflows. Corning is an innovator in materials science, relying on data to fuel innovations and patents. Corning engineers previously used Apache Airflow as their data orchestration tool but have fully migrated to Databricks Workflows. This shift enabled Corning to manage approximately 2,500 jobs (around 5 petabytes of data) and support 900 active global users. The platform’s automation capabilities streamlined data curation, broke down silos, and allowed data engineers and scientists to process vast amounts of data at scale while establishing repeatable, reusable workflows. These advancements enhanced operational efficiency and delivered faster insights crucial for innovation.
“Databricks Workflows plays a critical role in allowing us to repeatedly, on our own schedule, run a whole pipeline orchestration, with end-to-end data flow through the Data Intelligence Platform.”— Jibreal Hamenoo, Principal System Engineer, Data Engineering, Corning Incorporated
With Databricks Workflows, Corning improved monitoring and cost management by assessing resource usage and quickly pinpointing optimization opportunities. Enhanced observability empowered data teams to proactively address potential issues and repair subsets of tasks without restarting entire workflows. The result was a significant boost in agility, enabling Corning to respond swiftly to market changes, enhance supply chain reliability, and elevate customer experience.
Volvo's supply chain revamp: real-time inventory insights with Databricks Workflows
The Volvo Group has taken significant strides in optimizing its operations by switching from Azure Data Factory (ADF) to Databricks Workflows. This change streamlines inventory management for their massive global supply chain from supplier to truck dealer. Nearly 200,000 new Volvo trucks are sold yearly with millions more on the road that require hundreds of thousands of spare parts spread across warehouses globally. With real-time data processing, Volvo can monitor inventory levels continuously rather than waiting for shortages to strike. In a global supply-chain business, every minute counts and they don’t want to have their data become quickly outdated. This proactive and unified solution allows their data engineers to predict inventory needs more accurately, ensuring optimal availability for parts, while minimizing excess stock.
“Workflows has been a great orchestrator for us. We can query all the data using database APIs and build a monitoring report to see if a job is failing, how much time it’s taking on average and if it’s taking more than the average for that job.”— Bruno Magri, Senior Data Engineer, Volvo Group Service Market Logistics
By integrating Databricks Workflows with Delta Live Tables (DLT), Volvo streamlined its global supply chain operations. The automated workflows enable Volvo’s supply chain teams to better predict inventory needs, ensure optimal part availability, and minimize excess stock. Pairing DLT and Workflows not only improved operational efficiency - achieving up to 40% greater efficiency in handling large data volumes - but also empowered Volvo to quickly adapt to customer demands, driving smarter logistics and significantly enhancing their overall supply chain performance.
Cox Automotive boosts data efficiency and agility with Databricks Workflows
Cox Automotive has transformed its data operations with Databricks Workflows, employing automated workflows to manage large-scale data without bottlenecks.
Cox Automotive Europe is part of Cox Automotive, the world’s largest automotive service organization that is on a mission to transform the way the world buys, sells, owns and uses vehicles. The enterprise data services team maintains a data platform that primarily serves internal customers across business units, though they also maintain a few data feeds to third parties. The team collects data from multiple internal sources and business units.
“We use Databricks Workflows as our default orchestration tool to perform ETL and enable automation for about 300 jobs, of which approximately 120 are scheduled to run regularly.”— Robert Hamlet, Lead Data Engineer, Enterprise Data Services, Cox Automotive
Large amounts of data are processed in production pipelines today and scheduled jobs pull from different areas, both from within and outside of the company. Hamlet uses Databricks Workflows to deliver data to the data science team, to the in-house data reporting team through Tableau, or directly into Power BI. Workflows provides observability into every workflow run and every failure notification so they can get ahead of issues quickly and troubleshoot before the data science team is impacted.
Databricks Workflows simplifies Cox Automotive's data operations by automating ETL tasks and providing advanced observability across the pipeline. This integration enables data engineers to ensure smooth data flow, quickly detect issues and optimize performance. With Workflows orchestrating real-time data delivery to analytics tools, Cox can make faster, data-driven decisions, streamline operations, and improve responsiveness to market changes, boosting efficiency and agility.
Summary
These success stories from global manufacturing leaders Corning, Volvo, and Cox Automotive demonstrate the measurable impact of using Databricks Workflows. By using a unified orchestration service, fully integrated with the Databricks Data Intelligence Platform, these manufacturing leaders not only enhanced their operational efficiency but also accelerated innovation to deliver real-time, actionable insights. Whether it’s streamlining supply chains, ensuring inventory readiness, or automating workflows at scale, Databricks Workflows is enabling organizations to become more agile, efficient, and data-driven.
Getting started with Databricks Workflows
Databricks Workflows offers a simple, reliable orchestration solution for data and AI on the Data Intelligence Platform. Using Workflows you can have a unified workflow orchestration to implement ETL pipelines, ML training workflows and more. Workflows also offers enhanced control flow capabilities and supports different task types and triggering options.
Try Databricks for free
