In their mission to build North America’s most efficient digital transportation network, J.B. Hunt wanted to streamline freight logistics and provide the best carrier experience — but legacy architecture, their lack of AI capabilities and the inability to securely handle big data caused significant roadblocks. However, after implementing the Databricks Lakehouse Platform and Immuta, J.B. Hunt is now able to deliver operational solutions that range from improving supply chain efficiencies to boosting driver productivity — resulting in significant IT infrastructure savings and revenue gains.
In order to fulfill their goal of becoming the most efficient freight transportation network in North America, J.B. Hunt focuses primarily on connecting carriers with their ideal shipper, taking into consideration details such as price, weight and location.
“What we’re doing from a data science point of view is building pricing models and load recommendation models to improve operations,” explained Doug Mettenburg, Vice President of Engineering and Technology at J.B. Hunt. “But in my 20 years here, data refreshes have only been, at best, every night, and that’s the case across our industry. The problem is, trucks move. Having to wait overnight for data made a lot of what we wanted to provide around tracking and modeling impossible.”
Prior to Databricks, J.B. Hunt locked their data in legacy enterprise data warehouse (EDW) platforms. Their systems struggled to process and store the massive data generated by hundreds of thousands of equipment pieces. They also lacked the necessary levels of data security and the ability to support data streams generated by IoT sensors on their trucks and carriages. J.B. Hunt knew it was time for a change.
To achieve their goal of modernizing their data infrastructure and meet business objectives, J.B. Hunt moved to the Databricks Lakehouse Platform for its ability to unify data engineering and data science functions on one open and scalable platform. “As we look toward expanding our ML and real-time analytics capabilities, it was critical that we built upon a platform that provides the flexibility to quickly deploy use cases regardless of which cloud or tool sets are being leveraged across our diverse operations — and that’s what Databricks provided us.”
With Delta Lake, J.B. Hunt not only has the ability to put all their data in one place for easy access across the organization — they can also ensure the performance and reliability of streaming data pipelines at any scale. The support for Delta Lake as the open storage layer brought efficiency and portability to J.B. Hunt’s teams as they moved terabytes of their existing data onto the platform. By streaming in real time to Delta Lake — with web, mobile, location, IoT and other application data — J.B. Hunt can analyze larger, more complete data sets to run analytics and ML faster than ever. With MLflow, the data science team is now able to establish reproducibility of code and experiments to ensure they’re reusable by multiple data scientists.
“Before Databricks, people didn’t really understand the models, and couldn’t query our data in order to remedy that issue,” explained Joe Spinelle, the Director of Engineering and Technology at J.B. Hunt. “But now, we’re able to ask all kinds of questions about the data from various business unit perspectives, which has helped us make the improvements that needed to be made.”
Databricks is also being used in conjunction with Immuta, an automated data governance platform. “Databricks opens up many opportunities for self-service data analytics, data science, and enterprise reporting,” explained Ajay Sahu, the Director of Enterprise Data Management at J.B. Hunt. “Paired with Immuta, we can make all our data available to all types of business analysts, data scientists and data engineers.”
Immuta gives J.B. Hunt a level of data security that wasn’t possible with their legacy EDW. They are now able to automate the data governance process to ensure that access to data isn’t being given to people who aren’t supposed to have it.
In terms of collaboration, Databricks has succeeded in bringing the various data teams together to accelerate data science productivity.
“What Databricks has really given us is a foundation for the most innovative digital freight marketplace by enabling us to leverage AI to deliver the best carrier experience possible. Without Databricks, we’d be stuck in a very manual, very labor-intensive world that would really hamper analytics and data science,” added Joe.
The success of Databricks is reflected in J.B. Hunt’s phenomenal performance gains, including the ability to train thousands of ML models in less than 4 hours and the ability to deliver freight recommendations to drivers 99.8% faster than before.
J.B. Hunt has also experienced a significant impact on their business operations, realizing $2.7 million in infrastructure savings and productivity gains. In the next year, they’ve projected $4.4 million in realized value as they continue to scale their use of Databricks across the enterprise with Immuta serving as their data security and governance layer.
“Ultimately, Databricks is now the source of truth for J.B. Hunt,” added Doug. “It’s showing the real value of the data we bring to the entire company, as we create more AI solutions that greatly impact our business.”
What Databricks has really given us is a foundation for the most innovative digital freight marketplace by enabling us to leverage AI to deliver the best carrier experience possible.”
– Joe Spinelle, Director, Engineering and Technology, J.B. Hunt