Insights into Accelerating Retail’s Data and AI ROI

Register for the Insights into Accelerating Retail's Data and AI ROI virtual event.

The global pandemic has accelerated trends in retail in an often reported metric, the rate at which e-commerce has replaced physical channels, grew more in the first three months of 2020 than in the last decade. The pandemic compelled consumers – en masse – to shift their buying patterns more rapidly and completely than during any other time in history. Retail has become an ever-changing landscape - consumers are more in control than ever, more mobile (at least somewhat digitally mobile considering the world dynamics) and more socially connected. Brick and mortar retail remains important, but retailers have had to learn how to adapt and enhance an omnichannel shopper experience. To do this, they’ve responded with accelerated investments in technology, and now are looking at how they can optimize their operations to improve profitability.

Retailers have turned to machine learning ( ML) to address the challenges in supply chain and changing customer preferences — and its impact is estimated to deliver $.9 - 1.7 billion in value to retail businesses annually. In fact, the retail segment has the highest potential to benefit from AI and ML when compared to all individual industry segments. This value is being played out in all segments and at all points of the value chain, with the potential upside for advanced analytics in this sector set to redefine Retail as we know it.

Retailers and consumer packaged goods purveyors are making data-driven, real-time decisions to counter the threat of sticky supply chains or opaque relationships with consumers. As the two charts illustrate below, high value use cases (demand forecasting with edge data or speech-to-text analytics as examples) require a variety of data types (such as call center transcripts leveraging speech to text algorithms generated by natural language processing NLP) driving a need for a data architecture that can ingest and process the increased volume and variety in real-time.

Whether seeking insights into consumer behavior or diving into supply chain demand forecasting, we see four common top data + AI investment priorities in Retail as they turn to data powered use cases:

• The need for real-time decisions: driving streaming ingestion of data to power critical real-time decisions for demand forecasting or next best offers
• Demand sensing and forecasting: Improving analytics for better demand forecasts and inventory management is critical in today’s volatile markets.
• Personalization and loyalty: Increased data volume allows for deeper customer segmentation and retention analytics, improving personalization for greater loyalty and increased revenue
• Data Sharing and collaboration: Improving collaboration and service levels with suppliers, distributors & delivery partners through low-cost, open source-based data sharing.

These four investment priorities are actualized through use cases like these:

1. Enabling growth and expansion of brands and products at scale due to a more optimized supply chain processes
2. Predictive maintenance to drive capital and facility cost reductions
3. Improved supply chain management through effective inventory management and well monitored, synchronized product flow from manufacturer to consumer
4. Improved human-robot collaboration, improving employee safety conditions and boosting overall efficiency
5. Dynamic pricing, providing the ability to change prices leveraging competitive pricing and predictive customer response models
6. Responsive customer-focused merchandising, offering quick changes to predictive market demand
7. Loss prevention through streaming analytics that identify and alert in-store associates of potential (real-time) fraud and shrink
8. Personalized marketing for optimized pricing and promotions through personalized marketing

While these priorities seem to be common across most retailers, many still struggle to meet the needs of these goals and achieve tangible results. In fact nearly 30% of operational use case deployment falls into “proof of concept (POC ) purgatory” and never realize a portion of the $.9-$1.7B/year value created in these three tops categories:

1. Pricing and Promotion
2. Supply Chain and Inventory Optimization
3. Customer Acquisition and Lead Generation

Why? Challenges arise in the organization in the form of people, processes or technology. The reasons retailers struggle with data and AI are:

Legacy data systems don’t support real-time Retail: Legacy data systems are batch oriented, meaning they bring data in on a scheduled basis. And then they have to do additional processing before that data is available. When seconds matter, data warehouses deliver in minutes and hours.

Forced to make compromises on accuracy: It’s not merely enough to load data, businesses seek to act on it. But data warehouses aren’t designed for large analysis. Data needs to be extracted and analyzed elsewhere, or you’re greatly limited in what can be analyzed. When running historical demand forecasts using a data warehouse, the categories and depth of forecasts needed to be limited because optimal granular forecasts would take days or weeks.

Limited support for different types of data: Data warehouses weren’t built for today’s data.
Responding to the market means leveraging all types of data. And so companies started implementing separate systems for tapping into unstructured data. But these are expensive and require integration with the data warehouses, adding costs and complexity

Expensive & proprietary data sharing: Retailers seek to collaborate across their value chain, but current systems are expensive and limited to the biggest players. The historical solution is to share data with partners requiring separate data warehouse licenses for each partner. Many large retailers have hundreds or even thousands of partners, making this impractical.

Use case implementation

So if it's so difficult to attain success with these AI initiatives, what is the path to see results and a return on investment? To answer this, retailers face a classic make or buy decision. Building use cases internally is a solid choice for digitally mature IT organizations with large data engineering teams. Building internally can eliminate the “black box” effect when buying solutions off the shelf.

The alternative is working with a solution integrator to tailor a purpose-built solution to the specific needs of the organization. These solutions are designed for rapid implementation, and lack vendor lock-in attributes so abhorred with many “black box solutions.”

Global solution integrators – Accenture, Capgemini, Deloitte and Tredence – have industry wide insight and knowledge from specialized vertical practices across many geographies and business organizations. Using this knowledge, they have developed purpose built solutions that work seamlessly with Databricks’ Lakehouse for Retail to deliver use cases with a near term return on investment. Let’s dive deeper into four purpose-built solutions offered by global solution integrators on the Databricks Lakehouse for Retail.

AI/ML driven data quality management - Tredence
Retailers and consumer goods are increasingly turning to real-time data to deliver insights that address evolving customers needs, forecast sharp fluctuations in demand, and mitigate on the shelf availability. The challenge in moving to real-time analytics is that if data quality is compromised, advanced analytics lose their value. To address data quality issues, Tredence has developed Sancus, an AI/ML data quality management tool, which improves data quality by cleansing, deduping, creating golden records and presenting data on an interactive dashboard with configurable data quality metrics. Together, Tredence Sancus and Databricks Lakehouse for Retail enable real-time insights for high-value retail use cases that:

1. Promote customer data enrichment through third-party partnerships
2. Enable global address validation and correction using postal directories and third-party APIs
3. Improve product and material enrichment through web scraping, image processing, unstructured data analysis, hierarchy management, and data governance.

Supply chain optimization and demand planning - Accenture & Deloitte
Due to the importance of supply chain and inventory optimization, Accenture has developed a unified view of demand, an open, glass-box approach to demand planning. The solution maximizes accuracy, granularity and timeliness with a single-source-of-truth demand plan enabling better explainability and alignment across various functions. It ties sourcing, demand and profit planning to:

1. Increase forecast accuracy, speed and granularity
2. Shift from debating forecasts to input alignment
3. Scale patterns for consumption-led forecasting

Deloitte’s Trellis provides capabilities that solve retail’s complex challenges around demand forecasting, replenishment, procurement, pricing and promotion services. Deloitte has leveraged their deep industry and client expertise to build an integrated, secured and multi-cloud ready “as-a-service” solution accelerator on top of Databricks’ Lakehouse for Retail that can be rapidly customized and tailored as appropriate based on the segments’ unique needs. With Deloitte Trellis:

1. Focus on critical shifts occurring both on the demand side and supply side of Retail’s value chain
2. Assess recommendations, associated impact and insights in real time
3. Achieve significant improvement to both top-line and bottom-line numbers

Revenue growth management - Capgemini
Addressing the importance of increasing customer satisfaction by building a better recommendation engine, Capgemini is offering a revenue growth management engine. In today’s dynamic environment, knowing what a customer purchased three months ago does not always tell you what they will purchase tomorrow. To establish fine-grained and accurate demand-sensing patterns, you need to incorporate externalities like dynamic market data, indices and social media. Capgemini’s revenue growth management engine leverages the Databricks Lakehouse for Retail to rapidly perform analysis of invoice data, external market data, indices, news and web scraping data to explore patterns. This improves on all aspects of the selling and marketing cycle, including acquisition, conversion and retention. With Capgemini Revenue Growth Management, you can

1. Preconfigure a visual interface and filter to display revenue growth
2. Pre-populate PySpark code and notebooks within Databricks for migrations
3. Utilize frameworks for initial product backlog structure and model selection criteria

The Databricks Lakehouse for Retail is addressing challenges that retail companies have long tried to solve – but struggled due to limits in the capability of technology. Operating a real-time business opens up possibilities for use cases like never before in demand planning, delivery time estimation, personalization or consumer segmentation. Decisions that could take hours, now can be made in seconds, which for many companies can mean a difference between profit or loss. Combined with these purpose-built use case solutions from globally recognized solution integrators, a robust customer success program, one of the largest open source communities supporting the underlying technologies, and a value assessment program that helps identify where and how to start on your digital transformation journey, Databricks is poised to help you become a leader in retail through a data-driven business.

Get started

Want to learn more about Lakehouse for Retail? Click here for our solutions page, or here for an in-depth ebook. Retail will never be the same now that Lakehouse for Retail is here.