Connecting with viewers in a personalized way
Increase in operational efficiency lowers overall costs
"Moving from Hadoop to Databricks has unlocked significant business value. We are able to process more data faster, build reliable ETL pipelines, and make it easier for our analysts to build models that viewing experiences that our customers want.”
Viacom18 Media Pvt. Ltd. is one of India’s fastest-growing entertainment networks with 40x growth over the past decade, they offer a multi-platform, multi-generational, and multicultural brand experiences to 600+ million monthly viewers. In order to deliver more engaging experiences for their millions of viewers, Viacom18 migrated from their Hadoop environment due to its inability to process data at scale efficiently. With Databricks, they have streamlined their infrastructure management, increased data pipeline speeds, and increased productivity among their data teams. Today, Viacom18 is able to deliver more relevant viewing experiences to their subscribers, while identifying opportunities to optimize the business and drive greater ROI.
Growth in subscribers and TBs of viewing data pushes Hadoop to its limits
Viacom18, a joint venture between Network18 and ViacomCBS, is focused on providing its audiences with highly personalized viewing experiences. The core of this strategy requires implementing an enterprise data architecture that enables the building of powerful customer analytics on daily viewer data. But with millions of consumers across India, the sheer amount of data was tough to wrangle: they were tasked with ingesting and processing over 45,000 hours of daily content on VOOT (Viacom18’s on-demand video subscription platform) which easily generated 700GB to 1TB data per day.
“Content is at the heart of what we do,” explained Parijat Dey, Viacom18’s Assistant Vice President of Digital Transformation and Technology. “We deliver personalized content recommendations across our audiences around the world based on individual viewing history and preferences in order to increase viewership and customer loyalty.”
Viacom18’s data lake, which was leveraging on-premises Hadoop for operations, wasn’t able to optimally process 90 days of rolling data within their management’s defined SLAs, limiting their ability to deliver on their analytics needs which impacted not only the customer experience but also overall costs.
To meet this challenge head-on, Viacom18 needed a modern data warehouse with the ability to analyze data trends for a longer period of time instead of daily snapshots. They also needed a platform that simplified infrastructure by allowing their team to easily provision clusters with features like auto-scaling to help reduce compute costs.
Rapid data processing for analytics and ML with Databricks
To enable the processing power and data science capabilities they required, Viacom18 partnered with Celebal Technologies, a premier Salesforce, data analytics, and big data consulting organization based in India. The team at Celebal leveraged Azure Databricks to provide Viacom18 with a unified data analytics platform that modernizes its data warehousing capabilities and accelerates data processing at scale.
The ability to cache data within Delta Lake resulted in the much-needed acceleration of queries, while cluster management with auto-scaling and the decoupling of storage and compute simplified Viacom18’s infrastructure management and optimized operational costs. “Delta Lake has created a streamlined approach to the management of data pipelines,” explained Dey. “This has led to a decrease in operational costs while speeding up time-to-insight for downstream analytics and data science.”
The notebooks feature was an unexpected bonus for Viacom18, as a common workspace gave data teams a way to collaborate and increase productivity on everything from model training to ad hoc analysis, dashboarding, and reporting via PowerBI.
Leveraging viewer data to power personalized viewing experiences
Celebal Technologies and Databricks have enabled Viacom18 to deliver innovative customer solutions and insights with increased cross-team collaboration and productivity. With Databricks, Viacom18’s data team is now able to seamlessly navigate their data while better serving their customers.
“With Databricks, Viacom18’s engineers can now slice and dice large volumes of data and deliver customer behavioral and engagement insights to the analysts and data scientists,” said Dey.
In addition to performance gains, the faster query times have also lowered the overall cost of ownership, even with daily increases in data volumes. “Azure Databricks has greatly streamlined processes and improved productivity by an estimated 26%,” concluded Dey.
Overall, Dey cites the migration from Hadoop to Databricks has delivered significant business value — reducing the cost of failure, accelerating processing speeds at scale, and simplifying ad hoc analysis for easier data exploration and innovations that deliver highly engaging customer experiences.
