Comcast is the largest cable and internet provider in the US, reaching more than 30 million customers, and continues to grow it’s presence in the EU with the acquisition of Sky. Over the last couple years, Comcast has shifted focus to the customer experience. For example, Comcast has rolled out our Flex device which allows for customers to stream content directly to their TVs without needing an additional cable subscription. With the shift in focus to customer experience, Comcast has made a concerted effort to continue to make data driven decisions to understand how customers interact with our products while continuing to innovate with new products and subscriptions. The Product Analytics & behavior science (PABS) team plays a crucial role as an interpreter, transforming data into consumable insights and providing these insights to the broader product teams within Comcast. The PABS team does this on the entire Product ecosystem including X1, XFi and their brand new Flex devices, which is one of the largest streaming platforms in the world and this ecosystem is responsible for generating data at a rate of more than 25TBs per day with over 3PBs of data being used for consumable insights. In order for the PABS team to be able to continue to drive consumable insights on massive data sets while still being able to control the amount of data being stored, the PABS team have been using Databricks and Databricks Delta Lake to do high current low latency read/writes in order to build reliable real-time data pipelines to deliver insights and also be able to do efficient deletes in a timely manner. Some of the features from delta that we took advantage of to achieve the desired levels of efficiencies, optimization and cost savings are:
Jim Forsythe leads the Product Analytics & Behavior Science (PABS) team for the Technology, Product and Xperience organization at Comcast where he is responsible for transforming bits of data into consumable, productive insights. Jim's day is challenged with building data pipelines, researching new ideas, developing key metrics and informing data-driven decision making. Prior to Comcast, Jim led data science teams for a fortune 500 management consulting firm. He specialized in large scale product analytics, cloud platforms, user behavior research and retention modeling for new product initiatives.