Role of Spark in transforming eBay’s Enterprise Data Platform - Databricks

Role of Spark in transforming eBay’s Enterprise Data Platform

Download Slides

eBay has one of the most mature Enterprise Data Platform’s in the industry with over 200PBs of data stored in our Hadoop and Teradata Warehouses. On average 30 TB of transactional and behavioral data is extracted on a daily basis and thousands of metrics are computed, analyzed and monitored for decision making and detecting anomalies. eBay has embarked on an ambitious project to transform the batch oriented ETL processes which could take 24 to 48 hour for metric computation to near real time infrastructure based on Kafka for messaging, Spark Streaming for stream processing and Spark SQL for data preparation.

« back
About Seshu Adunuthula

Seshu Adunuthula is the Head of Analytics Infrastruture at eBay responsible for managing some of the world’s largest deployments of Hadoop, Teradata and ETL Ingest platforms. He is an industry veteran with over 20 years of Distributed Computing and Analytics Experience. Prior to eBay he was managing the Engineering team at MapR responsible for MapReduce, MapR-DB and MapR Control System Products. Seshu also held various Engineering Manager and Architect roles at Oracle and Microsoft. Follow Seshu on Twitter @seshuad