Karthik Vasudevan - Databricks

Karthik Vasudevan

Senior Software Engineer, T-Mobile 

Karthik Vasudevan is Senior Software Engineer at T-Mobile. He has hands-on expertise in designing and developing large scale distributed data solutions for both on-premise and cloud platforms. He has a master’s degree in computer science and he has delivered many highly complex data analytics products in telco and finance domains. He is currently leading petabyte size data modernization journey to cloud.


Scaling Data Engineering Pipelines for Massive Hybrid EnvironmentsSummit 2020

To be the best Network company in the US, the Network Engineering team at T-Mobile is always evolving the way we deliver faster and higher quality insights. To support ever-changing technological needs and faster go-to-market on multi-petabyte scale network data, we developed a hybrid approach for on-premises and cloud for network-scale data-driven decision making. The task of identifying the right toolsets and frameworks is exceptionally crucial for a successful cloud modernization hybrid model. In this talk, we will share our experience in modernizing our multi-petabyte data workload to the cloud AND building streaming pipelines using Databricks & Delta Lake. We will primarily focus on:

  • Why are we doing this?
  • What are the core benefits the business wants including the desired and delivered latency requirements?
  • Design patterns we adopted for data movement to the cloud and streaming pipelines?
  • How Databricks & Delta Lake fit in our architecture
  • Best practices we adopted to build a cost-controlled scalable data analytics platform in the cloud