Productizing a Spark and Cassandra Based Solution in Telecom - Databricks

Productizing a Spark and Cassandra Based Solution in Telecom

Download Slides

Ericsson has re-engineered its voucher management solution for prepaid billing, by replacing RDBMS with Cassandra. It facilitates clusters with large set of nodes which can easily scale up & scale down. This facilitates telecom service providers to scale up capacity of vouchers upto a billion in a single cluster. Earlier one required multiple clusters to support a billion vouchers. Voucher management requires several batch processing tasks like bulk generation and bulk loading of vouchers. Each voucher has an activation code which is used by subscriber to topup his/her prepaid balance. Moreover, records in the system are in order of billion in a clustered environment and it is also required to generate reports for reconciliation and audit. This makes it a suitable candidate to leverage scalable distributed computing of Spark along with Spark-Cassandra-Connector. Spark provides a very efficient solution for the scenarios when a full-table scan is required, e.g. generating a report. In 3-node cluster, it reduced time required to generate a report from 6 hours to 20 minutes. Moreover, Spark-based report generation has better scalability because Spark-Cassandra-Connector is aware of token range distribution across Cassandra nodes. A product can have hundreds of installations worldwide. However, Spark and Cassandra are more popular in clustered solutions which are targeted for a single installation. Key challenge in productizing a solution based on big data technologies is lack of availability of talent pool for their administration and maintenance. Another major challenge is with life cycle management. Our solution uses Thrift to interface with Cassandra, whereas Spark-Cassandra-Connector is based on CQL. Due to a bug in Cassandra there were challenges in consistency of records across pages, while reading data written through Thrift. Secondly, Spark was unable to access dynamic columns. Other challenges were to efficiently read schema-less-tables and efficiently carrying out those queries which require join of two Cassandra-tables.

About Brij Bhushan Ravat

Brij is Chief Architect for prepaid billing product in Ericsson. The product uses Cassandra and Spark in business support systems for telecom service providers. He has also led Centre of Excellence for Network Applications, which tracks emerging trends in the application development in the area of telecom. This includes telecom services, OSS/BSS & leveraging big data technologies for innovative new age solutions. His focus is on application of big data in telecom. This includes analytics using Spark & NoSQL databases.