Profit-Driven Retention Management with Machine Learning

Companies with the highest loyalty ratings and retention rates grew revenues 250% faster than their industry peers and delivered two to five times the shareholder returns over a 10 year period. Earning loyalty and getting the largest number of customers to stick around is something that is in the best interest of both a company and its customer base.

So why do companies struggle with retention? Other than some subscription-based businesses such as telecom that report Average Revenue Per User (ARPU), most companies aren’t required or compelled to disclose this in public filings. Many companies focus on functional priorities instead of the customer, believing customer loyalty will naturally emerge through these efforts. In fact, a recent survey by Nielsen highlights that addressing “customer churn is the last priority when it comes to companies’ marketing objectives.”

This is particularly problematic in light of increasing evidence that customers are rethinking how and where they spend their money. And while most studies identify these shifts as part of consumers’ response to COVID, the reality is that this growing disinterest in brand loyalty predates the current crisis.

Value Is Delivered Over a Customer’s Lifetime

Customer retention must become a priority for any company seeking long-term growth. In a series of recent posts on customer lifetime value in both subscription and non-subscription models, we examined how retention plays a critical role in building profitable customer relationships. At a minimum, customers need to remain engaged long enough for a firm to offset their acquisition costs, but an ideal relationship continues to deliver profits well beyond this.

Figure 1. Churn at different stages of the customer lifetime journey

The key to effectively managing retention, and reducing your churn rate, is developing an understanding of how a customer lifetime should progress (Figure 1) and examining where in that lifetime journey customers are likely to churn. In early stages, customers are still learning about the products and services they are consuming and how best to derive benefits from them. Proactive engagement to encourage the adoption of behaviors that maximize these benefits may help transition customers into later stages of sustained consumption. In those later stages, connecting with customers through brand identity can not only encourage continued loyalty but help customers become brand ambassadors, helping to organically bring new customers to the business and reducing on-boarding challenges.

You Can’t Save Them All...

When a customer abandons the relationship, it is important that we understand why. Some churn may represent the natural conclusion of a long-standing relationship which has finally run its course. In such scenarios, we may continue to derive value by transitioning the customer to other products and services within our portfolio or provided by a partner organization.  Or we may simply allow the customer to leave, content in knowing a satisfied customer is likely to continue to serve as a net promoter.

It’s when a customer leaves prematurely that we need to take corrective action. Churn in early stages of the lifetime journey may indicate difficulty in using products or services or recognizing value through them.  Churn in later stages may indicate diminished value, real or perceived, due to changes in the product, its delivery or the competitive landscape.  And at any stage, business process issues such as the failure to identify expiring credit cards may inadvertently push customers out.  The specific reasons for churn are highly varied and each requires a different kind of response both at the individual and the organizational levels.

Nor Should You Try 

When addressing the individual, it’s important to consider the costs and benefits of any corrective action. Every customer has a potential value to the firm, derived over the lifetime of a relationship. The cost of avoiding churn whether through promotions, discounts or other incentives should never exceed the residual value we might hope to preserve. Our goal should always be to retain profitability.

This requires not only a careful consideration of an individual’s CLV but the cost of implementing an active retention campaign on the whole. The planning and administration as well as the labor costs associated with consistent, sustained engagement must be averaged over the (ideally large) fraction of at-risk customers retained.

This is in no way intended to discourage organizations from pursuing a retention management strategy.  Indeed, numerous studies have shown that it costs 5-times (or more) to acquire a new customer than retain an existing one, and that firms may see as much as a 95% increase in profits with each 5% reduction in churn. Still, we must be careful to recognize and address the macro-level patterns behind churn to keep retention pressure down while also selectively engaging the at-risk customers with whom there is the most at stake.  And this is where machine learning and predictive analytics  can help.

Use Machine Learning to Quantify Likelihood of Churn 

The signals customers emit ahead of departure are often buried in the noise of overall customer activity.  Preventing a customer from leaving requires us to have some amount of advanced notice which is obtained through the careful examination of large volumes of historical data, something for which machine learning models are ideally suited.

The classic techniques, such as the use of logistic regression or decision trees, for proactive churn detection may be insensitive to events such as churn which (ideally) occur with low frequency (Figure 2). More modern techniques such as neural networks and gradient boosted trees are much more capable of picking up on the subtle shifts in patterns that denote churn, but require careful configuration and evaluation to do so.

Figure 2. The imbalance between churning and not-churning classes in a real-world dataset

The key to success with these models is to move away from a will-they or won’t they mindset and instead to embrace the uncertainty inherent in any churn prediction. When we begin to examine all vulnerable customers as having a quantifiable risk of churning, we can focus on removing uncertainty from our calculations.  Armed with more reliable predictions of churn risk, we can more carefully examine the residual CLV associated with individual customers and make more targeted decisions regarding when and how to intervene.

Use Databricks to Focus on Business Outcomes

Machine learning and data science in general is not easy. But bringing together the data with specialized software, managing the infrastructure to enable model processing and frequent reprocessing, and delivering outputs to downstream business systems shouldn’t be what consumes your organization’s time.

Leveraging elastic, cloud-based infrastructure under a platform with the most popular machine learning libraries pre-integrated, your data scientists have immediate access to the capabilities needed to get in motion. Using pre-integrated frameworks like hyperopt and mlflow, the previously laborious and time-consuming chore of optimizing model performance and configurations can be automated (Figure 3). And backed by a powerful, dynamically-scalable data processing engine, the mountains of data within which customer signals reside can be quickly and efficiently examined.

Figure 3. Model precision relative to various hyperparameter values

To see how these capabilities come together to tackle customer churn prediction, check out our Solution Accelerator assets which demonstrate how to go from raw data to prediction leveraging real-world data:

Get the Solution Accelerator