Businesses need to know when customers end their business relationships, an act called “churn.” In a subscription business model, a customer churns by actively canceling their contract. The company can therefore detect and record this churn with absolute certainty. But when no explicit contract exists, churn is more passive and difficult to detect. Without any direct feedback from the customer, companies cannot determine whether the customer has lapsed temporarily or permanently.
Until now, detecting churn in such non-contractual relationships has been mostly arbitrary and more art than science.
Various analysts deal with the non-contractual churn conundrum in different ways. One popular approach is to assume the customer has churned if they lapse for a sufficiently long consecutive period of time. A problem with this approach, apart from it being guesswork, is that the chosen threshold for the length of the lapse period is often too high. This causes the business to wait too long to identify any churn problems. In Prediction of Advertiser Churn for Google Adwords, the authors are only able to measure churn after 12 months! Such a long wait period reduces the value of churn detection and the business’s ability to address problems. In analyses that estimate the churn period as a specified percentile of a distribution of buy cycles—time between successive customer purchases—choosing an optimal percentile (90th, 95th, 99th, etc) is difficult.
In this blog post, we present an improved scientific approach for defining non-contractual churn. Our approach avoids the struggle of choosing an optimal percentile by minimizing a well-defined objective function of type I and II errors.
Theory
Churn period (d) is the minimum length of consecutive silent (no transaction) periods beyond which a customer is considered to have ended their business relationship. Companies commonly partition a book of business into active and churned customers. Where customer relationships are non-contractual, any specified d will have associated type I & II errors. Therefore we should choose a definition that minimizes an objective function of these errors. In our approach, we specify the function to be a weighted average of the errors.
where:
- e1(d) is the expected type I error associated with churn definition d; Type I error is labeling the customer as churned when they are active;
- e2(d) is the expected type II error associated with churn definition d; Type II error is labeling the customer as active when they have churned;
- w is the weight the analyst places on type I errors relative to type II errors; it can be interpreted as the relative costs of the errors.
The optimal churn definition, denote d*, therefore minimizes F(d). We call F(d) the d-curve.
To compute the error functions, e1(d) and e2(d), we need to introduce another set of notation:
- ci represents the true churn status of customer i, 0=Active, 1=Churned;
- li represents the number of consecutive periods customer i has lapsed.
With the above definitions, e1(d) and e2(d) are derived as follows.
From (2) and (3), we see that e1(d) is the overall proportion of active customers mislabeled as churned. Similarly, e2(d) is the overall proportion of churned customers mislabeled as active.
Implementing the theory
Suppose you have data that has recorded the periods associated with all customer transactions from time S to T.
To determine the optimal churn definition, complete the following experiment:
- Specify the minimum number of periods, D, beyond which you are almost sure that the customer has truly churned. You can do this by empirically examining distributions of customer buy cycles (the difference in periods between successive customer transaction dates) and choosing a sufficiently high percentile. We’ll call D the validation period. This means that the subjects of the experiment have to be limited to the subset of customers who have at least one transaction prior to T-D; else we cannot calculate the customer’s true churn status, ci. Also, the length of the entire data (T-S) should be long enough to allow you to evaluate the selected domain of churn definitions for the d-curve, F(d). For example, if the domain is {d:d<K+1), then T-S must exceed K+D.
- If you are only interested in voluntary churn, remove all customers otherwise terminated involuntarily by the company.
- For each customer i, determine the last purchase period as of time T:
Calculate lapse period as of time T:
And calculate the true churn status:
- For each customer i, calculate the last purchase period as of time T-D:
Calculate lapse period as of time T-D:
- Select the domain of churn definitions, {d:dK}, on which you want to minimize F(d).
- For each churn definition in the selected domain, d =0, 1, 2…K, predict churn status for each customer as of time T-D, and measure the type I and II errors (e1(d) and e2(d)). Notice that e1(d) and e2(d) can be calculated from the data as follows:
- Select an appropriate weight, w.
- For d=0, 1, 2, …K, derive F(d) using (1).
- Choose the d that minimizes F(d) as your optimal d.
Results from real world application
We identified one of Indeed’s non-contractual products—job sponsorship—and applied both the percentile and d-curve methods to defining its churn period. We used monthly transaction data from September 2016 (S) through September 2019 (T).
Note that while the trends and insights we share are consistent with actual findings, we adjusted the actual results to protect the security of Indeed’s data.
Percentiles method
In this approach, we calculate the buy cycles for each customer. We can then represent each customer by a summary statistic (mean, median, and max) of their buy cycles. We then generate the distribution of the summary statistic across different customers:
| Quantiles | Mean | Median | Max |
|---|---|---|---|
| 0 | 1 | 1 | 1 |
| 0.2 | 2 | 2 | 2 |
| 0.4 | 2 | 2 | 2 |
| 0.6 | 3.5 | 3 | 5 |
| 0.8 | 4.7 | 3 | 9 |
| 0.9 | 6.2 | 5 | 13 |
| 0.95 | 8 | 7 | 17 |
| 0.99 | 15 | 15 | 25 |
| 1 | 38 | 38 | 38 |
| All figures illustrative | |||
These results illustrate the analytic dilemma associated with the percentiles method. The distribution varies by the choice of summary statistic. Even with a given summary statistic, it’s not clear which percentile (90th, 95th or 99th) is optimal. Apart from that, any reasonable choice of percentile results in unnecessarily high churn definitions. For example, the 95th percentile of the distribution of mean buy-cycles is 8 months, while that of maximum buy-cycles is 17 months! And we will see in the next approach that while such longer definitions have lower type I errors, they have higher type II errors.
The d-curve approach deals with all of these problems by choosing the churn definition with the minimum weighted sum of the type I and II errors.
D-curve approach
We parameterized our model as follows:
- w=0.5
- D=12
- S= 09-2016
- K=12
- T=09-2019
- T-D=09-2018
| Churn period | Type I error (%) | Type II error (%) | Weighted error (%) |
|---|---|---|---|
| 0 | 100.0 | 0.0 | 50.0 |
| 1 | 43.8 | 6.4 | 25.1 |
| 2 | 33.0 | 13.1 | 23.1 |
| 3 | 26.6 | 19.0 | 22.8 |
| 4 | 21.9 | 24.8 | 23.3 |
| 5 | 17.8 | 30.8 | 24.3 |
| 6 | 14.7 | 36.8 | 25.7 |
| 7 | 12.2 | 42.4 | 27.3 |
| 8 | 10.4 | 46.7 | 28.6 |
| 9 | 8.9 | 50.8 | 29.9 |
| 10 | 8.0 | 54.1 | 31.0 |
| 11 | 6.9 | 58.2 | 32.5 |
| 12 | 5.8 | 62.6 | 34.2 |
Using the d-curve, we choose 3 months as our optimal churn definition. A hypothesis test at 1% level of significance rejects the null hypothesis that the error for d=3 equals that of d=4.
More applications for the d-curve
We have formulated a framework for optimally selecting thresholds. While we apply the approach to define churn periods for non-contractual relationships, our approach has many other real world applications, chief of which is determining threshold probabilities in classification.
Acknowledgements
We are particularly grateful to Trey Causey, Ehsan Fakharizadi and Yaoyi Chen for their review and excellent feedback. We are, however, responsible for any mistakes in the post.