How has customer churn evolved in the recent past?

Suppose that your boss wants to get a hold of customer churn. As a first step, she may ask you to diagnose the magnitude of the problem. After wrangling with the data you come up with the following two plots (Figure 2-1). The left plot shows a time series of daily churn rates. Confidently, you state two things: after having a relatively stable beginning of the year, churn is now on the rise. Second, there is a clear seasonal pattern, with weekends having lower than average churn. In the right panel you show that municipalities with higher average income also have higher churn rates, which of course, is a cause of concern since your most valuable customers may be switching to other companies.

Figure 2-1. Descriptive analysis of our company’s churn rate

This is a great example of what can be achieved with descriptive analysis. Here we have a relatively granular, up-to-date picture of the problem, and unfortunately, news are not good for the company: your boss was right to ask for this analysis, since churn is on the rise, having reached a yearly record at the end of May with no signs of going back to previous levels. Moreover, our remarkable ability to recognize patterns allows us to clearly identify three patterns in the data: changes in the trend and the existence of seasonal effects in our time series, and positive correlation between two variables in the scatterplot.

There are problems and risks with this type of analysis, however. As you’ve probably heard elsewhere, correlation does not imply causation, a topic that will be discussed at length later in this chapter. For instance, one plausible recommendation could be that the company should pull away from richer municipalities, as measured by average household income. This follows from a causal interpretation going from household incomes to churn rates, albeit an incorrect one.

Moreover, the question remains as to how to create value from this picture. We now know that churn is rising (admitedly, this is better than not knowing) but since we don’t know why, we cannot device a retention policy. You may argue that if we further inspected the data we may find the root causes for this upward trend, and this is the right way to proceed: formulate hypothesis that guide our analysis of the data. Inspecting data without advancing some plausible explanations is the perfect recipe for making your analytics and data science teams waste valuable time.

How many customers will churn this period?

As a next step, your boss may ask you to predict churn in the future. How should you proceed? It really depends and what you want to achieve with this analysis. If you work in finance, for example, you may be interested in forecasting the income statement for the next quarter, so you’d be happy to predict aggregate churn rates into the future. If you are in the marketing department, however, you may want to predict which customers are at risk of leaving the company, possibly because you may try using different retention campaigns. Each of these questions can be answered with different methods and hypotheses at different levels.

What is the best lever we can pull to prevent customers from churning

Finally, suppose that your boss asks you to recommend alternative courses of action to reduce the rate of customer churn. This is where the prescriptive toolkit becomes quite handy and where the impact of making good decisions can be most appreciated. You may then pose a cost-benefit analysis for customer retention and come up with a rule such as the following: retain a customer whenever the expected profits from retention are positive, taking into account your budget; profits are calculated by subtracting the expected costs from the expected savings. You can then order all customers according to this metric in a decreasing fashion, and target your campaign to those customers with largest net savings until you exhaust your budget (or when you’ve reached the customer where the campaign breaks even).

We will have the opportunity to go into greater detail on this use case, but let me just single out two characteristics of any prescriptive analysis: as opposed to the two previous analyses, here we actively recommend courses of action that can improve our position, by way of incentivizing a likely-to-leave customer to stay longer with us. Second, prediction is used as an input in the decision-making process, helping us calculate expected savings and costs. AI will help us better estimate these quantities, necessary for our proposed decision rule. But it is this decision rule that creates value, not prediction by itself.

One of the objectives of this book is to prepare us on how to translate business questions into prescriptive solutions, so don’t worry if it’s not obvious yet. We will have time to go through many step-by-step examples.

Simulated data

Let’s start by analyzing the data in Figure 2-4. Here, again, we immediately identify a very strong positive correlation between variables $Y$ and $X$. Can it be that two variables move together in such a strong manner, and there is no causal relationship between them? One thing should be clear from the outset: there is no way we can device causal stories without having some context, that is, without knowing what $X$ and $Y$ are and how they relate to the world.

Figure 2-4. A simulation of two highly correlated variables

This is an example of a spurious correlation, the case when two variables falsely appear to be related. The source of this deception is the presence of a third variable $Z$ that affects both variables, $Z⟹X$ and $Z⟹Y$; if we don’t control for this third variable, the two will appear to move together even when they are not related at all. I know that this is the case, because the following Python code was used to simulate the data.

# fix a seed for our random number generator and number of observations to simulate
np.random.seed(422019)
nobs = 1000
# our third variable will be standard normal
z = np.random.randn(nobs,1)
# let's say that z --> x and z--> y
# Notice that x and Y are not related!
x = 0.5 + 0.4*z + 0.1*np.random.randn(nobs,1)
y  = 1.5 + 0.2*z + 0.01*np.random.randn(nobs,1)

A simulation was used to unequivocally show the dangers of a third variable that is not taken into account in our analysis, so you may wonder if this is something we should worry about in your day-to-day work. Unfortunately, spurious correlations abound in the real world, so we should better learn to identify them and find workarounds. The effect of misrepresenting causation on the quality of a decision will not only lead to ineffective decision-making but also to a loss of valuable time when developing the predictive algorithms.

Churn and Income

Let us quickly revisit the positive association found between churn rates and households’ average income per municipality (Figure 2-1) and imagine that a strong competitor has entered the market with an aggressive pricing strategy targeted at customers in the medium-to-high income segments. It will be the case, then, that this third variable explains the positive correlation: more of your higher income customers will churn across all municipalities, but the effect will be higher in those where their relative size is also larger.

Can divorces in England explain pollution in Mexico?

Consider now the examples in Figure 2-5. The top left panel plots a measure of global CO2 emissions and per capita real gross domestic product (GDP) in Mexico for the period 1900-2016. The top right panel plots the number of divorces in Wales and England against Mexican GDP for 1900-2014. The bottom panel plots the three time series, indexed so that the 1900 observation is 100.⁷

Inspecting the first scatter plot, we find a very strong, almost linear relationship between the state of the Mexican economy, as measured by per capita real GDP, and global CO2 emissions. How can this be? Let’s explore causality in both directions: it is unlikely that CO2 emissions cause Mexican economic growth (to the best of my knowledge, CO2 is not an important input for any production processes in the Mexican economy). Since the Mexican economy isn’t that large on a global scale, it is also unlikely that Mexico’s economic growth has had such an effect on global contaminants. One can imagine that fast-growing economies like China and India (or the US and Great Britain during the 19th and 20th Centuries) would be responsible for a big part of global CO2 emissions, but this is unlikely for the case of Mexico.

Figure 2-5. Top left panel plots global CO2 emissions against real per capita Gross Domestic Product (GDP) for Mexico for the period 1900-2016. Top right panel does the same, replacing CO2 emission with the number of divorces in Wales and England during 1900-2014. Bottom plot shows the time series for each of these variables.

The second scatterplot shows an even more striking relationship: per capita real GDP in Mexico is positively related to the number of divorces in England and Wales, but up to a certain point (close to $10K dollars per person); after reaching that level the relationship becomes negative. Causal stories in this case become rather convoluted. Just for illustration, one such story —  from economic growth in Mexico to divorce rates in the UK —  could be that as the Mexican economy developed, more English and Welch people migrated to the North American country to find jobs and share the pie of economic prosperity. This could have created broken homes and a high prevalence of divorce. This story is plausible, but highly unlikely, so there must be some other explanation.

As before, there is a third variable that explains the very strong but spurious correlations found in the data. This is what statisticians and econometricians call a time trend, that is, the tendency of a time series to increase (or decrease) over time. The bottom plot depicts the three time series over time. Observe first the evolution of per capita GDP and CO2 emissions. The two time series evolve hand-by-hand until the late 1960s and beginning of the 1970s, thereafter maintaining different, but still positive, trends or growth rates. A similar comment can be made for the number of divorces. These trend or growth rates are the third variable that is common to the three time series, creating strong but spurious correlations. For this reason, practitioners always start by detrending or controlling for the common trend among different time series allowing them to extract more information from this noisy data.

High value customers have lower Net Promoter Scores

Let’s give some examples closer to the business now starting with the risks of making customer satisfaction comparisons across customer segments. Recall that the NPS is a metric commonly used to track customer satisfaction, so it is natural to compare it across segments as in Figure 2-6.

Figure 2-6. Net Promoter Score (NPS) for two different value segments

The plot depicts average NPS for two different customer value segments, Premium and Regular, corresponding to customers with high and low customer lifetime values (CLV), respectively. The bars show that NPS is negatively correlated with the value of the customer, as measured by the CLV. Since this is a customer-centric company, one possible recommendation could be to focus only on lower-value customers (since they are the most satisfied). This is a causal interpretation that goes from value to satisfaction, and a customer’s value is treated as a lever. It could be, however, that a third variable is affecting both the NPS and the CLV, and that once we control for this intervening variable the relationship disappears. One such possible third value is our customers’ socioeconomic level, possibly capturing the higher quality expectations that we described in the churn example.

Selection effects and the health status of our employees

Another important reason why we cannot immediately identify causality when analyzing our data are selection effects.⁸ Suppose that our Chief Human Resources Officer is considering saving costs by eliminating the company’s on-site medical service, since in his opinion most employees use the company-provided off-site services. Since this is a data-driven company, she decides to apply an anonymous survey asking the following two questions to the employees:

1. How is your health in general? Please provide an answer from 1 to 5, where 1 is Very Bad, 2 is Bad, 3 is Fair, 4 is Good and 5 is Excellent.

2. In the last 2 months, have you been treated by our on-site doctors?

Figure 2-7 shows the average self-reported health status for groups of employees that declare having used the medical service or not. The CHRO was deeply concerned with the result presented by the Analytics Unit: treated employees report having worse health than those not treated. It seems that our on-site medical service is generating the exact opposite result it was designed for.

Figure 2-7. Self-reported health status for employees treated not treated on-site. Vertical lines correspond to confidence intervals.

But is this analysis sound? One data scientist pointed out that it could be the result of selection effects. Her rationale is that employees that feel sick may be self-selecting themselves into using the on-site service. She concluded that the results show the higher likelihood of going to the doctor of the sick and not the causal effect of providing on-site assistance on the employees’ health, so, she concludes, we must close it.

Figure 2-8 shows the two directions of causality. Self-selection (1) implies that sick employees are more likely to use the on-site service. They get treated according to standard medical practices, on average improving their conditions (2). The causal effect that the CHRO expected to find was given by (2), but unfortunately the selection effect was strong enough to counter the positive impact that the company’s doctors have.

Figure 2-8. Self-selection explains why treated employees show worse health conditions than those that have not attended the on-site medical service

Adverse selection in the credit card business

Let’s discuss now adverse selection, a type of selection effect common in situations where the involved parties have asymmetric information about the other side. Since it is commonly used to analyze credit outcomes in the financial sector, let us suppose that the Chief Risk Officer for a bank is deeply concerned about the recent upsurge in delinquency rates for the credit card products. After discussing alternative courses of action in the weekly Credit Committee they decide to increase interest rates hoping that this would lower the demand for credit and help cover the extra cost of risk.

One quarter later they find that, indeed, their credit card portfolio shrunk. However, delinquency rates increased. What happened here? At the new higher cost for credit, higher-risk customers were the only ones willing to accept such high interest rates. Our choice of levers incentivized the exact market segment we wanted to avoid to self-select and ask for credit card loans.

Can investing in infrastructure increase customer churn?

Another example from a capital intensive industry like telecommunications will show the dangers of decision-making in the presence of selection effects. Telcos have very large capital expenditures (CAPEX) since they need to constantly invest in building and maintaining a network to provide high quality communication services to their customers. Suppose that our Chief Financial Officer must decide where to focus our investing efforts during the next quarter. After looking at the data they plot churn rates in cities with no CAPEX last year and those where there was positive investment (Figure 2-9).

Figure 2-9. Average churn rates in cities with and without CAPEX during the previous year

The results are both surprising and frustrating: it appears that CAPEX has had unintended consequences as churn is higher in cities where they invested last year relative to those without CAPEX. Could it be, maybe, that competitors reacted strategically investing even more heavily in those cities and capturing an increasing market share? That is likely, of course, but here, the most plausible explanation are selection effects. Last year they focused the investment efforts in cities that were lagging in terms of customer churn and satisfaction. The result is exactly like the one in the medical condition: the patients (the cities) still haven’t recovered fully so the data still shows the disadvantaged initial conditions.

Marketing mix and channel optimization: the case of online advertising

Our Chief Marketing Officer asks us to estimate the revenue impact of advertising spending on different channels, with a specific focus on our digital channels. After devoting considerable effort getting the data, we find some very pleasant news: the online advertising Return on Investment (ROI) was 12% for the year, the second consecutive year with double digits. However, members from the data science team raised concerns about us overestimating the real impact. Their logic was as follows (Figure 2-10).

Our retargeting partner usually waits for internet users to visit our webpage. When that happens, they place a cookie so that we can track their online behavior. Some time later, an ad is served on a publisher’s website with the hope of converting this lead. Some of these customers end up buying our products on our website, so it seems that our advertising investment has done wonders for us.

Figure 2-10. Selection effects explains high digital marketing ROI

The problem here is conceptual, though: ideally, advertising should be done to convert a user who was not planning to buy from us into a buyer. But those who end up buying from us had already self-selected themselves by visiting our webpage, hence showing interest on our products. Had we not placed the ad, would they have purchased anyway? If the answer to this counterfactual is affirmative, then there is a case that the ROI might be overestimated; it could even be negative! To get a reasonable estimate we must get rid of our customers’ self-selection.

Estimating the impact that influencers have on our revenues

Our data-driven CMO also wants us to evaluate the impact that our army of influencers had on our revenues. Ideally, influencers’ reach should create a network multiplier effect, the multiplier given by the number of followers, their followers’ followers, and so on. That’s both the beauty and complexity of social networks.

Influencing is difficult to measure, however, since followers self-select themselves to follow the influencer. No one forced them. They already liked something about the influencer, making them more likely to behave like the influencer, not because of the influencer, but since they share some common, possible unobserved to us, tastes. This is the classic chicken and egg problem.

Figure 2-11. Counterfactual analysis used to estimate the effect that influencers have. Situation (A) depicts the case where followers were influenced and changed their consumption patterns. In situation (B) followers self-selected themselves.

Figure 2-11 shows two alternative scenarios, A and B, at two different moments in time: before we measure the influencer’s impact and during the measurement. Most of us would agree that scenario A is closer to what we would expect from an influencer: her followers changed their behavior thanks to her own change (not observed here). Similar to the online advertising case, here our influencer converted non-customers into customers so she may well be well worth her price. Scenario B however can be potentially explained by self-selection of equals, and our influencer’s impact seems lower or even null.

Problem 1: We can’t observe counterfactuals

As discussed in the previous sections, there are several problems that make our identification of causal effects much harder. The first one is that we only observe the facts so we must imagine alternative counterfactual scenarios. In each of the previous examples, we knew that direct causal interpretation was problematic since we were able to imagine alternative universes with different outcomes. It is an understatement that one of the most important skills analytical thinkers must develop is to question the initial interpretation given to empirical results, and to come up with counterfactual alternatives to be tested. Would the consequences be different, had we pulled different levers, or the same levers but under different conditions?

Let’s stop briefly to discuss what this question entails. Suppose we want to increase lead conversion in our telemarketing campaigns. Tom, a junior analyst who took one class in college on Freudian psychoanalysis suggests that female call center representatives should have higher conversion rates, so they decide to make all outbound calls for a day with their very capable group of women representatives. The next day they meet to review the results: lead conversion went from the normal 5% to an outstanding 8.3%. It appears that Freud was right, or better, that Tom’s decision to take the class had finally proven correct. Or does it?

To get the right answer, we need to imagine a customer receiving one call from the female representative in one universe, and the exact same call from a male representative in a parallel universe (Figure 2-12). Exact customer, exact timing, exact mood and exact message; everything is the same in the two scenarios: we only change the tone of voice from that of a male to a female. Needless to say, putting in practice such counterfactual sounds impossible. Later in this chapter we will describe how we can simulate these impossible counterfactuals through well-designed randomized experiments or A/B tests.

Figure 2-12. Counterfactual analysis of lead conversion rates in a call center

Problem 2: Heterogeneity

A second problem is heterogeneity. Humans are intrinsically different, each and every one the product of both our genetic makeup and lifetime experiences, creating unique world visions and behaviors. Our task is not only to estimate how behavior changes when we choose to pull a specific lever —  the causal effect —  but we must also take care of the fact that different customers react differently. An influencer recommending our product will have different effects on you and me: I may now be willing to try it while you may choose to remain loyal to your favorite brand. How do we even measure heterogenous effects?

Figure 2-13 shows the famous bell curve, the normal distribution, the darling of statistical aficionados. I’m using it here to represent the natural variation we may encounter when analyzing our customers’ response when our influencer recommends our product. Some of his followers, like me, will accept the cue and react positively —  represented as an action right of the vertical dashed line, the average response across all followers, followers’ followers and so on. Some will have no reaction whatsoever, and some may even react negatively; that’s the beauty of human behavior, we sometimes get the full spectrum of possible actions and reactions. The shape of the distribution has important implications, and in reality, our responses may not be as symmetric; we may have longer left or right tails and reactions may be skewed towards the positive or the negative. The important thing here is that people react differently, making things even more difficult for us when we try to estimate a causal effect.

Figure 2-13. A normal distribution as a way to think about customer heterogeneity

The way we usually deal with heterogeneity is by dispensing of it by estimating a unique response, usually given by the average or the mean (the vertical line in Figure 2-13). The mean, however, is overly sensitive to extreme observations, so we may sometimes replace it by the median, having the property that 50% of responses are lower (to the left) and 50% higher (to the right); with bell-shaped distributions the mean and the median are conveniently the same.

Problem 3: Selection Effects

One final problem covered in detail in the previous section is the prevalence of selection effects. Either we choose the customer segments we want to act upon, or they self-select themselves, or both. One general result when empirically comparing the average outcomes of two groups, say those treated or not treated is:⁹

$\text{Observed}\text{Difference}=\text{Causal}\text{Effect}+\text{Selection}\text{Bias}$

It is standard practice to plot average outcomes as in the left panel of Figure 2-14. In this case, the outcome for the treated (customers receiving a call from female reps) is 0.29 units (say dollars) higher than for those not exposed to our action or lever. This number corresponds to the left-hand side of the previous equation. The right panel shows the corresponding distributions of outcomes. Using the mean to calculate differences is standard practice, but it is useful to remember that there are a full spectrum of responses, in some cases with a clear overlap between the two groups: the shaded areas show responses from customers in the two groups that are indistinguishable from each other.

Figure 2-14. Left panel plots the observed differences in average sales for customers receiving a call from female and male representatives. Right panel shows the actual distributions of outcomes.

In any case, the difference in observed outcomes (left-hand side) is not enough for us since we already know that it is potentially biased by selection effects; since our interested is in estimating the Causal Effect we must therefore device a method to cancel the pervasive effect from selection.

Statisticians and econometricians, not to mention philosophers and scientists, have been thinking about this problem for centuries now. Since it is physically impossible to get an exact copy of each of our customers, is there a way to assign our treatments and circumvent the selection bias? It was Ronald A. Fisher, the famous 20th century statistician and scientist who put on firm grounds the method of experimentation, the most prevalent among practitioners when we want to estimate causal effects. The idea is simple enough to describe without making use of technical jargon.

A/B testing in practice

In the industry, the process of randomizing to assign different treatments is called A/B testing. The name comes from the idea that we want to test an alternative $B$ to our default action $A$, the one we commonly use. As opposed to the techniques in the machine learning toolbox, A/B testing can be performed by anyone without a strong technical background. We may need, however, to guarantee that our test satisfies a couple of technical statistical properties, but these are relatively easy to understand and put in practice. The process usually goes as follows:

1. Select an actionable hypothesis you want to test: for example, call center female representatives have a higher conversion rate than men. This is a crisp hypothesis that is falsifiable.

2. Choose a relevant and measurable KPI to quantify the results from the test: in the example, we have chosen the average conversion rates. If conversion rates for female reps are not sufficiently higher than those for men we have falsified the hypothesis.

3. Select the number of customers that will be participating in the test: this is the first technical property that must be carefully selected and will be discussed below.

4. Randomly assign the customers to both groups and check that randomization produced groups that satisfy the ex-ante indistinguishable property.

5. After the test is performed, measure the difference in average outcomes. We should take care of the rather technical detail of whether a difference is generated by pure chance or not.

If randomization was done correctly, we have eliminated the selection bias, and the difference in average outcomes provides an estimate of the causal effect.

Understanding power and size calculations

Step 3, selecting the number of customers, is what practioners call power and size calculations, and unfortunately there are key trade-offs we must face. Recall that one common property of statistical estimation is that the larger the sample size the lower the uncertainty we have about our estimate. We can always estimate the average outcome for groups of 5, 10 or 1000 customers assigned to the $B$ group, but our estimate will be more precise for the latter than for the former. From a strictly statistical point of view, we prefer having large experiments or tests.

From a business perspective, however, testing with large groups may not be desirable. First, our assignment must be respected until the test comes to an end, so there is the opportunity cost of trying other potentially more profitable treatments, or even our control or base scenario. Because of this, it is not uncommon that the business stakeholders want to finish the test as quickly as possible. In our call center example, it could very much have been the case that conversion rates were lower with the group of female reps, so during a full day we operated suboptimally which may take an important toll on the business (and our colleagues’ bonuses). We simply can’t know at the outset (but a well designed experiment should include some type of analysis of this happening).

Because of this trade-off we usually select the minimum number of customers that satisfies two statistical properties: experiments should have the right statistical size and power so that we can conclude with enough confidence if it was a success or not. This takes us to the topic of false positives and false negatives.

False positives and false negatives

In our call center example, suppose that contrary to Tom’s assumption, male and female representatives have the exact same productivity or conversion efficiency. The real but unobserved effect of our treatment should then be zero, but when we compare the average outcomes the difference will always be non-zero, even if small. How do we know if the difference in average outcomes is due to random noise or if it is showing a real, but possibly small difference? Here’s where statistics enter the story.

There is a false positive when we mistakenly conclude that there is a difference in averages between groups, that is, when we conclude that the treatment had an effect. The choice of the size of the test is done to minimize the probability of this happening.

On the other hand, it could be that the treatment actually worked, but we chose a sample small enough that we are not able to conclude with confidence that it is not caused by random noise. Note that this is quite a common scenario, and could happen if our business counterparts think that by having a larger test they will risk the results of the current quarter. The result is that we end up with an underpowered test. In our call center example, we falsely conclude that representatives’ productivity is the same across genders when indeed one has higher conversion rates.

The left panel in Figure 2-15 shows the case of an underpowered test. The alternative $B$ treatment creates 30 additional sales, but because of the small sample sizes, this difference is estimated with insufficient precision. The right panel shows the case where the real difference is close to 50 extra sales, and we were able to precisely estimate the averages and their differences.

Figure 2-15. Left panel shows the result of an underpowered test: there is a difference in the average outcomes for the treated and untreated but the small sample sizes for each group cannot estimate this effect with enough precision. Right panel shows the ideal result where there is a difference and we can correctly conclude this is the case.

Let’s briefly talk about the costs of false positives and false negatives in the context of A/B testing. For this, recall what we wanted to achieve with the experiment to begin with: we are currently pulling a lever and want to know if an alternative is superior for a given metric that impacts our business. As such, there are two possible outcomes: we either continue pulling our A lever, or we substitute it with the B alternative. In the case of a false positive, the outcome is making a subpar substitution. Similarly, with a false negative we mistakenly continue pulling the A lever, which also impacts our results. In this sense both are kind of symmetric (in both cases we have an uncertain long-term impact), but it is not uncommon to treat them asymmetrically, by setting the probability of a false positive at 5% or 10% (size), and the probability of a false negative at 20% (one minus the power).

There is however the opportunity cost of designing and running the experiment. With an underpowered experiment we cannot conclude anything meaningful: if there is a difference across treatments, we just won’t be able to identify it with confidence. If there isn’t we won’t be able to distinguish it from pure chance. That’s why most practitioners tend to fix the size of a test and find the minimum sample size that allows us to detect some minimum effect.

# Example: calculating the sample size for an A/B test
from scipy import stats
def calculate_sample_size(var_outcome, size, power, MDE):
    '''
    Function to calculate the sample size for an A/B test
    MDE = (t_alpha + t_oneminusbeta)*np.sqrt(var_outcome/(N*P*(1-P)))
    df: degrees of freedom when estimating the variance of the outcome
    (usually large enough)
    '''
    df = 1000
    t_alpha = stats.t.ppf(1-size, df)
    t_oneminusbeta = stats.t.ppf(power, df)
    # same number of customers in treatment and control group
    P = 0.5
    N = ((t_alpha + t_oneminusbeta)**2 * var_outcome)/(MDE**2 * P * (1-P))
    return N

sample_size_for_experiment = calculate_sample_size(var_y, size, power, MDE)
print('We need at least {0} customers in experiment'.format(
np.around(sample_size_for_experiment),decimals=0))

In practice, we start by setting these values (if symmetry seems like an appealing property, setting them to same level sounds like a reasonable alternative, otherwise you can follow standard practices such as the one described above). We then must choose a minimum detectable effect, that is, the minimum impact on the metric we wish to affect (say profits) that makes the experiment worthwhile for the business. We can then reverse engineer the sample size we need.