Tier 1 Commercial Bank in Latin America

A Tier 1 commercial bank, headquartered in Latin America, comprising both retail and wholesale banking operations, had a problem with maintaining asset quality in their loan portfolio, while meeting ambitious growth targets by acquiring new customers. It became especially challenging against a backdrop of rising interest rates and increases in loan-level defaults.

To effectively scale, the bank decided to invest in a new risk modeling platform that operationalized the risk model lifecycle. The platform needed to be robust enough to automate the development and deployment of machine learning models. It became a “machine learning model factory” where highly structured and industrialized model development and deployment processes were put in place. These included data preparation and model development processes, where nonregulatory machine learning models for selected market segments could be developed and automatically deployed into production.

The bank used the opportunity to modernize its technology to also improve its processes. It analyzed each step in its model-building process and identified bottlenecks and their causes. This allowed the bank to improve, for example, model dataset creation from months to days. This delivered an important lesson—simply buying new technology without addressing existing infrastructure, human, or process issues does not yield great results.

The Machine Learning Factory dramatically reduced model development and deployment time from more than 12 months to less than one day. Target markets were able to be segmented at a more granular level, producing better models with more accurate prediction outcomes. A faster, more efficient model lifecycle allowed for more models to be developed and deployed, increasing the number of models under management fivefold. It was achieved by improved model pipeline designs that enabled some machine learning algorithms to update dynamically, with continuous development of challenger models that were used to contest the production champion models. When the challenger was deemed to be better performing, it was deployed into production automatically.

This resulted in better quality of loans and millions of dollars in credit losses avoided. New loans could also be issued with more ease, which enabled expansion of the bank's portfolio across many markets.

Tier 1 Financial Institution in Asia Pacific

Many financial institutions in Asia Pacific are actively using machine learning in nonregulatory settings. One interesting application is the detection of complex transactions that indicate instances of fraud. In some cases, banks are relying on financial technology companies that specialize in AI to build these models. But with easier access to platforms and no-code/low-code interfaces for AI and machine learning, increasingly organizations are developing these themselves.

The occurrence of credit application fraud represents a very small fraction of the population, less than 1%. However, the losses are typically severe. A major concern at a Tier 1 commercial bank in Asia Pacific was that although the number of fraud cases was small, the vast majority bypassed the bank's existing fraud rules. This was a complex problem. The best efforts of the bank, including manual interventions and contacting applicants, meant that some incidents still slipped through the credit- and fraud-checking processes.

To detect the fraudulent behavior, the bank built predictive models based on actual historical data of the approved applications using a gradient boosting machine (GBM). This technique used thousands of decision trees and hundreds of variables as a basis of generating weak prediction models that get ensembled to a final prediction via a score.

Although the GBM models were great at detecting suspicious behavior, the obstacle the bank faced was that the current infrastructure was not robust enough to support the deployment and implementation of advanced machine learning models like GBMs. In the end, the sophistication of the GBM needed to be completely stripped back to a linear model, which meant that the predictive capability was reduced. This story illustrates the potential power of machine learning in improving predictions—but it also reminds us that infrastructure capability needs to be addressed before deploying resources, in the form of funds and time, toward a machine learning project.

Process Automation for Claims Processing

Several organizations have started to use AI and machine learning to develop AI-based behavioral models that can be used to support other applications to improve customer experience. One such model was developed using over a decade of insurance data to find hidden pathways that exist between the question sets asked and the most profitable 20–40 customer segments. Based on the insights of the model, over 30 questions previously used to ascertain medical history were reduced to 7 key questions needed to confirm or deny coverage. What this predictive underwriting capability of the model supplies is faster, easier, and more accurate insurance outcomes that greatly improve the customer experience.

Other AI and machine learning models have been developed to identify the key reason for excluding customers for medical insurance, such as those with a particular type of disability, and whether the excluded population has a higher propensity to make claims at a later point in time. Again, this has contributed to a more efficient underwriting process by reducing time, cost, and paperwork needed to make insurance decisions.

The one condition for all models developed for this project was that it needed to be transparent, meaning that all the inputs and output decisions can be easily explained. The avoidance of “black box” models in favor of those that can be explained easily is a common requirement in financial institutions. As such, explainable AI is a key consideration with any development project and is discussed in greater detail in Chapter 4.

Navigating through the Storm of COVID-19

Like many financial institutions across the globe, a Tier 1 commercial bank in the Asia Pacific region provided repayment holidays to eligible small business customers during the COVID pandemic. Since the customers' risk of default could not be measured using current systems and processes, payments appeared up-to-date due to the referred repayment period, putting the bank at risk. If income were lost during the repayment holiday period, the current behavioral models would not accurately reflect these events, as these models were not designed to predict short-term and immediate events, like a sudden loss of income due to widespread business shutdowns. What this meant is that the current definition of “delinquency” were not a reliable differentiator of risk for small business customers who received deferral arrangements under emergency procedures enacted during COVID-19.

Suddenly, the bank needed to assess how this would affect customer payments, and to do this they turned to machine learning. The bank used data immediately before the COVID-19 pandemic, using 3 months of historical data rather than the 12 months performance window used by the legacy behavioral models. The smaller historical period of 3-months meant that shorter-term variables that are highly impacted by COVID19 needed to be derived, rather than using the long-term drivers of the current models. For the short-term variables, the modelers derived these using advanced feature engineering techniques from customer transactions data, to ensure that recent changes in cash flows could be detected and modeled by the algorithm. Importantly, the transaction data–based feature engineering used income and expense variables highly affected by COVID19. Other variables used in the model included point-in-time ratios, early delinquency at customer group level, and drivers related to velocity of change such as utilization, disposable income, and business-specific risk drivers and self-employment flags. The bank observed that a GBM with a small depth and <100 trees were able to better predict which customers were likely to suffer sudden loss of income events and likely to experience default post-expiry of the COVID-19 holiday repayment scheme.

Approximation of Complex Risk Calculations

Like other sophisticated risk calculations, expected credit loss calculations are typically run as a batch, back-end process. Forward-looking, marginal losses are calculated for a range of scenarios, discounted back to the net present value, and weighted according to the probability of the scenario. If the probability of default has significantly increased since origination, the loan is recognized as stage 2 and lifetime losses are recognized. If not, a 12-month loss is recognized as provision for expected credit loss.

Before deciding on a new loan, what is the expected credit loss under this scenario? This can be approximated using machine learning. The benefit of machine learning is that it can quickly inform the estimated expected credit loss in real time, at the time of loan origination, leading to better decisions.

Artificial Intelligence

In 1950, the famous English mathematician Alan Turing asked the question, “Can a machine think?”⁴ That question has provoked the computer science domain from its very beginning. Turing's work during World War II on a machine that was able to decipher code from the Nazis' “Enigma” machine provided the edge that the Allied Forces needed to win the war in Europe. The machine underwent several improvements and influenced the development of the first digital computers.

Later in 1955, the American father of AI, the computer scientist John McCarthy, first coined the term artificial intelligence while preparing for a conference that explored ways to make machines reason like humans, capable of abstracting thought and solving problems.⁵ And this is what AI is essentially about, a computer system that can do tasks that humans need intelligence to do.

In general, AI is often used to describe computer-based systems that can think, learn, and respond to stimuli in their environment. Importantly, like human intelligence, the AI computing system can also subsequently act in response to these stimuli and create its own objective.⁶

Let us further solidify the understanding of AI by taking a purely hypothetical situation of a “paper clip maximizer”⁷: a machine that is programmed to make as many paper clips as possible. What if the machine became intelligent and, like a human, decided to create new paper clips by manufacturing machines to achieve its programmed goal and never stopped production? In this situation, humans may then intervene to create a governance framework to oversee the manufacturing process to ensure that only a million paper clips are created. However, what if, in turn, the paper clip machines decided to check their own work to ensure that they had correctly counted a million paper clips? To achieve this, the machines become self-teaching to make themselves smarter. The now smart and intelligent machines then create new raw-computing material to further check all aspects of the paper clip production that includes constraints (only 1 million paper clips are produced daily). But with each new check, the machines start to question what raw materials work best and so in response, the machines decide to create new raw materials. The result is that the now super-intelligent machines are mimicking every aspect of learning. They display features of intelligence and are thus essentially simulating what it is to be human.⁸

AI has already become deeply embedded in society and used by most of us daily, even if we are not aware. The wide availability of AI is a further tribute to the legacy of McCarthy. McCarthy did not leave us with a collection of theories and exploratory questions on machines and intelligence, but “list processing” (LISP), which was the language of choice for natural language processing (NLP) functions.⁹ To this day, LISP remains the standard AI language due to its ability to integrate LISP macros, which are a sequence of instructions that allow control over when and whether arguments are evaluated. More generally, NLP is used by programmers when an AI-enabled system interprets human language in the form of speech and text.¹⁰

Some great contemporary examples of LISP-enabled AI systems include the voice recognition technology used in iPhone's Siri or Amazon's Alexa that allow mobile devices to interpret essential elements of human language to return an answer.¹¹ LISP is also used in navigation systems and applications that instantly translate language. Interestingly, from 2016, speech recognition has had profound improvements and has an ever-decreasing error rate. It is also about three times faster than typing text on a mobile phone.¹² Currently, although LISP is used for keyword recognition and text to speech, it tends to be weaker on contextual analysis and informal expressions like “slang” that are used quite commonly in many spoken human languages.

Machine Learning

Machine learning was created by Arthur Samuel at IBM very early in the quest to develop AI. Samuel created a program that had the ability to learn parameters of a function for evaluating the position of checkers on a board during a game of play.¹³ Other programmers then created additional methods to enable self-learning by a computer, the most prominent of which is symbolic AI.¹⁴ Symbolic AI uses high-level symbolic representations of a problem that are human readable. The symbols are manipulated in an attempt to replicate human intelligence.¹⁵ One of the most successful methods of symbolic AI is so-called production rules that connect symbols to a relationship that shares close similarity to an IF–THEN statement in many programming languages. The system then processes the rules to determine if additional information is needed to solve a problem.¹⁶ However, the symbols can only be used when the input problem is definite.¹⁷ If any uncertainty to the problem exists, as is the case when making a prediction, then a neural network needs to be developed based on “fuzzy logic,” which is a type of stochastic search.

Coming back to the definition of machine learning, it is a subset of AI, and over many decades, has matured into a discipline to focus on solvable, practical problems using models and methods borrowed from statistics and probability theory.¹⁸

In essence machine learning refers to the ability of a machine to learn from data and to keep improving its performance without need of human interference.¹⁹ This means that in the process of a machine learning, a human does not need to continually dictate how to accomplish tasks, while the machine has the ability to utilize an automated self-learning/feedback loop. Machine learning is divided into three broadly defined categories of algorithms: (1) supervised learning, (2) unsupervised learning, and (3) reinforcement learning. Let us further look at each of these.

Deep Learning

The machine learning approaches described thus far have limitations in that they rely on a programmer to determine what is informative for making a decision that can achieve a preset goal.²⁸ This means that the machine learning algorithm is dependent on the programmer and thus programmer insight. Furthermore, the machine learning approaches and algorithms previously described have all demonstrated to reach a definable plateau in their performance as the amount of data increases.

This is not the case with a more scalable machine learning algorithm referred to as deep learning where performance relies on large amounts of data to build models more accurately.²⁹ Essentially, deep learning is performed when nets of neural networks that resemble layers are utilized for learning. Classes of deep learning include: (1) convolutional neural networks that analysts widely use for image recognition and classification, and (2) recurrent neural networks for sequence data like time series, text, speech, etc. The other side is that deep learning is not suited to smaller datasets used to develop traditional machine learning models.³⁰

The reason deep learning does not deteriorate in terms of performance as the size of data increases is because the algorithms are able to learn independently by identifying structures in data (Figure 3.1). It then focuses on parts of these structures to achieve a goal.³¹ The power of deep learning is derived by use of a computing model that is inspired by the human brain³² and imitates the most basic biological processes of a brain cell, called the neuron, that in the brain contributes to learning and memory. Let us take a second to explain the neuron to better appreciate the biology that underpins the artificial process of deep learning. The human brain contains 8.6 billion fundamental units of nerve cells called neurons³³ and these cells are highly specialized to transmit information.³⁴ The 8.6 billion neurons are crammed into about 1.2L volume of brain tissue and form a vast network in that small space.³⁵ Neurons are composed of a main body with very tiny extensions that branch off the main body called dendrites. An impulse transmits along dendrites to the information center of the cell body or to a dendrite of a neighboring neuron.³⁶

To mimic the complexity of the biological neuronal network of the brain and create a deep learning model, the artificial neuron works like a biological cell where it receives a set of inputs (x) or impulses, each of which is assigned a specific weight (w). The neuron then computes numerous functions (y) on each of the now weighted inputs.³⁷ Weights assigned are not arbitrary and derived by training a neuron, achieved by showing neurons a large number of training examples. These are iteratively changed to reduce error on the training examples.

A classical challenge is overfitting. It is possible that a neural network becomes overfitted where the model produced by the algorithm perfectly performs each time on the training sample but fails in practice. The issue of overfitting occurs when too many parameters have been used and not enough training data points.³⁸ The problem can be overcome by either adding more training samples that can run into the hundreds of thousands, sometimes millions, of data points,³⁹ or by reducing the connectedness between neurons.⁴⁰

The way in which the weighted inputs are computed depends on the type of activation function used. An activation function is a mathematical transformation that acts as the transfer function: it translates the input signal to the output signal. The more common activation functions used are:

• Identity functions that are linear. The value of the argument used for training is not changed and its range is potentially not bounded.
• Sigmoidal functions are S-shaped like logistic and hyperbolic tangent (or tanh) functions that produce bounded values with a range of 0 to 1, or –1 to 1.
• Softmax functions are multiple logistic functions and are a generalized representation of logistic regression that forces the sum of their values to be 1.
• Value functions are bell-shaped functions such as a Gaussian function.
• Exponential and reciprocal functions are bounded by 0 and 1.

Linear activation functions take a linear combination of each weighted input, while nonlinear activation functions like the sigmoidal function take a sum of the weighted inputs into a logistic function. For example, when learning the sigmoidal logistic function initially approximates an exponential equation. Please note that these intrinsic characteristics of the sigmoidal logistic function are not to be confused with machine learning optimization discussed in Chapter 8. Furthermore, with a sigmoidal neuron, the value that gets returned is between 0 and 1 and is dependent on the weighed input that gets summed. For instance, if the weighed sum is negative, then the returned value is close to “0.” Despite their limitation, sigmoidal neurons are used more often than linear neurons, as the generated algorithms are much more versatile in comparison.⁴¹

Neurons are also the building blocks of the entire nervous system of the body and receive sensory information from the external environment and send commands to muscles among many other known functions.⁴² The functionality of the neuron is extremely complex, and the complexity in turn stimulates active areas of research and development that could, at some point, further augment and drive new ways of approaching and applying deep learning along the road to replicate human intelligence.

Artificial Neural Networks

An artificial neural network in a deep learning algorithm is created in an analogous manner to a biological neural network in that its creation is dependent on many interactions: from the input data, between many neurons, and the output generated from the neural network's response. Deep learning algorithms are further complicated by nets of neural networks that resemble layers. Neurons at the input layer are connected to the actual output or answer to the problem and need not connect to each other, while neurons at the output layer are connected to one another and directly receive input data. The network can be feed-forward where there are no connections between neurons at the input to those at the output layer, or recursive where there are connections between input and output layers⁴³ (Figure 3.2).

There can also be a hidden layer that exists between the input and output layers. The hidden layer itself is where the wondrous power of the network solves problems and derive features of the input data that are extracted to learn and derive an output. Technically, the hidden layer involves an extra transformation of some type but requires no extra parameters. Interestingly, the hidden layer is where the “deep” aspect of deep learning gets derived—a neural network with many hidden layers is considered “deep.”⁴⁴ The most popular form of a neural network is a multilayer perceptron (MLP) that has any number of inputs, usually at least one hidden layer, and many neurons in each hidden layer. The MLP can have a different number of neurons in each hidden layer and has weights between the following:

• The input layer and first hidden layer
• One hidden layer to the next hidden layer when there is more than one hidden layer
• The last hidden layer and the output

Putting It All Together

To summarize the terms of reference⁴⁸ AI is referred to as the “superset” and was the first to be discovered in the 1950s. It has rapidly evolved and expanded ever since. Machine learning is a subset of AI and was created as part of our continuing quest to simulate human intelligence. Although deep learning was discovered soon after machine learning, its power and scalability had to wait for appropriate computing power.⁴⁹ Deep learning is an innermost subset, being a class of machine learning that utilizes layers of neural networks that enable complex tasks like computer vision.