Master Data Management

Organizations have access to copious amounts of internal and external data. However, one of the largest gaps faced by financial institutions large and small is the key concept of data catalogs. Many institutions attempt to fill this gap with spreadsheet-based descriptions of data-sets that quickly become outdated as new columns or rows are added, like new financial products. Automation enabled by AI and machine learning can help build data catalogs that include the ability to easily search for data and shared columns or observations across datasets. The search queries utilize natural language processing to improve ease of use. Organizations can build custom solutions using a programmatic approach or utilize vendor-provided applications with built-in AI technologies.

For example, a standard data catalog provides the following basic metadata for each dataset:

• Column details
• Row details
• Size of the data
• Completeness of the data (i.e., number of “missing” fields)
• How the data is accessed and where it is located
• Data security
• Purpose of the data (this can be manually added or automated)
• Data lineage and history of usage (i.e., how many reports, models, and jobs have been completed using the data, and the main users of the table)

Standardizing Datasets and Ensuring Data Quality

For AI and machine learning, risk management teams may decide that all the data profiling metrics generated today are enough.

We recall a very profound statement made by a prolific professor at a conference when challenged about the output results of a model: “Well, the data is the data!” This can be true. Data that is sourced and used in a risk model is “the data,” but importantly, it can be improved and cleansed using sophisticated cognitive data-quality controls. These measures ascertain the data with regards to its uniqueness and descriptive statistics such as average values and mean, median, mode, and standard deviation. When more sophisticated and repeatable techniques are used, that “little bit of guesswork” is removed and the data cleansing becomes less manual.

In practical terms, the data catalog information can be extended further, by descriptive data quality metrics and whether the data matches the expected distributions (e.g., if the catalog expects integer data but it is a string, then it will be recorded as a mismatch). Data-quality metrics are also extremely useful to provide details on the completeness of the dataset. These include, but are not limited to:

• Frequency of uniqueness, missing values, number of mismatches
• Metadata measures that provide information on columns, including its type, format, length, and candidates for primary keys
• Security scores that will automatically detect columns that contain personally identifiable or sensitive information
• Semantic types by combining data-quality algorithms and natural language processing to automatically generate metadata (e.g., the columns representing numeric values, identification numbers, and postcodes)

The challenges with developing, deploying, and managing AI are better addressed by automating manual data-quality tasks to assure good quality data for AI.

Leveraging cognitive data-quality methods is one way to achieve a level of automation in data validation processes. Cognitive data quality autogenerate the data quality and cleansing routines, thus reducing the need for manual effort. This code can be executed as part of robust and repeatable data preparation control plans. The methods can scan the data, either structured or unstructured, and suggest possible data-quality transformations.

Utilizing “Alternative Data”

Alternative data is said to be any data gathered from nontraditional data sources—for example, online or geolocation data, sensor data, social media, satellite images, or network data. Increasingly, data used to develop risk models is becoming a blend of traditional and alternative types especially in applications utilizing AI and machine learning.

Typically, traditional data sources are structured and stored in relational databases. In some instances, AI systems employ alternative data that includes unstructured data in the form of real-time transactional data, news feeds, web browsing, geospatial data and so on.

A common data challenge with AI approaches when utilizing larger data volumes is that the time history is not sufficiently long for back-testing purposes (e.g., data capture may have started recently).

The two data types used to build risk models, traditional and alternative, are summarized in Table 2.1. Before we progress to the next parts of this chapter, we wanted to take time to expand on the common steps for “alternative data” before it becomes useful for analytically driven insights and models. Commonly, unstructured “alternative data” must be structured into a more useful tabulated form, so that can it can be combined with existing data used for risk management purposes, like risk modeling. The process also standardizes the data and reduces its dimensionality. For the inclusion of personal data, the firm will need to implement an information management standard to help govern its use. Such a framework is based on customer or consumer consent, ethical considerations, and regulations, as well as the internal guiding principles of the institution. Furthermore, organizations should also comply with the principles of BCBS239 to “alternative data”, as done for the more traditional data.

Extending Risk Data to “Alternative Data” for AI and Machine Learning

Data preparation and its management for risk model development starts with the identification of the relevant data sources. In some cases, these sources need to be justified, which means that the reason for using the data must be evidenced as appropriate for the model to be built without causing bias or unfair outcomes (Figure 2.2).

Importantly, when risk modelers select data sources, there are key organizational processes and policies to be considered. Any such considerations will continue to apply whether AI or machine learning replace traditional risk models or when AI or machine learning augment traditional risk models. The considerations depend on the type of model. Regardless of whether it is a traditional or innovative risk model, it is always good to ensure that the data sources selected are appropriate and justified for the model's intended use. In this case, the intended use refers to the model objective, purpose, and the setting in which the model will be applied. For example, in risk departments, a range of model types are commonly used. A selected set of risk model types are listed and explained below, as well as examples of how risk modelers are utilizing alternative data sources. For example, alternative data from mobile phones and utilities helps lend to the unbanked and underbanked in many countries. This has a significant impact on access to credit for consumers and micro-entrepreneurs, and further helps to ensure fairer access to credit in countries that have historically disadvantaged groups:¹

• Decision models, such as application and behavioral scorecards. For decision models, risk modelers may use transactional data as a source of alternative data to predict short-term and immediate events like loss of income. By enriching existing data with alternative sources, a more nuanced view of credit risk can be achieved, not only using banking data, but also telecommunications data like subscription data. For more information on the examples, refer to Chapter 3.
• Credit risk parameter estimates for probability of default, loss given default, and exposure at default. Risk parameters are widely used for regulatory capital, provisions, stress testing, and other internal calculations. Risk modelers are utilizing location data from property intelligence platforms and map applications to develop location scores. Risk modelers then use these scores to improve prediction of probability of default (PD) and loss given default (LGD) models for consumer loan portfolios and loans to small and medium enterprises (SMEs).
• Stress testing. These are forward-looking analyses to assess how adverse scenarios affect the resilience of a financial institution's balance sheet. This is normally done by stressing macroeconomic variables like GDP (gross domestic product), unemployment rates, and house price indices. A financial institution may take its individual portfolios and granular risk factors and stress its risk parameters using internally defined scenarios. Analysis can be applied to extract insights from news articles and social media to assess sentiment on emerging risks such as geopolitical uncertainty. For climate risk assessments, scenario-based analytics utilize climate scenarios such as those provided by the Network for Greening the Financial System (NGFS).

Synthetic Data Generation

With the use of AI and machine learning and the availability of additional compute power, the use of synthetic data is rising. Often, the available data used for training AI and machine learning has data limitations, such as gaps in the time series, insufficient data history, or in the level of granularity. Simulated data can help overcome these gaps. The methods that are typically applied range from bias adjustments (e.g., the fuzzy method for reject inferencing), repeat sampling (bootstrapping), and synthetic minority oversampling (SMOTE) to those that generate market states like restricted Boltzmann machines.²

Some AI and machine learning methods intrinsically generate their own data (generalized adversarial networks, variational auto-encoders). These methods can help stress test models and therefore improve the robustness of models as part of model validation. While these apply to high-dimensional data, it bears noting that by simulating data based on either an algorithmic approach or approximation, additional model risk can be introduced.

Typical Data Preprocessing, Including Feature Engineering

As described previously, AI and machine learning are increasingly leveraged in risk management to augment traditional risk models, because it realizes the long-needed risk transformation to automate mundane tasks and to process large volumes of data.

Getting the data correct for AI and machine learning typically involves standard tasks. Several of these have been covered, such as data quality, cognitive data quality, and extending risk data to “alternative data”. The risk model lifecycle also routinely involves feature engineering and dimensionality reduction.

Feature engineering helps to automatically build a range of features from input data. By using simpler features, AI and machine learning become easier to interpret and, possibly, to maintain over time. Well-engineered features also reduce the effort needed to optimize parameters for machine learning. Feature engineering is also important because high-dimensional data often observe sparsity in the data points as the number of inputs grow. This means that as more features are added, the feature space increases.

Certain machine learning algorithms are more sensitive to outliers, meaning that the output model is skewed toward the outlier population. Outliers are not easy to identify in a dataset and take time to interpret. Even well-trained analytics experts require time to explore the data for outliers and to then determine what to do with them. Are they real or due to a bona fide error? Outlier populations can be present in the data used for development of AI and machine learning due to system errors in data, for example, when updates occur, or files of data are transferred between different systems, or due to data entry errors caused by misinterpretation of internal policy, operational processes, and procedures.

It is also important to explore the relationships that exist between outlier populations, as these linked relationships can be helpful to interpret the outliers within, and between, datasets.

There are robust analytical techniques that are helpful to identify outliers. One way includes the use of Principal Components Analysis (PCA). Another unsupervised technique is the use of T-SNE data visualizations. The clusters or “outliers” can be further studied to determine if they are real sub-populations that could be separately modeled, or are outliers due to errors in the dataset.