Model risk management - Model identification and model risk classification

As of 17 May 2024, compliance with these five principles is required for all UK-incorporated banks, building societies and PRA-designated investment firms that have received internal model (IM) approval, and those looking to apply for such permissions. The PRA considers that those firms that don’t have IM approval may find the proposed principles useful and are welcome to consider them to manage model risk. These principles form a sound basis and good starting point for the management of model risk for all firms.

This article is the first in a series that will examine each principle in isolation, with the intention of analysing incoming changes and providing evaluation points relating to the practical changes that firms should consider when developing and implementing their MRM frameworks (for IM approved models and other models not subject to IM approval).

The core tenet of this principle is for firms to define and classify internal models to set a scope for the firm’s MRM. The regulator elaborates on this requirement in three sub-principles:

Holistically analysing these sub-principles and considering the extent to which your firm has already started to implement relevant policies and procedures will help to ensure compliance with the PRA’s expectations for MRM.

What firms should be doing to effectively implement this principle?

For firms to effectively implement this principle, it is essential to prioritise:

1. Defining ‘model’ for the purposes of MRM – this requires a scope that captures all quantitative methods used by the firm but needs to be specific enough to distinguish between models and non-models. The firm’s proposal for model definition must enable users to make this distinction.
2. Creating a model inventory – a living document, designed to be the sole source of truth for all models across their lifecycle.
3. Classify models based on risk – this should be done by first assessing the characteristics specified by the PRA (quantitative exposures, customer impact, influence on business decisions, data quality, etc.), as well as any firm-specific characteristics that may contribute to the materiality or complexity of an individual model’s risk.
4. Seek out external assurance and advice in instances where the firm does not possess sufficient expertise or knowledge to confidently implement aspects of the first principle.

1.    Producing a Compliant Model Definition

The PRA provides a general definition of a model in SS1/23:

“A quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into output. The definition of a model includes input data that are quantitative and/or qualitative in nature or expert judgement-based, and output that are quantitative or qualitative.”

Although the PRA’s definition represents an effective starting point for firms, the definition is quite broad. The regulator has consistently referred to the need for firms to implement the MRM principles in a manner proportionate and appropriate to their size, complexity, risk profile, and business model - firms are required to produce their own definition of models.

Based on our discussions with various market participants, it is our view that all quantitative methods that provide a final output on the firm’s ability to identify, measure, monitor, mitigate and report on regulatory risk, financial risk, and pricing risk, should be identified as a model.

 This means that a typical inventory (including models not requiring IM approval) would include those models that enable:

• Quantitative analysis of credit, market, or operational risk
• Calculation of regulatory capital, Credit Value Adjustments (CVA), and liquidity levels
• Financial accounting, pricing of products
• Detection of financial crime and market abuse.

In addition to the identification and classification of models, the PRA expects IM firms to evaluate their “deterministic quantitative methods such as decision-based rules or algorithms that are not classified as a model” but “have a material bearing on business decisions and are complex in nature.” The PRA suggests that firms should make a judgement on which aspects of the MRM principles are relevant to these quantitative methods and apply them. For example, the firm may include them within the model inventory and integrate them within the firm’s MRM governance. Examples of these deterministic methods provided by the regulator include complex electronic trading systems, financial crime monitoring systems, and anti-money laundering systems. 

2.    Maintaining an Effective Model Inventory

The PRA expects IM firms to possess a 'comprehensive model inventory'; a living document that lists all models used by the firm. A well-maintained and compliant model inventory should contain key information relating to each model’s application and be used throughout the model lifecycle (i.e., ‘implemented’, ‘under development’, or ‘decommissioned’). 

It is important for firms to recognise that a model inventory is required to be much more than a list of models. It should be akin to a work-flow management tool – which should contain information from as early as the stage of model ideation – until as late as evidence of the final signoff on post-validation remediation and model approval.

Several benefits emanate from the maintenance of a compliant model inventory:

• Supports the identification, measurement, and management of model risk.
• Represents a source to accurately report model risk.
• Facilitates the identification of inter-dependencies between models.

These benefits will enable effective management of risk throughout the model lifecycle.

3.    Implementing Representative Model Tiering

Firms should use a risk-based approach when implementing this principle, and a key aspect of this is generating representative risk-based model tiering. Models should be categorised based on the level of risk associated with them, this information can then be used to determine the level of MRM resources devoted to scrutinising their operation. Effectively, riskier models should be subject to more controls and more frequent validation than lower-risk models. The risk-level of a model is driven by its materiality and complexity. 

The tiering system adopted should assess the materiality and complexity associated with models and assign them an appropriate tier. The PRA suggests that materiality should consider quantitative measures as well as qualitative factors, such as exposure size, book value, number of customers affected and the extent to which business decisions are influenced. 

Complexity should be assessed in terms of:

• Data models used, including the use of alternative and unstructured data.
• Methodology choice, including the technological complexity of the model construct.
• Difficulty explaining the purpose of the model in non-technical terms.
• Difficulty recomputing/reperforming for validation purposes.

The breadth of risk tiers is at the firm’s discretion, but the regulator expects a logical and consistent approach to be implemented across the board. Validation of the tiering approach should be performed on a regular basis, while individual assignments to tiers should be independently reassessed as part of this process.

Get in touch

To speak to our prudential risk experts about model risk management principles, get in touch using the button below.

Contact us today