In recent years, more and more organisations are rapidly integrating predictive models across different business functions. Model use offer many benefits, but it also exposes organisations to greater operational risk.
Model risk is the risk that arises when a model fails or does not perform according to expectations.
When we use mathematical models to support decision making, we face model risk since mathematical models can produce incorrect outcomes.
Models can fail due to technical mistakes, such as lack of mathematical rigour, data issues, implementation bugs, and other issues and these types of model risk incidents can be considered part of operational risk. However, models produce incorrect results as well simply as a consequence of the inherent uncertainty related to mathematical modelling. This in itself is a characteristic feature of modelling but a failure to recognize this uncertainty properly constitutes a key element of model risk.
A final category of model risk accidents is related to using models that are not well understood, i.e. in situations when we are unable to properly explain why a model has come to a certain conclusion. In the latter case, there is a link with operational risk.
Model failure can lead to financial losses, regulatory or legal penalties, or damage to a bank’s reputation, resulting from the use of a model that contains data or assumptions that were not appropriate for the model.
In order to manage model risk, governance policies and frameworks should be applied.
The cornerstone of model risk governance is the implementation of three lines of defence:
- first line (the model developers) is responsible to persist in a structured document all assumptions, both on the data and on the actual model. Once a model is used in production, it is most often a first line’s responsibility as well to monitor the model to detect issues as quickly as possible.
- second line (the model validators) has to independently review this document to ensure that it is self-contained, that the results are reproducible and that the limitations of the models are well understood by the decision-makers, the users and the developers. This independent review consists of model validation.
- third line (audit) is responsible for verifying that the processes between first and second line are effectively implemented.
Two examples of model risk are the Credit Crisis of 2008 and the London Whale Trading Incident in 2012.
In the Credit Crisis, banks were relying on very simple models to estimate mortgage risk. Although the used models were mathematically correct, they were too simple to deal with the highly complex derivatives that were traded at that time.
In the London Whale Trading Incident, JPMorgan lost over $6 billion due to a mistake in a credit risk model.
In both cases, the root cause was a lack of proper governance that should have detected the flaws in the models that were being used.
