(This diagram illustrates the different stages in the model lifecycle, where each stage requires sharing of accurate documentation that can be challenging when done manually.)
What challenges do financial services firms face?
Jos identified several factors firms should consider as part of their model documentation process:
- Handovers across various stages within the model lifecycle necessitate constant documentation updates. Ensuring consistency is challenging when everyone has their own documentation style.
- Multiple stakeholders require information; without a comprehensive understanding of model documentation from different stages, it’s difficult for teams to perform their tasks effectively.
- Numerous dependencies within a model may affect the model itself, emphasizing the need for a systematic approach to content and documentation management.
- Model documentation is often manual, making it difficult for firms to systematically track and update documentation. Outdated documentation that no longer accurately represents the model can create operational risks.
Challenges in model documentation often arise from inconsistencies in the produced documentation, as well as varying quality, and outdated documentation. Inconsistencies in documentation stem from several factors. Often multiple documents per model have to be created: per geography (SR11-7 vs. CP6/22), and per stakeholder type (e.g., credit officer vs. internal model review team). Furthermore, model dependencies have to be captured accurately, which has become increasingly challenging with the rapid deployment and adoption of AI models.
The level of detail and quality of model documentation often varies significantly, particularly when outsourcing model development or validation activities. The author’s written language skills and the time they have available will also influence the quality of the documentation. Another important challenge is that model documentation is often outdated when content is not captured dynamically. This frequently happens when documentation is generated manually. An automated document refresh should be triggered, such as by model monitoring, recurring validation, model version upgrades, etc.
Last but not least, model documentation is a tedious, manual process.
Jos elaborated, “Models are continuously developed, monitored, and validated, requiring users to regularly update the model documentation. In practice, manual documentation can be labor-intensive, and people often overlook detailing all changes, leading to future problems.”
The Role of Technology
During his presentation at the Advanced Model Risk USA, Jos showcased how technology can help manage and streamline the model documentation process, enabling financial services firms to save costs and maintain compliance. He explained that 80% of elements found in the documentation are shared, allowing for standardization.
“By leveraging technologies such as the Yields MRM Platform, firms can manage individual content elements separately, addressing consistency issues. This standardization enables firms to automate processes, resulting in significant efficiency gains, which reduces the risk of non-compliance and ensures up-to-date model documentation whenever a model undergoes changes.”
Would you like to learn how a G-SIB leveraged the Yields MRM Platform to automate model testing and documentation? Explore our latest case here.
