Model monitoring efficiency

The Scoring Center at BNP Paribas Personal Finance is responsible for the development, deployment, and ongoing monitoring of credit risk scoring models. These scoring models help the bank to evaluate the creditworthiness of clients and predict the likelihood of default on credit obligations. By leveraging credit scoring models, BNP Paribas Personal Finance effectively mitigates the risk of default and minimizes potential losses, enabling efficient capital allocation, and adherence to regulatory requirements. To ensure that the performance of the credit risk models meets expectations consistently, the models need to undergo periodic monitoring.

The central teams orchestrate and manage ongoing model monitoring activities in 20+ entities. BNP Paribas Personal Finance was looking to optimize the effectiveness of its monitoring and data analysis tools.

One of the key issues was the fact that data ownership was fragmented, which required data to be passed on to central teams for aggregation, validation, testing, and eventually monitoring reporting. Local teams were not able to actively contribute to the process.

There was also a need to ensure that certain machine learning-based models were fully covered by monitoring tools.

Every year, up to 10 people, or one quarter of the central team, used to spend about 2 months on monitoring tasks, resulting in a high burden on the team. Additionally, BNP Paribas Personal Finance was concerned that monitoring some ML-based models (e.g.: fraud detection) wasn’t feasible with the outdated legacy systems they were using.

Another major challenge BNP Paribas Personal Finance faced was the lack of widespread access to a comprehensive model inventory that could dynamically track all model information, including text content.

To standardize and scale monitoring activities while remaining compliant with regulatory standards, BNP Paribas Personal Finance began searching for a new technology solution, favoured vs the development of an in-house solution. As a specialized model risk management solution, Yields responded to the vendor selection process. A determining factor turned out to be Yields’ modular architecture and flexible design that facilitates integration with existing technologies at BNP Paribas Personal Finance. This aspect is illustrated by the fact that the Yields platform now interacts with BNP Paribas Personal Finance’s primary model inventory and also feeds data into BI tools for further analysis and reporting. 

A second important feature driving the selection of Yields was the availability of customizable computation infrastructure to quantify the model performance during the monitoring process. A third significant added value was the flexibility it provides to configure and update the solution in accordance with changing requirements over time. 

By utilizing the Yields platform, the risk management team at BNP Paribas Personal Finance has implemented repeatable workflows to organize and execute the periodic model monitoring campaign across all concerned local entities. The process automatically triggers standardized tests and generates model monitoring reports in accordance with the BNP Paribas Personal Finance templates. During a monitoring campaign, the central team has a clear overview of the status of the processes across all concerned local entities.

Model findings are created and followed up within the platform, and the model risk teams have direct access to inspect the details and to create action plans.

The solution fulfills the requirements for collaboration between local entities and the central team to ensure that all communications and notifications are now managed within the Yields platform. With the ability to run monitoring processes on the platform, the responsibility for validating data and generating model monitoring reports has been shifted to local entities. They now have the ability to carry out additional controls outside of regular follow-up campaigns.

During the model monitoring process, the solution builds up a full audit trail that serves as evidence to auditors and regulators.

Throughout the implementation project, BNP Paribas Personal Finance and Yields’ product and customer success teams worked closely together. BNP Paribas Personal Finance appreciated the involvement and flexibility of the Yields team as well as the integration of custom features into the product roadmap.