Event-driven Model Monitoring
Summary
Model Management Standards issued by regulatory bodies such as the Federal Reserve Board (SR11-7) and the CBUAE, emphasize the importance of regular performance monitoring, suggesting that the frequency of assessments should be determined by the potential risk posed by each model. The guidance suggests that monitoring activities should be determined by the model’s materiality, complexity, and the rapidity with which its operating environment is changing. Therefore, for models, including those based on AI that are used for high-stakes or rapidly-changing environments, the recommendation is for more frequent, possibly continuous, monitoring.
In this video, we will demonstrate how to:
- Initiate an automatic monitoring workflow when new monitoring data arrives
- Inspect monitoring scripts interactively
- Compare KPIs against a benchmark model
- Generate findings and action plans when metrics fall below defined thresholds
