Automated End-to-End Model Iteration in Intelligent Risk Control Systems

The talk introduces the concept of intelligent risk control, which combines multiple technologies to make decisions across business stages and emphasizes the need for a fully automated engineering pipeline to improve precision and efficiency.

It outlines the hierarchical structure of an intelligent risk control system, from data acquisition and feature development to model training and decision-making, focusing on the model layer for automation.

Various risk models are described for pre‑loan, in‑loan, and post‑loan stages, highlighting the challenges of manual iteration and the motivation to build a generic framework for automated model updates.

An end-to-end automated model iteration workflow is presented, covering sample selection, feature processing, model training, evaluation, and deployment, with references to existing modeling tools that handle feature selection and training.

The architecture design is divided into three parts: functional modules, databases for data exchange, and external services. Modules are executed sequentially, with dependencies between them, and the workflow is split into three phases: sample/feature acquisition, data preprocessing and model training (supporting both Python and Spark), and model deployment, evaluation, and monitoring.

Sample selection is made flexible through configurable parameters, supporting three dimensions: customer segment filtering, Y‑label definition, and time‑window strategies (fixed rolling window or expanding window).

A modular feature store is built, separating internal and external data sources, with mechanisms for online updates and offline T+1 calculations, allowing selective inclusion of new features via configuration.

Feature preprocessing includes missing‑value imputation with distinct sentinel values, rule‑based hard filtering, and metric‑driven selection using IV, KS, missing rate, or tree‑based importance.

Multiple algorithms (tree models, XGBoost, LightGBM, logistic regression) and hyper‑parameter optimization methods (Bayesian optimization, random search) are combined to generate diverse model versions.

An automated evaluation system assesses models on three dimensions: effectiveness (KS, AUC), monotonicity (bucket reversals and severity), and stability, using ranking‑based scoring to normalize metrics.

Model scoring aggregates weighted scores from each dimension to rank models, with an example showing an offline model outperforming an online one due to better monotonicity and stability.

Deployment ensures hot‑loading of the selected model, automated testing of data pipelines, and consistency checks between offline and online scoring, including PSI monitoring and alerting.

The automation dramatically reduces iteration cycles from weeks of manual work to daily updates, maintaining model performance and preventing degradation, while also opening possibilities for automating rule extraction and threshold setting.

The presentation concludes with a review of the framework, challenges of system integration, and future directions such as extending automation to rule‑based systems.