Why Model Validation and Sensitivity Analysis Matter in HiMCM 2020 A Solutions

Many modeling papers lack validation, sensitivity analysis, or comparative analysis, which weakens the paper; validation proves model effectiveness and the author places examples, sensitivity analysis, and comparative analysis in the "validation results" section because they all relate to model credibility.

1 Validation Results

1.1 Example Illustration

Representative cases demonstrate that applying the model can partially validate it; the 2020 A problem requires teams to generate data and explain its use, effectively providing an example analysis.

1.2 Sensitivity Analysis

Sensitivity analysis examines how changes in uncertain factors affect evaluation metrics and includes single‑factor and multi‑factor approaches, though usually only single‑factor analysis is required.

Single‑factor sensitivity analysis examines the impact of varying one uncertain factor while keeping others constant, assessing its influence on economic evaluation indicators. Multi‑factor analysis considers simultaneous changes in two or more independent uncertain factors.

Sensitivity analysis is a common model verification method, often testing model weights. An excerpt from a top‑scoring 2020 A paper illustrates this process.

Our result shows that our model can make most suitable suggestions to the student based on the weights of their preferences. We also want to know how sensitive our model is, i.e., the stability of our job recommendation system under changes in the parameters...

Notice the black vertical line indicates the weight produced from AHP, and its intersection with other lines indicates the final scores for those jobs. Moving the line left or right creates different points of intersections that create different final scores...

The author performed single‑factor analysis on the wage‑rate weight, varying it from 0 to 1 while keeping other weights fixed, and observed score and ranking changes. Around the original weight (~0.6) the ranking shifts slightly, showing the top recommendation is robust but lower rankings are sensitive.

Further analysis of other factor weights makes the sensitivity study more comprehensive.

The value of sensitivity analysis lies in revealing whether core factors significantly affect results; small impact means results can be trusted, large impact requires careful factor selection.

Among five top‑scoring 2020 A papers, four included sensitivity analysis, highlighting its importance for model completeness.

1.3 Comparative Analysis

Comparative analysis compares results from different models to showcase differences or superiority. In modeling contests, usually only one final model is presented, so comparative analysis is less common than in academic papers.

2 Validation Results Are Often the Weakest Part of Modeling Papers

Reasons include lack of awareness, insufficient experience with the full modeling workflow, and time pressure during contests, leading to omitted validation.

Students focus on learning models and programming, neglecting validation.

They lack experience with the complete modeling process and paper writing.

Most contest time is spent building models, leaving little for validation.