Why Model Validation and Sensitivity Analysis Matter in HiMCM 2020 A

1 Validation Results

Many modeling papers lack validation, sensitivity analysis, or comparative analysis, which weakens the paper. Validation proves model effectiveness. Examples and sensitivity analysis are placed in the "validation results" section because they confirm model credibility.

1.1 Example Illustration

Representative cases demonstrate model application, fulfilling the requirement of HiMCM 2020 A to generate data and explain results. Interested readers can refer to the analysis article.

1.2 Sensitivity Analysis

Sensitivity analysis examines how changes in uncertain factors affect evaluation metrics. It includes single‑factor and multi‑factor analyses, though most competitions only require single‑factor analysis.

Single‑factor analysis studies the impact of varying one uncertain factor while keeping others constant, revealing its influence on economic evaluation indices. Multi‑factor analysis considers simultaneous changes of two or more independent factors.

In the award‑winning paper 11135, the authors altered the weight of the wage‑rate factor while holding others constant to identify the most critical factor (Absolute‑Top) that changes the top‑ranked job with the smallest weight adjustment.

Our result shows that our model can make most suitable suggestions to the student based on the weights of their preferences. ... We also want to know how sensitive our model is, i.e., the stability of our job recommendation system under changes in the parameters.

Notice the black vertical line indicates the weight produced from AHP, and its intersection with other lines indicates the final scores for those jobs. Moving the line left or right creates different points of intersections that create different final scores.

The authors performed single‑factor analysis on wage‑rate, varying its weight from 0 to 1 while keeping other weights fixed. The ranking changes only around the original weight (~0.6); the top recommendation remains stable, but lower rankings shift.

Further analysis of other factor weights provides a more comprehensive sensitivity study.

Sensitivity analysis reveals whether core factors significantly affect results. Small impact suggests results are reliable; large impact indicates the need for careful factor selection.

Four of the five top‑award papers for 2020 A included sensitivity analysis, highlighting its importance for model completeness.

1.3 Comparative Analysis

Comparative analysis contrasts different models to showcase distinct characteristics or superiority. While competitions often settle on a single final model, academic papers frequently employ this analysis.

2 Validation Results Are Often the Weakest Part of Modeling Papers

Reasons include lack of awareness, insufficient experience with the full modeling workflow, and time pressure during contests.

Students focus on learning models and programming, neglecting verification.

They lack experience with the complete modeling process and paper writing.

Most contest time is spent building models, leaving little for validation.

Download the referenced papers by replying “2020A11135” in the public account chat.