Prioritizing Code Redline Scan Bugs Using Regression Models and Feature Engineering

In the code redline scanning workflow, numerous bugs and warnings are reported, but developers often ignore the default priority order (red, safe, block, serious, risk, warning, style, suggestion, normal), leading to inefficient triage.

To help developers quickly locate the bugs that must be fixed, the authors propose using past fix histories to predict the likelihood of a bug being repaired and then re‑rank new scan results accordingly.

Goal and Evaluation

The objective is to predict the repair probability of each scanned bug (a regression problem) and sort the list by this probability. Common regression metrics such as explained_variance_score, mean_absolute_error, mean_squared_error, and r2_score are used for evaluation, with special attention to the error between predicted and actual 0/1 labels.

Feature Selection

Based on observed data, the following features are chosen: svn_path (repository path), svn_file (file path), error_id (bug title), msg (description), cat (bug type). The target label is status (0 = unfixed, 1 = fixed).

Data Cleaning

Dirty data arising from version changes and business logic is addressed: duplicate records with inconsistent labels, severe class imbalance, temporal sample drift (e.g., early C++/SVN data vs. later Java/Android/Git data), and mandatory‑fix categories (red, serious, safe, block) are all normalized.

Feature Quantization

Categorical fields with limited vocabularies (error type, project) are vectorized using a dictionary and CountVectorizer. High‑cardinality path features are encoded via hashing and SimHash after segment‑wise accumulation (e.g., "/A/B/C" → ["A", "AB", "ABC"]).

Model Selection

A gradient‑boosted decision tree (sklearn GBDT) is selected for regression.

Hyperparameter Tuning

Parameters are explored using a coarse‑to‑fine grid search with cross‑validation ( cross_val_score) to find the optimal configuration.

Model Evaluation Results

explained_variance_score: 0.9689 mean_absolute_error: 0.00207 mean_squared_error: 0.00167 r2_score: 0.9689 Average error for positive samples: 0.00103 Average error for negative samples: 0.01927

These metrics demonstrate that the model can reliably estimate the repair probability of each bug, enabling an effective re‑ranking of redline scan outputs.