GeoReward: Enabling Vision‑Language Models to Sense Market Context for Country‑Specific Ad Creative

Problem

Vision‑Language Models (e.g., Qwen2‑VL, InternVL) achieve strong performance on visual QA, captioning and reasoning, but when used for cross‑market ad preference prediction they ignore the “country” variable. A fine‑tuned Qwen2‑VL‑7B selects the same image for France and Korea despite real CTR differences. This failure is named Contextual Variable Overestimation (CVE).

Mathematical analysis of CVE

In the auto‑regressive factorisation the self‑attention distributes most energy to high‑capacity tokens (product description, image patches). The sparse country token (1‑2 tokens) receives little attention, effectively marginalising the variable in the posterior distribution.

MACP benchmark

Multi‑Country Ad Click Preference (MACP) contains 823 k training and 180 k test samples from ten countries (BR, CL, ES, FR, KR, JP, US, MX, AU, SA). Each sample provides product text, two candidate ad images, and the target country. Preference labels are defined by CTR differences ≥10 %; samples with lower differences are filtered.

Evaluation metrics

Accuracy : proportion of predictions matching the higher‑CTR image.

Sensitivity : joint probability that the model predicts correctly for the same product across any two different countries, measuring attention to the country token.

GeoReward framework

GeoReward introduces three cascaded gates operating on data, feature and gradient levels:

Retrieval Gate (MA‑RAG) : before training or inference, retrieve market‑specific historical preference records and embed them as a prior.

Consolidation Gate (CGVM) : a lightweight adapter uses the country embedding to generate scale and shift vectors that modulate visual token representations.

Sensitivity Gate (SSL) : a selective sensitivity loss adds a penalty proportional to the attention deficit on country, product and image tokens only when the prediction is wrong.

Experimental results

On the MACP test set GeoReward reaches 60.37 % accuracy and 40.84 % sensitivity, surpassing the best baseline (Qwen2‑VL‑7B + FC head, 55.60 % / 36.73 %). Ablation studies show performance drops when any gate is removed (e.g., without MA‑RAG: 56.81 % / 37.40 %).

Using GeoReward as a reward model, a Design Generation Model (DGM) based on Qwen2‑VL‑7B is fine‑tuned with Direct Preference Optimization (DPO). After DPO, GeoReward scores improve from 56.04 % to 59.60 %, and generated ads exhibit country‑specific backgrounds (Paris streets for France, wooden interiors for Korea, tropical beaches for Brazil).

Conclusion

The work formalises CVE, releases the MACP benchmark, and demonstrates that the three‑gate GeoReward architecture mitigates CVE, improves cross‑market preference prediction, and enables end‑to‑end country‑aware ad generation.

References

[1] Bayes, T. and Price, R. An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 53:370–418, 1763.

[2] Wang, P., Bai, S., Tan, S., et al. Qwen2‑VL: Enhancing vision‑language model’s perception of the world at any resolution, 2024.

[3] Rafailov, R., Sharma, A., Mitchell, E., et al. Direct preference optimization: Your language model is secretly a reward model. NeurIPS, 2023.