Causal Inference with Propensity Score Matching for Marketing Campaign Value Evaluation

The article begins by highlighting common pitfalls of mistaking correlation for causation and introduces causal inference as a scientific approach to uncover true cause‑effect relationships, using randomized A/B tests as the gold standard.

When random experiments are impractical—such as evaluating a Ctrip summer marketing activity—the authors propose using Propensity Score Matching (PSM) to adjust observational data for confounding variables.

PSM works by estimating each user’s propensity to receive the treatment (participate in the campaign) via a supervised classification model, typically logistic regression, and then matching treated users with control users who have similar propensity scores.

The paper details the practical steps taken: selecting a candidate control pool, designing 104 features across demographics, consumption ability, and travel intent, cleaning and standardizing the data, training a logistic regression model (precision 75.22%, recall 65.40%, AUC 0.79), and addressing class imbalance through down‑sampling.

Matching is performed using a 1‑nearest‑neighbor algorithm with a distance threshold of 0.001, achieving balanced propensity score distributions (KS‑stat = 0.00057, p‑value = 0.9771). Effect‑size metrics (Cohen’s d and w) confirm that all covariates are well balanced after matching.

With balanced groups, the authors compare average repeat‑purchase revenue, finding a statistically significant uplift (T‑stat = 35.21, 99% confidence) and compute the total incremental value of the campaign.

Robustness checks—including adding random variables, placebo tests, and random sub‑sampling—support the validity of the causal assumptions.

Finally, the article concludes that PSM offers a viable solution for value estimation in scenarios lacking randomization, while acknowledging that other causal methods (difference‑in‑differences, regression discontinuity, synthetic control) remain promising future directions.