Long-term impact

DataFunTalk

Dec 14, 2023 · Fundamentals

Evaluating Long-Term vs Short-Term Effects in A/B Experiments

While A/B testing is widely used for data-driven decisions, short-term experimental results often diverge from long-term impacts, leading to misguided strategies; this article examines why such inconsistencies arise and reviews major methods—including extended experiments, holdout groups, post‑analysis, CCD, and surrogate‑metric modeling—to reliably estimate long‑term effects.