How Causal Inference Can Unlock High‑Impact Product Requirements

Introduction

Product managers must identify valuable demand directions amid complex business contexts, which requires not only experience but also systematic causal inference to uncover the root of problems.

1. Wallace Scientific Loop in Business Scenarios

1.1 Wallace Scientific Loop

The 1971 Wallace scientific loop describes the cycle "theory‑hypothesis‑observation‑generalization or testing‑new theory". Researchers start from existing theory, deduce hypotheses, design observations to test them, then analyze results to accept or reject the hypothesis, ultimately forming new theory. The deductive phase moves from theory to observation, while the inductive phase moves from observation back to theory.

1.2 Product Demand Hypothesis and Validation Process

In fast‑moving commercial environments, hypotheses originate from business experience, data, or user feedback rather than abstract theory. Managers must surface timely opportunities from their own data, apply causal inference to filter plausible hypotheses, and use experiments only after a strong evidential foundation is built.

Example: Users who have enjoyed a certain JD service show higher purchase frequency and GMV. The naive hypothesis is that promoting the service will boost GMV, but questions remain about causality and profitability, requiring causal analysis before costly experiments.

The overall workflow: start from business experience, use causal inference to select valuable hypotheses, validate through experiments, and continuously refine knowledge.

2. Causal Inference and Its Methods

2.1 Definition and Common Pitfalls

Causal relationship means a change in an explanatory variable X leads to a change in outcome Y, holding other factors constant. Two sub‑questions: which is cause and which is effect, and how large the effect is.

Example (Simpson’s paradox) shows that without controlling for activity level, the aggregate data misleadingly suggests the service reduces GMV, highlighting the need to control confounders.

Randomized controlled trials are ideal for eliminating confounders, but when infeasible, observational data requires econometric methods.

2.2 Econometric Methods

2.2.1 Matching

Principle: compare treated units with untreated units that share the same observable characteristics. Assumptions include conditional independence and common support. Methods: direct matching or propensity‑score matching, with steps for estimating scores, balance checking, and effect calculation.

2.2.2 Difference‑in‑Differences (DID)

Uses pre‑ and post‑intervention panel data for treatment and control groups. Effect = (post‑treatment – post‑control) – (pre‑treatment – pre‑control). Assumes parallel trends and common support.

2.2.3 Regression Discontinuity Design

Exploits a continuous running variable with a cutoff that determines treatment assignment, creating a quasi‑experimental situation near the threshold.

2.2.4 Instrumental Variable (IV) Method

Introduces an instrument Z that affects the explanatory variable D but is independent of the error term, allowing isolation of the causal effect of D on Y. Requires relevance and exogeneity of the instrument.

3. Practical Case

Because suitable instruments are hard to find, we combine Propensity Score Matching (PSM) with DID (PSM+DID) to assess the impact of a service on subsequent GMV using historical non‑experimental data.

Step 1: Use PSM to create a comparable control group for users who did not experience the service, balancing on purchase frequency, membership status, page depth, etc.

Step 2: Apply DID on the matched panels to compute the causal effect: first‑difference (pre‑trend), second‑difference (post‑intervention), then the DID estimate.

Step 3: The final estimate quantifies the service’s contribution to future GMV.

4. Conclusion

With limited resources, product managers must avoid subjective guesses and rely on systematic causal inference to improve decision quality. By continuously extracting insights from data, validating hypotheses with robust methods, and iterating knowledge, managers can grow professionally and deliver higher‑impact products.