Causal Analysis: Challenges, Methodology, and Practice at Beike

The talk begins with an overview of causal inference, highlighting its broad applications—from climate change to drug discovery—and emphasizing its importance for AI‑driven internet industries.

Three core challenges are identified: (1) mistaking correlation for causation, (2) the presence of confounding variables, and (3) selection bias in samples or experiments.

To address these, Judea Pearl’s three‑level causal framework is presented: association (discovering relationships), intervention (predicting effects of actions), and counterfactual (asking what would happen if the cause did not occur). Classic examples such as the Nobel‑winning work on causal inference, the chocolate‑Nobel award paradox, and the smoking‑lung‑cancer debate illustrate each level.

Beike’s practical implementation focuses on a smart client‑management tool aimed at improving real‑estate transactions. The product’s value is evaluated through two experimental designs:

Scheme 1: grouping agents by tool usage frequency (high vs. low) and comparing average transaction volumes.

Scheme 2: randomizing stores across ten cities and using a difference‑in‑differences approach on per‑store transaction metrics.

Results from Scheme 1 showed a 25% uplift for high‑usage agents but revealed confounding factors (more skilled agents tend to use the tool more). Scheme 2, with randomization, demonstrated a modest 2.5% increase in transactions, supporting the tool’s effectiveness while mitigating bias.

Further analysis traced the tool’s impact pathway: increased agent effort on high‑value clients, higher “show‑up” rates for these clients, and ultimately greater transaction volumes for the top client segment.

The presentation concludes with a summary of the causal analysis workflow, reflections on experimental design choices, and a Q&A covering causal graph construction, statistical significance, and mitigation of selection bias.