Mastering Difference-in-Differences: Theory, Example, and Python Implementation

Problem Context

Suppose I want to understand the economic impact of a national policy (e.g., a pandemic control measure). How can I test this impact?

A natural idea is to compare regions that implemented the policy (treatment group) with those that did not (control group). However, pre‑existing differences often make simple comparisons unreliable, so we also need to compare before‑after differences for both groups.

This idea can be implemented statistically as the Difference-in-Differences (DiD) method.

Difference-in-Differences

DiD is a widely used statistical technique in economics, social sciences, and public health to estimate the effect of a policy, intervention, or event on one or more groups. Its core idea is to compare differences between groups over time and the changes before and after the intervention.

Specifically, DiD uses two time points—pre‑policy and post‑policy—along with a treatment group (exposed to the policy) and a control group (not exposed). Researchers first compare the treatment and control groups at each time point, then compare the differences across time to obtain the policy effect.

The method’s advantages include addressing many confounding factors without a randomized experiment, though it is sensitive to group selection and assumes parallel trends.

Concrete Example

Imagine evaluating a city’s “restricted traffic” rule on congestion. The rule limits certain license plates during specific hours. We measure traffic flow and speed before and after the rule, using the treated area as the experimental group and an untreated area as the control.

We compare traffic flow and speed before (e.g., 2020) and after (e.g., 2021) the rule. If the treated area shows a significant reduction in flow and higher speed while the control shows little change, we conclude the rule reduced congestion.

Mathematical Model

The DiD model can be expressed as:

Y_it = α + β1*Treatment_i + β2*Post_t + β3*(Treatment_i*Post_t) + ε_it

where Y_it is the observed outcome for unit i at time t , Treatment_i is a binary indicator for being in the treatment group, Post_t indicates the post‑policy period, α is the intercept, β1 captures the treatment group effect, β2 captures the time effect, β3 is the DiD estimator, and ε_it is the error term.

The DiD estimator β3 is calculated as:

β3 = (Y_T,post - Y_T,pre) - (Y_C,post - Y_C,pre)

where Y_T,post and Y_T,pre are the post‑ and pre‑policy outcomes for the treatment group, and Y_C,post and Y_C,pre are the corresponding outcomes for the control group.

Python Implementation

Consider data where city A placed three billboards and city B serves as a control. The billboard installation occurred in July.

<code>import warnings
warnings.filterwarnings('ignore')
import pandas as pd
import numpy as np
from matplotlib import style
from matplotlib import pyplot as plt
import seaborn as sns
import statsmodels.formula.api as smf
%matplotlib inline
style.use("fivethirtyeight")
data = pd.read_excel('data/billboard.xlsx')
</code>

The first few rows of the dataset contain columns for deposit , a binary indicator A (1 for city A, 0 for city B), and july (0 for before July, 1 for after July).

Compare deposit changes before and after billboard placement in city A:

<code>a_before = data.query("A==1 & july==0")["deposit"].mean()
a_after = data.query("A==1 & july==1")["deposit"].mean()
a_after - a_before
</code>

The difference is 41.04775, but this may reflect a general time trend rather than the billboard effect, so a control group is needed.

<code>b_after = data.query("A==0 & july==1")["deposit"].mean()
a_after - b_after
</code>

The result is -119.10175, indicating city A’s average deposit after July is lower than city B’s, possibly because city B was richer to begin with.

<code>b_before = data.query("A==0 & july==0")["deposit"].mean()
diff_in_diff = (a_after - a_before) - (b_after - b_before)
diff_in_diff
</code>

The DiD estimate is 6.52456, suggesting a positive impact.

Plot the results:

<code>plt.figure(figsize=(10,5))
plt.plot(["Before", "After"], [a_before, a_after], label="A", lw=2)
plt.plot(["Before", "After"], [b_before, b_after], label="B", lw=2)
plt.plot(["Before", "After"], [a_before, a_before+(b_after-b_before)],
         label="Counterfactual", lw=2, color="C2", ls="-.")
plt.legend()
</code>

The yellow line shows the counterfactual trajectory for city A derived from city B’s change.

Fit the DiD model and view coefficients and standard errors:

<code>smf.ols('deposit ~ A*july', data=data).fit().summary().tables[1]
</code>

The coefficients for A and july are significant, while the interaction term is not. The baseline is the control group’s pre‑July level (city B). The interpretation follows standard DiD conventions.

Reference:

Matheus Facure; Michell Germano, Python Causality Handbook: First Edition