How Offline Spatiotemporal Splitting Eliminates Bias in AB Experiments

Introduction

In the first article of the series we introduced challenges of A/B testing in a freight two‑sided market, such as interference between treatment and control groups and user discrimination.

Background

Conventional experiments (user‑id or session‑id splitting) often produce biased results because the groups share the same scarce resources (e.g., drivers), violating the SUTVA assumption.

Offline Spatiotemporal Splitting

Spatiotemporal splitting isolates treatment and control groups in either time or space. By dividing the space into “spatial units” and time into “time units”, each “spatio‑temporal slice” receives a distinct intervention, reducing bias and avoiding user discrimination.

Different spatial and temporal granularities are illustrated in the tables below.

Bias‑Variance Trade‑off

Bias : systematic deviation caused by interference (e.g., driver competition).

Variance : random fluctuation (AA variation) that grows when sample size shrinks.

Coarser splitting lowers bias but increases variance; finer splitting does the opposite.

Design Process

Business communication – define key metrics and their definitions.

Data preparation – aggregate historical metrics for each spatio‑temporal slice.

Offline spatial grouping – formulate an objective function to minimize pre‑experiment differences and generate groups (e.g., Monte‑Carlo, genetic algorithm).

Time‑slot ordering – enumerate possible time‑slot sequences and select the best according to the objective.

Evaluation – estimate variance via bootstrapping on a test set and assess bias using cross‑group order rate.

Risks

Potential issues include time‑slot conflicts, unexpected events (e.g., pandemic), sample representativeness problems, and difficulty measuring bias.

Summary

An ideal A/B experiment should have both low bias and low variance. Offline spatiotemporal splitting isolates groups in time and space, effectively reducing bias from resource competition and user discrimination while controlling variance through careful pre‑experiment design. This method is widely used in Huolala experiments.