On this page
Difference-in-Differences (DiD)
Difference-in-Differences (DiD)
DiD is a quasi-experimental causal inference method that estimates the treatment effect by comparing the change over time in an outcome for a treated group against the change in an equivalent control group over the same period. In ecommerce, DiD underpins geo holdout and incrementality tests: a retailer withholds a marketing intervention from a set of control markets (or pauses a campaign) and measures whether the treated geos gained more than the control geos over the test window.
Core mechanic
The canonical estimator:
uplift = (Y_treatment_post − Y_treatment_pre) − (Y_control_post − Y_control_pre)
Equivalently, the ATT (average treatment effect on the treated) is the coefficient on the interaction term treated × post in an OLS regression that includes unit and time fixed effects (Angrist & Pischke, 2009, as cited by Google Meridian GeoX documentation, 2026).
Key terms
| Term | Meaning |
|---|---|
| ATT | Average Treatment effect on the Treated — what DiD estimates |
| Parallel trends | Pre-treatment, treated and control units trend together; core identifying assumption |
| Pre-period | Baseline window used to fit counterfactual; GeoX recommends 4–5× the test duration |
| Test window | Active intervention period; Google GeoX minimum 15 days for fast-moving products, 4–6 weeks for noisy signals (as-of 2026-09-10) |
| Staggered adoption | Multiple units begin treatment at different times; standard TWFE estimator is biased here |
| TWFE | Two-Way Fixed Effects — the standard regression implementation of DiD |
| MDE | Minimum Detectable Effect — the smallest uplift a test is powered to detect |
| Winner's curse | Upward bias on estimated MDE when too few backtests are used for power analysis |
Parallel trends assumption
The parallel trends (PT) assumption — that treated and control units would have followed the same trajectory absent treatment — is untestable for the post-treatment period (this is universally accepted in the academic literature and reiterated in GeoX documentation, 2026).
Pre-period placebo tests are the standard validation proxy: fit the DiD model on historical windows where no intervention occurred and check that estimated effects cluster around zero. Meridian GeoX implements this as "Design-Aware Placebo Inference" — placebo windows are selected using the same geographic assignment algorithm as the real test, reducing selection bias (Google Research, 2026, as-of 2026-09-10).
Variants
| Variant | Key mechanic | Main use-case |
|---|---|---|
| Classic TWFE DiD | OLS with unit + time FE | Balanced panels; simultaneous treatment |
| Synthetic Difference-in-Differences (SDiD) | Unit weights (ω) + time weights (λ) to construct a "synthetic" pre-trend match | Small N geo tests; unbalanced panels |
| Synthetic Control Method (SCM) | Weighted average of control units to match pre-period treated path | Very few treated units (1–5 geos) |
| Augmented Synthetic Control Method (AuGSynth) | SCM + doubly-robust bias correction; used in Meta GeoLift | Multiple treated geos; imbalanced baseline |
| Time-Based Regression (TBR) | Uses pre/post time variation as counterfactual rather than cross-sectional control | Google's default in Meridian GeoX |
| CausalImpact | Bayesian structural time-series state-space model; Google open-source R/Python package | Time-series data where geo cross-section is unavailable |
SDiD vs TBR as the "right" default for geo experiments: Google's Meridian GeoX (2026) uses TBR as its headline estimator and treats SDiD as one of three supported methods; Arkhangelsky et al. (2021, AER) argue SDiD dominates plain DiD and SCM on bias-variance grounds. The GeoX docs do not adjudicate between TBR and SDiD head-to-head. Neither source has been independently verified against the other's empirical claims.
Staggered adoption estimator consensus: Callaway & Sant'Anna (2021) and Sun & Abraham (2021) both propose "honest" group-time ATT estimators for staggered DiD but differ in aggregation strategy. de Chaisemartin & D'Haultfœuille and Borusyak, Jaravel & Spiess offer further alternatives. No single estimator is universally recommended — choice depends on heterogeneity assumptions and data structure. Source: academic review cited by the Causal Inference for the Social Sciences community and reiterated in practitioner YouTube tutorials (as-of 2026-09-10).
Benchmarks (as-of 2026-09-10)
- Minimum geo count: Meridian GeoX requires ≥ 10 geos; ≥ 50 geos recommended for stable TBR estimates (Google Meridian GeoX documentation, 2026).
- Pre-period length: GeoX recommends 4–5× the intended test window for reliable placebo calibration (Google Meridian GeoX documentation, 2026).
- Minimum test duration: 15 days for fast-moving products; 4–6 weeks for noisy or low-volume signals (Google Meridian GeoX documentation, 2026).
- Winner's curse magnitude: With fewer than 8 backtests, MDE estimates exhibit meaningful optimism bias; empirical size inflation observed at 0.135 vs nominal 0.10 (Design-Aware Placebo Inference paper, Google Research, 2026).
Platform feature details (geo minimums, default estimators) are from September 2026 release notes and may change.
What practitioners report
Geo assignment
Geo assignment quality dominates estimator choice — a poor split undermines both TBR and SDiD. Meridian GeoX automates geo assignment using a reproducible algorithm and re-runs it until placebo distributions pass calibration (Google Meridian GeoX documentation, 2026).
Pre-period length in practice
Practitioner YouTube tutorials (as-of 2026-11-07, partial transcript) note that most retailers run far shorter pre-periods than recommended (sometimes equal to the test window), which degrades placebo calibration and inflates confidence intervals.
Commercial platform landscape (as-of 2026-11-07)
| Platform | Method | Status |
|---|---|---|
| Meridian GeoX | TBR + SCM + SDiD (Design-Aware Placebo Inference) | GA, September 9–10, 2026 |
| Haus | Synthetic Control Method ("Haus Holdout"); Fixed Geo Tests | Fixed Geo Tests launched June 23, 2025; active |
| Meta GeoLift (R) | Augmented SCM (AuGSynth) | Last code push May 2024; in maintenance mode |
Haus Holdout vs SCM naming: Haus marketing materials call their method "Haus Holdout" while academic literature and GeoX docs use "Synthetic Control Method." Whether Haus Holdout is a strict implementation of Abadie & Gardeazabal's SCM or a proprietary variant has not been independently confirmed in sources available to this run.
Meta GeoLift maintenance status: The last code push on GitHub was May 2024 (as-of 2026-11-07), which practitioners read as maintenance mode. Meta's official documentation does not explicitly deprecate the tool. Both claims are filed without resolution.
Feeds into
- Marketing Mix Modelling (MMM) — geo experiment results used as Bayesian priors in Meridian MMM to calibrate channel effectiveness
- Incrementality Testing — DiD is the statistical engine behind most geo holdout incrementality frameworks
- Geo Holdout Experiments — the operational container for DiD-based market testing
Next frontier concepts (dangling links)
- Synthetic Difference-in-Differences (SDiD) — no dedicated page yet
- Design-Aware Placebo Inference — GeoX's proprietary extension; no page yet
- Staggered DiD — heterogeneous treatment timing; no page yet
- Meta GeoLift — entity page stub exists but content thin