On this page
concept

Synthetic Difference-in-Differences (SDiD)

Created 2026-11-09 29 connections

Synthetic Difference-in-Differences (SDiD)

Synthetic Difference-in-Differences is a causal inference estimator introduced by Arkhangelsky, Athey, Hirshberg, Imbens, and Wager in a 2021 American Economic Review paper (111(12): 4088–4118) that combines the two core mechanics of its predecessors: Synthetic Control Method|synthetic control's data-driven unit re-weighting and Difference-in-Differences (DiD)|difference-in-differences' additive two-way fixed effects. In ecommerce, it is applied to Geo Holdout Experiments|geo holdout and Incrementality Testing|incrementality tests where parallel trends are suspect and the treated market count is small to moderate.

Core mechanic

SDiD estimates two sets of weights simultaneously, then runs a weighted two-way fixed effects (TWFE) regression (MetricGate, retrieved 2026-11-09; dannyredel.github.io, retrieved 2026-11-09):

Unit weights (ω̂): assigned to control units so that the weighted average of their pre-treatment outcomes matches the treated unit's pre-treatment trajectory. A ridge (L2) penalty and simplex constraint prevent degenerate solutions. The penalty scales as (N_tr × T_post)^(1/4), meaning larger test regions are regularised more strongly (mlsynth GEOX documentation, retrieved 2026-11-09).

Time weights (λ̂): unique to SDiD. Selected pre-treatment time periods are assigned weights so that the control units' weighted average over that window best predicts the control units' post-treatment average. This down-weights less informative periods (dannyredel.github.io, retrieved 2026-11-09).

The final observation weight is the outer product of unit and time weights. An intercept in the unit-weight optimisation absorbs any level difference between treated and control units, so SDiD matches the shape of the pre-period trajectory rather than its level — unlike pure SCM, which requires the synthetic match to reach treated levels without an intercept (MetricGate, retrieved 2026-11-09).

Key terms

TermMeaning
ATTAverage Treatment effect on the Treated — what SDiD estimates
Unit weights (ω̂)Per-control-unit weights that match the treated unit's pre-period trend trajectory
Time weights (λ̂)Per-period weights that emphasise more informative pre-treatment windows; unique to SDiD
Simplex constraintUnit weights are non-negative and sum to 1
Ridge penaltyL2 regularisation on unit weights; scales with panel size
TWFETwo-Way Fixed Effects — the regression framework shared by DiD, SCM, and SDiD
Staggered adoptionMultiple treated units begin treatment at different times; standard TWFE is biased here
Sequential SDiDExtension of SDiD to staggered rollouts; see § Extensions

SDiD vs DiD vs SCM — unified framework

All three methods are special cases of one weighted TWFE regression that differ only in how they set unit weights, time weights, and whether an intercept is included (dannyredel.github.io, retrieved 2026-11-09):

MethodUnit weightsTime weightsIntercept
Difference-in-Differences (DiD)UniformUniformYes
Synthetic Control MethodData-driven (no regularisation)UniformNo
SDiDData-driven (regularised)Data-drivenYes

When to prefer SDiD

The MetricGate analysis (retrieved 2026-11-09) presents a selection guide:

  • Parallel trends roughly holds + many treated units → DiD is best; SDiD adds safety margin
  • Single treated unit + long pre-period → SCM is best; SDiD is comparable
  • Many treated units + parallel trends violated → SDiD is best
  • Short pre-period → SDiD is best; SCM overfits because weights have few data points to fit
  • Staggered adoption → SDiD via cohort-block extension (standard TWFE is biased per Goodman-Bacon)

SDiD is described as "doubly robust" — consistency holds if either the unit weights or the time weights are credible (MetricGate, retrieved 2026-11-09).

Danny Redel's Monte Carlo simulations (200 replications, τ = 10) show SDiD has the highest RMSE in Scenario 1 when parallel trends hold (SDiD 0.232 vs DiD 0.190), but the lowest bias in Scenarios 2 and 3 when interactive fixed effects are present (SDiD bias 0.005 vs DiD bias 2.824 in Scenario 2; SDiD bias 0.005 vs SCM bias 15.966 in Scenario 3). This confirms SDiD's characterisation as "minimax" — it avoids the worst case but does not optimise for the best case — and directly contradicts any claim that SDiD universally outperforms (dannyredel.github.io, retrieved 2026-11-09).

Benchmarks (as-of 2026-11-09)

  • In a 40-market US metro panel (mlsynth GEOX tutorial), the SDiD design placed weight on only 7 of 91 pre-treatment days, illustrating the sparsity of time weights (mlsynth documentation, retrieved 2026-11-09).
  • In the diff-diff library's simulated geo panel, SDiD ATT was 311.85 against a true effect of +300 per treated market per week (21% lift on a ~1,500 baseline), with placebo SE 7.27 and 95% CI [297.61, 326.09] (diff-diff documentation, retrieved 2026-11-09).
  • Amazon Science research (Meloni, Hut, Islam, presented CODE@MIT 2024) found SDiD has lower variance and lower standard errors than standard DiD for geo-randomised experiments, producing higher statistical power.

The Amazon Science benchmarks (Meloni et al.) were presented in 2024 (CODE@MIT); no 2026 replication was found.

The Amazon Science paper reports SDiD has lower variance and higher power vs DiD (Meloni et al., CODE@MIT 2024), while the same paper notes SDiD can be biased in some settings — including A/A tests and experiments with homogeneous 1–5% treatment effects. A data-driven choice of the time-weight regularisation parameter may fix the bias in those settings. The two findings are compatible (lower variance does not imply zero bias) but practitioners citing only the power advantage may underweight the bias risk (Amazon Science publication page, retrieved 2026-11-09).

Inference

Three methods are supported (diff-diff documentation, retrieved 2026-11-09):

  1. Placebo permutation (Algorithm 4 in Arkhangelsky et al.) — reassigns treatment across donors and observes how often the placebo effect is as large as the estimated ATT; the preferred method in mlsynth GEOX because SDiD's time weights mean pre-treatment periods are not exchangeable, making conformal inference invalid.
  2. Jackknife — suggested when at least 4–5 treated units are available; described as "mildly anti-conservative on smaller panels."
  3. Bootstrap — valid but 5–30x slower than placebo.

The diff-diff library notes that pre-fit shape RMSE measures trajectory matching only; the constant level gap is differenced out and reported separately. A good pre-fit supports the design's plausibility but does not validate the counterfactual by itself.

Platform implementations

Platform adoption of SDiD as the primary estimator is inconsistent across the geo-testing ecosystem. Meridian GeoX (Google, GA 2026-09-09) lists SDiD as one of three counterfactual model types in its research paper (alongside TBR and SCM), but the developer-facing docs describe only "time-based regression and stratified sampling" without explicitly confirming SDiD is fully implemented vs. described as a candidate. Haus uses synthetic control methodology, not SDiD, as confirmed by its blog. GeoLift (Meta) uses ASCM and GSC, not SDiD. The open-source mlsynth GEOX module uses SDiD as its default engine. It is unclear why commercial geo-testing platforms have been slower than the academic and open-source community to adopt SDiD (research.google/pubs/pub1090193/, 2026; haus.io blog, retrieved 2026-11-09; GeoLift methodology docs, retrieved 2026-11-09; mlsynth documentation, retrieved 2026-11-09).

Meridian GeoX (Google)

Meridian GeoX entered general availability on 2026-09-09 (closed beta from May 2026). Its research paper lists SDiD as one of three counterfactual model types alongside Time-Based Regression and Synthetic Control (research.google/pubs/pub1090193/, as-of 2026-09). Google internally claims "budget savings of more than 31% for large advertisers" vs other open-source equivalents — a vendor claim with no published sample size, test period, or comparison set (ppc.land, 2026-09).

[!unverified] GeoX's claim of 31% budget savings and 94% reduction in design generation time are based on unverified internal benchmarks only (ppc.land, 2026-09).

GeoLift (Meta/Facebook)

GeoLift documentation describes its methodology as Augmented Synthetic Control Method|Augmented Synthetic Control Methods (ASCM) for point estimation and Generalised Synthetic Control (GSC) for inference. SDiD is not mentioned (GeoLift methodology docs, retrieved 2026-11-09).

Haus

Haus uses a synthetic control methodology: it builds a synthetic control from weighted combinations of untreated regions, runs multiple candidate models with different assumptions, and uses AI to assign weights based on out-of-sample placebo-period performance. Its blog does not confirm SDiD as one of the candidate models. Haus's internal simulation (one representative customer's data) found its synthetic control "4x more precise" than matched market tests — a vendor-reported, single-customer result (haus.io, retrieved 2026-11-09).

[!unverified] The Haus "4x more precise" claim is from a single customer simulation, vendor-reported and not peer-reviewed (haus.io, retrieved 2026-11-09).

mlsynth GEOX (open-source Python)

The mlsynth library's GEOX module uses SDiD as its default engine and applies both unit and time weights simultaneously. It also offers an augmented synthetic control engine ("augsynth") — the ridge-augmented SCM of Ben-Michael, Feller and Rothstein (2021) — and a Bayesian synthetic control engine ("mvbbsc") with a uniform-Dirichlet prior sampled by NUTS (mlsynth documentation, retrieved 2026-11-09).

diff-diff (Python library)

The diff-diff Python library implements SDiD with diagnostics and inference, citing Arkhangelsky et al. Its decision tree routes practitioners to SDiD specifically on the "few test markets" branch (diff-diff documentation, retrieved 2026-11-09).

Extensions

Sequential SDiD (staggered adoption)

Sequential SDiD (arXiv:2404.00164, Arkhangelsky and Samkov, presented at AEA January 2025) extends SDiD to staggered treatment rollouts by iterative imputation: early-treated cohort estimates become the counterfactuals used for later cohorts. The resulting estimator is asymptotically equivalent to an oracle OLS in a linear interactive fixed-effects model. It does not restrict unit and time weights to be nonnegative (unlike the original), and reduces to a sequential DiD estimator when the ridge regularisation parameter is set to infinity (arXiv:2404.00164, 2024).

arXiv:2404.00164 published April 2024. The applied AEA January 2025 presentation is the most recent venue found; no 2026 follow-up paper or replication was surfaced.

Sequential SDiD requires independent idiosyncratic errors across units, which the paper describes as a key limitation.

What practitioners report

Reddit discussion (r/ecommerce, r/analytics, r/advertising, 2025) does not mention SDiD by name. Practitioners discuss incremental testing platforms (Measured, Haus) and geo holdout approaches in terms of rigor and cost — not in terms of the underlying estimator. The Reddit signal on this method is effectively zero by name.

Platforms like Measured and Haus are described as the most rigorous options for incrementality testing, with estimated costs of $50k+ per year (r/ecommerce, 2025).

All Reddit findings are from 2025 (pre-2026).

Competing methods

The August 2025 arXiv paper (arXiv:2508.20335) benchmarking Dynamic Synthetic Controls vs. Panel DML|Panel-Aware Double Machine Learning for geo marketing found that Augmented SCM "consistently demonstrates severe bias and near-zero coverage" in scenarios with nonlinearities or external shocks, while Panel-DML variants "dramatically reduce this bias"; no single model wins across all scenarios. SDiD was not directly benchmarked in that paper.

A KDD 2025 workshop paper found panel-aware DML models generally outperformed synthetic control (not SDiD specifically) across most stress tests, with the conclusion "no single best model." This contradicts recommendations from the diff-diff library that position SDiD as the go-to for few-treated-market geo experiments.

diff-diff docs recommend SDiD for "few treated markets" geo experiments (retrieved 2026-11-09). The KDD 2025 workshop paper found DML generally outperforms SCM in most stress tests (arXiv:2508.20335, 2025-08). Neither paper directly compares SDiD (not plain SCM) against Panel DML, so the contradiction may dissolve under a direct head-to-head comparison.

Research agent · 2026-11-09