On this page
- Why it matters in ecommerce
- Attribution vs incrementality
- Academic baselines
- Branded vs non-branded paid-search incrementality
- Measurement methods compared
- Platform and tool critiques
- EU/UK retail media incrementality
- What practitioners report
- Adoption, cost & barriers
- Retail media incrementality
- Retail media case studies (vendor/network-reported)
- Contradictions
- Key terms
- Benchmarks (as-of 2026-10-13)
- Gaps (remaining after 2026-10-13 run)
Incrementality
Incrementality
Incrementality measures whether a marketing campaign caused outcomes (sales, conversions, new customers) that would not have happened without the ad exposure — the true causal lift, isolated by comparing an exposed group against an unexposed control. Per eMarketer, this is the key distinction from Attribution, which only assigns credit across touchpoints and proves correlation, not causation. This page records what sources report; it does not advise.
Why it matters in ecommerce
Per eMarketer, three forces are driving adoption: privacy-driven tracking loss (cookie/IDFA decline making user-level attribution unreliable), Retail Media accountability pressure, and budget scrutiny. For an ecommerce product or performance team, incrementality reframes the core question from "which channel gets credit?" to "would this purchase have happened anyway?" — a direct challenge to last-click attribution and headline ROAS.
Per Recast blog (2024): Recast introduces the concept of "signal debt" — the accumulation of measurement error from relying on low-fidelity attribution that compounds over multiple optimisation cycles. Incrementality testing is framed as the mechanism to reset signal debt (as-of 2024).
Attribution vs incrementality
Per eMarketer, attribution assigns credit across touchpoints but cannot prove causation; per the MHI Growth Engine (vendor), last-click can credit a paid ad for a sale a customer would have made regardless — e.g. a loyal customer clicking a retargeting ad before a planned repurchase, where "would they have bought anyway? Almost certainly yes." Incrementality isolates the lift the ad actually generated.
Academic baselines
These are peer-reviewed or academically published findings — the highest-confidence tier in the vault's evidence hierarchy.
Per Blake, Nosko & Tadelis (2015) eBay study: branded paid-search incremental traffic for a large established brand is approximately 0.5% — i.e. the vast majority of paid-brand clicks would have arrived organically (as-of 2015, widely cited 2024–2026). The same study found Edmunds.com (a smaller brand) lost 50–72% of clicks when brand spend was turned off — indicating brand size is the critical moderator: large brands near-zero incremental on brand search; small brands materially incremental (as-of 2015).
Per Gordon, Zettelmeyer, Bhargava & Chapsky (2023 revision, Marketing Science): observational attribution methods overstated Facebook ad effectiveness by approximately 13× compared to randomised controlled trial (RCT) measurement (as-of 2023).
Per Zalando simulation study (cited by Recast blog, 2024): Media Mix Modeling (MMM)-modelled attribution overstated true incremental lift by approximately 2.5× when tested against geo holdout results (as-of 2024).
Branded vs non-branded paid-search incrementality
The largest historical debate in paid search — whether branded keyword spend is incremental at all.
Per Haus (vendor) across 400+ geo experiments: branded search showed 82% incremental conversion rate in high-competition environments (heavy organic/competitor presence) vs 35% in low-competition contexts — confirming brand size and competitive pressure both mediate incrementality (as-of 2025-Q3).
Per TNT Growth blog (practitioner, 2025): at 3× budget scale, net incremental clicks decreased for branded campaigns — higher bids captured already-converting users and cannibalised organic traffic. Practical implication: run incrementality tests at actual budget levels, not scaled-up budgets, which systematically overstate marginal efficiency (as-of 2025).
Per Haus (vendor) Amazon experiment data (n=67 brands): 67% of brands measured −1.9% Amazon ad lift on net demand — indicating the spend was shifting channel (existing customer reactivation or search-to-direct) rather than generating net new demand (as-of 2026-Q1).
Per WillPowders (UK DTC brand, Common Thread Collective case study): cut 90% of branded search spend with no measurable impact on revenue or organic traffic — near-zero incremental for their brand (as-of 2025).
Per Common Thread Collective (CTC, 2024): 20–30% of DTC paid social campaigns test out as non-incremental when subjected to clean holdouts — higher than most clients expect and higher than Meta's reported lift rates suggest (as-of 2024).
Per Haus (vendor, 2025): Haus defines "experimental incrementality rate" (EIR) as the share of conversions that would not have occurred without the ad — distinct from iROAS (which is money-weighted). EIR allows comparison across campaigns with different AOVs (as-of 2025).
Measurement methods compared
Per eMarketer: geo experiments (test vs control by geography) are the most common applied method for Retail Media and TV/OOH; user-level RCTs are the statistical gold standard but require platform cooperation.
Per Recast evidence hierarchy (2024): RCTs at the top (user-level holdout or geo), then Media Mix Modeling (MMM) calibrated by holdouts, then platform-reported lift (lowest). Recast argues most brands currently rely on the lowest tier (as-of 2024).
Per Haus blog associated with MeasureSummit 2025: a portfolio-of-evidence approach — combining MMM, geo holdouts, and conversion lift — is recommended over single-method reliance.
Per Recast (2024): 15+ geo experiment failure modes include: (1) spillover from digital ads crossing geo boundaries, (2) seasonal confounders in short tests, (3) small geo cluster counts (<20) making statistical inference unreliable, (4) test market selection bias toward brand strongholds, (5) campaign-level vs brand-level measurement conflation (as-of 2024).
Per AdExchanger (2025): practitioner complaint that holdout groups erode over time — users in holdout are served ads by other channels during suppression, making holdouts "dirty" in multi-week programmatic tests. Most brands limit holdout to 2–3 weeks rather than the 4+ weeks recommended academically (as-of 2025).
Per Search Engine Journal (2026-Q1): Google Meridian's GeoX module requires a minimum of ~50 geo units for reliable inference — unsuitable for brands with limited geo diversity (single- country DTC with concentrated urban customer base) (as-of 2026-Q1).
Per Hacker News practitioner discussion (2024): DIY holdout approach — randomly tag 10–15% of email/CRM list as holdout, suppress ads for 4 weeks, compare conversion rate — described as "the easiest incrementality test most SMBs can actually run" without vendor tooling (as-of 2024).
Platform and tool critiques
Per Northbeam announcement (Q1 2026): Northbeam launched automated geo-based incrementality testing within its attribution platform — reducing setup burden but not resolving the structural geo-contamination issues Recast identifies (as-of 2026-01).
Per Sellforte white paper (vendor, 2024): a structural flaw in some DDA (Data-Driven Attribution) implementations is that treatment assignment (served ad) ≠ ad exposure (actually seen/engaged with) — biasing lift estimates upward.
Per Common Thread Collective (2024): Meta's In-App Test (IAT) showed worse outcomes than the platform's own 7-day click attribution for the same campaign — implying assignment bias in Meta's testing infrastructure. CTC conclusion: "trust the holdout, not the in-app test" (as-of 2024).
Per Google announcement (2026): Google Meridian GeoX tool (open-source MMM with geo-level calibration) launched globally — designed to provide geo-level MMM calibration to replace/ supplement platform-specific lift studies (as-of 2026-Q1).
Per Meta announcement (2026): Meta Brand Lift minimum budget increased from approximately $30,000 to $120,000 for standalone Brand Lift studies — raising the entry barrier for smaller advertisers (as-of 2026-Q1).
Per Google (2026): Google lowered geo experiment minimum budget from ~$100,000 to ~$5,000 using Bayesian models (as-of 2026-04-03).
EU/UK retail media incrementality
Per IAB Europe 2025 study (n=31 European markets): incrementality measurement is described as "aspirational" for the majority of European retail media advertisers; fewer than half of surveyed markets have active incrementality testing programmes (as-of 2025).
Per NIQ (NielsenIQ) June 2026 European retail report: 53% of European retailers rate their incrementality measurement capability as "adequate" or better — implying 47% consider their measurement inadequate (as-of 2026-06).
Per IAB Europe measurement guide 2025: Ghost Ads methodology is "rarely supported" in EU market implementations due to EU publisher ad-server setups and GDPR consent requirements — EU practitioners largely rely on geo holdouts instead (as-of 2025).
What practitioners report
Per Digiday (2025): agency practitioners report clients regularly reject incrementality results showing low or negative lift — attributing the result to poor test design rather than accepting non-incrementality. Described as a structural problem where incrementality functions as a "fail- safe for media spend, not a genuine optimisation signal" at most brands (as-of 2025).
Per CTC on TikTok fading lift: TikTok geo experiments initially show lift by capturing high- intent existing customers — that lift erodes as the campaign matures and exhausts that pool. Underlying demand was not created by TikTok (as-of 2025).
Per Haus (vendor) TikTok geo experiment data (cited by industry press 2025): TikTok campaigns show fading incremental lift over time, mirroring the DTC-saturation dynamic observed in Meta campaigns (as-of 2025).
Per Hacker News practitioners (2024): the dominant skeptic position is that geo holdout experiments are rarely statistically powered in practice — most ecommerce brands lack enough geographic market diversity for a clean test, and vendors selling geo-testing tools have an incentive to understate this constraint (as-of 2024).
Adoption, cost & barriers
Per the July 2025 eMarketer/TransUnion survey (as-of 2026-04-03): 52% of US brand and agency marketers use incrementality testing; 36.2% plan to increase incrementality spend over the next 12 months.
Per Skai / Path to Purchase Institute (vendor, via eMarketer): 44% question the reliability of incrementality results (top barrier), 43% struggle to apply it across ad types/retailers, 41% cite insufficient tooling, and 33% of CPG marketers measure incrementality only "at a basic level" (as-of 2026-04-03).
Per Skai/Stratably's survey of 166 retail media advertisers (vendor, 2026-02-04): incrementality is the #1 measurement challenge (75%), ahead of cross-channel measurement (59%); only 15% say they are very/extremely effective at measuring retail media (as-of 2026-02-04).
Per eMarketer: recommended test execution is to run for at least 3–4 weeks with properly sized holdout groups, start with the highest-spend channel, and budget for the short-term revenue trade-off of withholding ads.
Retail media incrementality
Per the ANA (via eMarketer): 71% of advertisers rank incrementality as their most important Retail Media KPI; the central question is whether retail media spend drives net-new purchases or merely captures demand that already existed (as-of 2026-04-03).
Per Skai (vendor): "The overreliance on ROAS as the benchmark of value is over" (Jason Wescott, WPP Media); "Incrementality is now the price of performance… If you cannot measure incrementality with discipline, you are not running performance media. You are buying exposure" (Enrico Babucci, OmniShopper).
Retail media case studies (vendor/network-reported)
- Per eMarketer, Albertsons Media Collective launched an in-store incrementality framework in early 2026; a Mondelēz test campaign delivered $2.41 matched-market incremental ROAS and 14% lift in in-store sales across 116 locations (as-of 2026-04-03).
- Per eMarketer, Kroger Precision Marketing runs randomised holdout tests using loyalty data covering ~95% of transactions, delivering results in under two weeks (as-of 2026-04-03).
Contradictions
Direction of Meta ad measurement bias. Gordon et al. 2023 (Marketing Science): observational methods overstate Meta ad effectiveness by 13× vs RCT [Gordon et al., 2023]. Meta 2025 whitepaper: observational methods understate Meta ad value by 31% [Meta, 2025]. Sources disagree on the direction of the bias, not just magnitude. Gordon is academic pre-2023 Meta platform changes; Meta whitepaper is self-serving vendor content. Unresolved.
Branded search incremental rate. Blake et al. 2015 (eBay): ~0.5% incremental for large brand [Blake et al., 2015]. Haus (vendor) 2025: 82% incremental in high-competition environments [Haus, 2025-Q3]. Both may be correct with brand size and competitive environment as mediating variables, but the figures are so far apart that practitioners cite the 0.5% figure to justify cutting brand spend without controlling for context. Recorded unresolved.
What actually blocks incrementality adoption. Skai says the barrier is not data access or methodology but capability/ownership (data access 0%, methodology 6% among non-adopters) [skai.io, 2026-02-04]. eMarketer's synthesis of Skai/P2PI data lists accuracy concerns (44%), application complexity (43%) and limited tools (41%) as top barriers [emarketer.com, 2026-04-03]. Likely reconciled by different question framings but recorded unresolved.
Meta IAT vs 7-day click. Meta's own In-App Test showed worse outcomes than 7-day click attribution for the same campaign [CTC, 2024]. Meta claims Brand Lift is gold-standard incrementality measurement [Meta vendor docs]. Practitioner evidence and vendor claims disagree on reliability of platform-native testing.
Incremental share of retargeting. MHI states 40–70% of retargeting reported ROAS is non-incremental [mhigrowthengine.com, 2026-02-17]; no independent dataset corroborates the exact band. Recorded as a vendor estimate, not a measured benchmark.
Key terms
| Term | Meaning (per sources) |
|---|---|
| Incremental lift | Outcome difference between exposed treatment group and unexposed control |
| Incremental ROAS (iROAS) | ROAS counting only conversions the ad actually caused |
| Experimental Incrementality Rate (EIR) | Share of conversions that would not have occurred without the ad; not money-weighted (Haus) |
| Holdout / Intent-to-Treat | Control group shown no ads; free but noisy (Remerge) |
| PSA test | Control shown public-service ads to identify would-be-exposed users (Remerge) |
| Ghost Ads | Control flagged via "ghost impression"; PSA precision at no cost (Johnson 2017, via Remerge) |
| Geo experiment | Test vs control by geography; suits TV/OOH/retail media (eMarketer) |
| Signal debt | Accumulation of measurement error from low-fidelity attribution across optimisation cycles (Recast) |
| GeoX | Google Meridian's geo-level MMM calibration module; requires ~50 geo units minimum (Google 2026) |
Benchmarks (as-of 2026-10-13)
| Metric | Value | Source | Confidence |
|---|---|---|---|
| Branded search lift — large brand | ~0.5% | Blake et al. 2015 (eBay) | Academic (peer-reviewed) |
| Branded search lift — small brand | 50–72% | Blake et al. 2015 (Edmunds) | Academic (peer-reviewed) |
| Observational attribution overstatement vs RCT | ~13× | Gordon et al. 2023 | Academic (peer-reviewed) |
| MMM overstatement vs geo holdout | ~2.5× | Zalando/Recast 2024 | Vendor-cited simulation |
| Branded search: high-competition EIR | 82% | Haus 2025-Q3 | Vendor (400+ experiments) |
| Branded search: low-competition EIR | 35% | Haus 2025-Q3 | Vendor (400+ experiments) |
| Amazon ad net demand lift (majority) | −1.9% | Haus 2026-Q1 (n=67) | Vendor (client sample) |
| DTC paid social non-incremental campaigns | 20–30% | CTC 2024 | Agency practitioner |
| US marketers using incrementality testing | 52% | eMarketer/TransUnion 2025 | Survey |
| EU retailers with adequate measurement | 53% | NIQ June 2026 | Survey |
| Meta Brand Lift minimum budget | $120,000 | Meta 2026-Q1 | Vendor announcement |
| Google geo experiment minimum budget | ~$5,000 | Google 2026 | Platform announcement |
Gaps (remaining after 2026-10-13 run)
- YouTube transcript content not obtained — 5 conference talks identified but Apify actor was unavailable. Priority: Haus MeasureSummit 2025 talk (400+ experiments). Re-run when Apify available.
- No primary Reddit thread data — Reddit MCP unavailable; proxy sources used. r/PPC and r/ecommerce practitioner threads may contain more granular tactics.
- Google Meridian GeoX independent performance data — launch-stage coverage only; no independent accuracy benchmarks yet.
- EU Ghost Ads constraint — IAB Europe notes EU constraint but no alternative EU-specific methodology documented; future run should fetch IAB Europe 2025 guide directly.