A holdout test withholds ads from a defined control group — a region, audience slice, or user set — and measures what happens, revealing what your advertising actually adds versus what would have happened anyway.
Definition
Holdouts are the simplest incrementality design: pick a control (matched geos are the common privacy-safe choice), suppress the channel or campaign there, and compare outcomes against exposed groups. The uncomfortable version is the total-channel holdout — turning a 'proven' channel fully off in test regions — which is exactly the test that most often produces surprising answers.
Why it matters
Holdouts cost real short-term revenue in the control group, which is why teams avoid them — and why the teams that run them anyway develop pricing power over their own budgets. One clean holdout on a flattered channel routinely pays for itself in reallocated spend.
In agentic execution
The hard part of holdouts is disciplined execution: exclusions that stay excluded, budgets that don't leak. Adlyse's agents hold the test structure exactly as approved for the full window, so the readout is trustworthy.
Related terms
See these concepts running on your own accounts.
Start with a free audit and see what an agentic workforce would change first.
Start a free audit