New Customers agent
New Customers agent · Google Ads, Meta · At work
Winner found
Google Ads and Meta · every product line
Roast-date messages for coffee
“New customers care more about how fresh the coffee is than about how it's made.”
Outcome
Roast dates won: 37% more first orders per click, 95% confident. For brewing gear alone there was no difference.
Next: Shift budget toward the Holiday Blend and gift sets. You approved this on Oct 21.
Ran 14 daysOct 7 – Oct 20
The versions
| Version | Clicks | First orders | Rate | vs current | Confidence |
|---|---|---|---|---|---|
Small batchCurrent version “Small-batch coffee, roasted with care.” | 1,270 | 46 | 3.6% | — | — |
Roast dateLeading “Roasted Monday, at your door by Thursday.” | 1,246 | 62 | 5.0% | +37% | 95% |
Confidence is against the current version. The upright mark is the 95% a version needs at the planned end to be called the winner; before then it is a reading.
Oct 7Oct 20
Set-up
- Where
- Google Ads and Meta · every product line
- Split
- Half of traffic to each
- Measured on
- First orders per click
- Planned length
- 14 days
- Started
- Oct 7
Related
How experiments are judged
- The current version always runs alongside the challengers, so results are compared fairly.
- A test is called once, at its planned end: a version wins only at 95% confidence or more, with at least 20 conversions in every version.
- Before the planned end, a lead can still reverse, however strong it looks, so the test isn't called early.
- If nothing clears the bar at the planned end, the current version stays.