New Customers agent
New Customers agent · Google Ads, Meta · At work
Running
Google Ads and Meta · gift sets
Roast date vs “small batch”, gift sets
“People buying a gift respond more to how fresh the coffee is than to how it's made.”
Where it stands
Roast date is ahead, 87% confident after 8 full days. The call comes at the planned end, Nov 2, and needs 95%.
Day 9 of 14Oct 20 – Nov 2
The versions
| Version | Clicks | First orders | Rate | vs current | Confidence |
|---|---|---|---|---|---|
Small batchCurrent version “Small-batch coffee, roasted with care.” | 410 | 22 | 5.4% | — | — |
Roast dateLeading “Roasted Monday, at your door by Thursday.” | 400 | 29 | 7.2% | +35% | 87% |
Confidence is against the current version. The upright mark is the 95% a version needs at the planned end to be called the winner; before then it is a reading.
Oct 20Oct 28
Set-up
- Where
- Google Ads and Meta · gift sets
- Split
- Half of traffic to each
- Measured on
- First orders per click
- Planned length
- 14 days
- Started
- Oct 20
How experiments are judged
- The current version always runs alongside the challengers, so results are compared fairly.
- A test is called once, at its planned end: a version wins only at 95% confidence or more, with at least 20 conversions in every version.
- Before the planned end, a lead can still reverse, however strong it looks, so the test isn't called early.
- If nothing clears the bar at the planned end, the current version stays.