Experiments

How experiments are judged

The current version always runs alongside the challengers, so results are compared fairly.
A test is called once, at its planned end: a version wins only at 95% confidence or more, with at least 20 conversions in every version.
Before the planned end, a lead can still reverse, however strong it looks, so the test isn't called early.
If nothing clears the bar at the planned end, the current version stays.

Running now · 2

Running

Google Ads and Meta · gift sets

People buying a gift respond more to how fresh the coffee is than to how it's made.

Day 9 of 14Oct 20 – Nov 2

Reading so far: Roast date ahead · 87% confident

Running

Product pages, all paid traffic

Showing how close a cart is to free shipping will lift first orders without lowering the average order.

Day 3 of 21Oct 26 – Nov 15

Too early to call

Winner found

Google Ads and Meta · every product line

New customers care more about how fresh the coffee is than about how it's made.

Ran 14 daysOct 7 – Oct 20

Roast dates won: 37% more first orders per click, 95% confident. For brewing gear alone there was no difference.