Reorders agent
Reorders agent · Email · At work
Running
Klaviyo · first-time buyers
Brew-guide sequence vs the old one
“Three short emails with a brew guide for the coffee they bought will get more first-time buyers to a second order than the old five-email sequence.”
Where it stands
Too early to call. Every version needs at least 20 second orders by the planned end, Nov 6.
Day 5 of 14Oct 24 – Nov 6
The versions
| Version | Customers | Second orders | Rate | vs current | Confidence |
|---|---|---|---|---|---|
Previous sequenceCurrent version Five emails over four weeks, mostly about the roastery. | 560 | 9 | 1.6% | — | — |
Brew-guide sequence Three short emails, each with a brew guide for the coffee they bought. | 560 | 12 | 2.1% | +33% | 75% |
Confidence is against the current version. The upright mark is the 95% a version needs at the planned end to be called the winner; before then it is a reading.
Oct 24Oct 28
Set-up
- Where
- Klaviyo · first-time buyers
- Split
- Alternate new customers between sequences
- Measured on
- Second orders per customer
- Planned length
- 14 days
- Started
- Oct 24
Related
How experiments are judged
- The current version always runs alongside the challengers, so results are compared fairly.
- A test is called once, at its planned end: a version wins only at 95% confidence or more, with at least 20 conversions in every version.
- Before the planned end, a lead can still reverse, however strong it looks, so the test isn't called early.
- If nothing clears the bar at the planned end, the current version stays.