How do you A/B test cold emails?

Split one list randomly into two groups and change a single element, such as the subject line or call to action, between them. Send both at the same time, to the same kind of prospects. Track positive replies per version, and only pick a winner once both versions have enough sends.

How many emails do you need for a cold email A/B test?

It depends on how different the versions are and how high your reply rates are. With low reply rates, a few hundred sends per version is often still too few to detect a modest difference. Set a minimum before you start and treat anything below it as inconclusive.

What is a two-proportion z-test in email testing?

It is a standard way to check whether two rates, such as the reply rates of version A and version B, differ by more than you would expect from chance. At about 95 percent confidence, a z value beyond 1.96 or below minus 1.96 suggests the difference is likely real.

Free template · Outbound & prospecting

Cold Email A/B Test Tracker

Test one element at a time and see which version really wins, with a built-in significance check.

Excel · 2 tabs

What's inside

50 test rows with version A/B inputs
Automatic reply rates and difference
A two-proportion significance check
'Need more data' until your minimum sample is reached

Who it's for

Teams improving reply rates with real tests instead of hunches.

Why it matters

Small cold email lists make it easy to fool yourself. One version gets six positive replies and the other gets eleven, and it looks like a clear winner. With a few hundred sends, a gap like that can easily be chance. Teams then roll out the 'winning' subject line and wonder why results do not hold up.

Early-stage teams also tend to change several things at once: a new subject, a new opener and a new offer in the same send. When results move, nobody knows why. A simple log that forces one change per test, and holds back a verdict until there is enough data, keeps your learning honest and easy to repeat.

How to use it

1Set the minimum sends per version at the top of the Tests tab before you start, so you decide your sample size before seeing any results.
2Add one row per test, pick the element you are changing from the dropdown and write versions A and B, changing nothing else between them.
3Enter sends and positive replies for each version as results come in. Count positive replies only, not out-of-office or unsubscribe responses.
4Read the Result column. Wait while it says Need more data, then keep the winner or log No clear winner and test something else.

Read alongside it

Follow Enablemnt AI on LinkedInShort, practical posts on GTM, outbound and RevOps for early-stage B2B founders.Follow

Cold Email A/B Test Tracker: FAQ

How do you A/B test cold emails?: Split one list randomly into two groups and change a single element, such as the subject line or call to action, between them. Send both at the same time, to the same kind of prospects. Track positive replies per version, and only pick a winner once both versions have enough sends.
How many emails do you need for a cold email A/B test?: It depends on how different the versions are and how high your reply rates are. With low reply rates, a few hundred sends per version is often still too few to detect a modest difference. Set a minimum before you start and treat anything below it as inconclusive.
What is a two-proportion z-test in email testing?: It is a standard way to check whether two rates, such as the reply rates of version A and version B, differ by more than you would expect from chance. At about 95 percent confidence, a z value beyond 1.96 or below minus 1.96 suggests the difference is likely real.

More free templates

View all 45

Rather have it built than download it?

We build your ICP, CRM and live outbound in a 14-day sprint, documented and handed over.

Join the sprint