Statistical signal
A two-sided pooled two-proportion z-test estimates how surprising the observed gap would be if both variants performed equally.
Free · no signup · statistically grounded
Is your cold-email winner actually a winner?
Enter the results from two variants. Get a plain-English verdict, the likelihood that the gap is ordinary noise, and the sample you would need to resolve it reliably.
Judge the test on
Use delivered sends when available. Pick one primary outcome before looking at the result.
Not enough evidenceTwo-sided test · 95% threshold
Variant B looks 50% better on replies. If the emails were equally effective, a gap at least this large would appear about 26% of the time. To detect a difference this size with 80% power, you would need about 2,517 sends per variant. You have 400 and 400.
Variant A replies3.0%
Variant B replies4.5%
p-value0.264
Observed difference1.50 percentage points
95% interval for B − A-2.30 to 5.28 pp
Sends needed per variant2,517
What the calculator proves
A bigger number is not automatically better evidence.
Uncertainty range
A 95% Wilson-score interval shows the plausible range for the difference in rates instead of hiding behind one lift number.
Sample requirement
The required sends use a 5% two-sided significance threshold and 80% power, assuming equal allocation and the observed effect size.
From one test to every campaign
Want this checked automatically?
Variantt connects to your sending tools, validates every experiment, and recommends the next test your evidence can support.