Ab Test Duration Calculator
Calculate how long to run an A/B test for statistical significance given traffic and lift. Enter values for instant results with step-by-step formulas.
Reviewed by Daniel Agrici, Founder & Lead Developer
Formula
n = (Zα√(2p̄q̄) + Zβ√(p₁q₁+p₂q₂))² / (p₂-p₁)²
The sample size per variation is calculated using the standard two-proportion z-test formula, where Zα is the z-score for your significance level, Zβ is the z-score for statistical power (80%), p₁ is the baseline conversion rate, and p₂ is the target conversion rate (baseline × (1 + MDE)).
Worked Examples
Example 1: E-commerce Checkout Test
Problem:An e-commerce site has a 3% conversion rate, 10,000 daily visitors, and wants to detect a 10% relative improvement at 95% significance.
Solution:Baseline: 3%, Target: 3.3%, MDE: 10%\nSample per variation: ~38,572\nTotal sample: ~77,144\nWith 5,000 visitors per variation/day: ~8 days
Result:Run test for at least 8 days with 77,144 total visitors needed
Example 2: Landing Page Headline Test
Problem:A SaaS landing page converts at 8% with 2,000 daily visitors. They want to detect a 15% relative improvement at 95% significance.
Solution:Baseline: 8%, Target: 9.2%, MDE: 15%\nSample per variation: ~5,286\nTotal sample: ~10,572\nWith 1,000 visitors per variation/day: ~6 days
Result:Run test for at least 6 days with 10,572 total visitors needed
Frequently Asked Questions
How long should I run an A/B test?
The duration depends on your daily traffic, baseline conversion rate, and the minimum effect you want to detect. Most A/B tests require at least 1-2 weeks to gather statistically significant results. Never stop a test early just because one variation appears to be winning — this leads to false positives. Use Ab Test Duration Calculator to determine the exact number of days needed based on your specific parameters.
Why should I not stop an A/B test early?
Stopping a test early (also called 'peeking') inflates the false positive rate dramatically. Early in a test, random fluctuations can make one variation appear significantly better. This is known as the multiple comparisons problem. Studies show that peeking at results daily with 95% significance can result in actual false positive rates of 25-30%. Always run the test for the full calculated duration to get reliable results.
References
Reviewed by Daniel Agrici, Founder & Lead Developer · Editorial policy