Ab Test Duration Calculator
Calculate how long to run an A/B test for statistical significance given traffic and lift. Enter values for instant results with step-by-step formulas.
Reviewed for accuracy by Daniel Agrici, Founder & Lead Developer
Ab Test Duration Calculator
Calculator
Adjust values & calculateEnter your values below. Every result is computed in your browser — no data is sent to any server.
Formula: n = (Zα√(2p̄q̄) + Zβ√(p₁q₁+p₂q₂))² / (p₂-p₁)²
Worked example — Run test for at least 8 days with 77,144 total visitors needed
Formula
n = (Zα√(2p̄q̄) + Zβ√(p₁q₁+p₂q₂))² / (p₂-p₁)²
The sample size per variation is calculated using the standard two-proportion z-test formula, where Zα is the z-score for your significance level, Zβ is the z-score for statistical power (80%), p₁ is the baseline conversion rate, and p₂ is the target conversion rate (baseline × (1 + MDE)).
Worked Examples
Example 1: E-commerce Checkout Test
Problem:An e-commerce site has a 3% conversion rate, 10,000 daily visitors, and wants to detect a 10% relative improvement at 95% significance.
Solution:Baseline: 3%, Target: 3.3%, MDE: 10% Sample per variation: ~38,572 Total sample: ~77,144 With 5,000 visitors per variation/day: ~8 days
Result:Run test for at least 8 days with 77,144 total visitors needed
Example 2: Landing Page Headline Test
Problem:A SaaS landing page converts at 8% with 2,000 daily visitors. They want to detect a 15% relative improvement at 95% significance.
Solution:Baseline: 8%, Target: 9.2%, MDE: 15% Sample per variation: ~5,286 Total sample: ~10,572 With 1,000 visitors per variation/day: ~6 days
Result:Run test for at least 6 days with 10,572 total visitors needed
Frequently Asked Questions
How long should I run an A/B test?
The duration depends on your daily traffic, baseline conversion rate, and the minimum effect you want to detect. Most A/B tests require at least 1-2 weeks to gather statistically significant results. Never stop a test early just because one variation appears to be winning — this leads to false positives. Use Ab Test Duration Calculator to determine the exact number of days needed based on your specific parameters.
What is Minimum Detectable Effect (MDE)?
Minimum Detectable Effect is the smallest improvement you want your test to be able to detect. A smaller MDE requires more traffic and longer test duration. For example, detecting a 1% relative improvement requires roughly 100x more samples than detecting a 10% improvement. Choose an MDE that represents a meaningful business impact — typically 5-20% relative change for most tests.
What statistical significance level should I use?
The standard is 95% significance, meaning there is only a 5% chance the result is due to random chance (false positive). Use 90% for early-stage or low-traffic tests where speed matters more than precision. Use 99% for critical changes like checkout flow modifications where false positives are costly. Higher significance requires larger sample sizes and longer test durations.
What is statistical power in A/B testing?
Statistical power (typically set at 80%) is the probability that your test will correctly detect a real difference when one exists. A power of 80% means there is a 20% chance of a false negative — missing a real improvement. Higher power (e.g., 90%) reduces false negatives but requires larger sample sizes. The industry standard of 80% offers a good balance between test duration and reliability.
Why should I not stop an A/B test early?
Stopping a test early (also called 'peeking') inflates the false positive rate dramatically. Early in a test, random fluctuations can make one variation appear significantly better. This is known as the multiple comparisons problem. Studies show that peeking at results daily with 95% significance can result in actual false positive rates of 25-30%. Always run the test for the full calculated duration to get reliable results.
References
Reviewed for accuracy by Daniel Agrici, Founder & Lead Developer · Editorial policy
Related Calculators
🧮Marketing ROI Calculator
Calculate marketing campaign return on investment from spend and revenue generated.
🧮Content Marketing ROI Calculator
Calculate the ROI of content marketing efforts from production costs and generated revenue.
🧮Domain Authority Checker
Check and compare domain authority scores for competitive SEO analysis.
🧮Keyword Density Checker
Analyze keyword density and frequency distribution in any webpage or text content.
🧮Meta Tag Length Checker
Verify that title tags and meta descriptions fit within Google SERP display limits.
🧮Robots.txt Generator
Generate a robots.txt file with crawl directives for search engine bots.
🧮Xml Sitemap Size Calculator
Estimate XML sitemap file size and check if it exceeds the 50MB/50K URL limits.
🧮Canonical URL Generator
Generate canonical URL tags to prevent duplicate content issues in search engines.