Ab Test Duration Calculator
Calculate how long to run your A/B test and the required sample size. Free A/B test duration calculator with statistical significance support.
Ab Test Duration Calculator
Calculator
Adjust values & calculateEnter your values below. Every result is computed in your browser — no data is sent to any server.
Formula: n = (Zα√(2p̄q̄) + Zβ√(p₁q₁+p₂q₂))² / (p₂-p₁)²
Worked example — Run test for at least 8 days with 77,144 total visitors needed
Formula
n = (Zα√(2p̄q̄) + Zβ√(p₁q₁+p₂q₂))² / (p₂-p₁)²
The sample size per variation is calculated using the standard two-proportion z-test formula, where Zα is the z-score for your significance level, Zβ is the z-score for statistical power (80%), p₁ is the baseline conversion rate, and p₂ is the target conversion rate (baseline × (1 + MDE)).
Worked Examples
Example 1: E-commerce Checkout Test
Problem:An e-commerce site has a 3% conversion rate, 10,000 daily visitors, and wants to detect a 10% relative improvement at 95% significance.
Solution:Baseline: 3%, Target: 3.3%, MDE: 10% Sample per variation: ~38,572 Total sample: ~77,144 With 5,000 visitors per variation/day: ~8 days
Result:Run test for at least 8 days with 77,144 total visitors needed
Example 2: Landing Page Headline Test
Problem:A SaaS landing page converts at 8% with 2,000 daily visitors. They want to detect a 15% relative improvement at 95% significance.
Solution:Baseline: 8%, Target: 9.2%, MDE: 15% Sample per variation: ~5,286 Total sample: ~10,572 With 1,000 visitors per variation/day: ~6 days
Result:Run test for at least 6 days with 10,572 total visitors needed
Frequently Asked Questions
How long should I run an A/B test?
The duration depends on your daily traffic, baseline conversion rate, and the minimum effect you want to detect. Most A/B tests require at least 1-2 weeks to gather statistically significant results. Never stop a test early just because one variation appears to be winning — this leads to false positives. Use Ab Test Duration Calculator to determine the exact number of days needed based on your specific parameters.
What is Minimum Detectable Effect (MDE)?
Minimum Detectable Effect is the smallest improvement you want your test to be able to detect. A smaller MDE requires more traffic and longer test duration. For example, detecting a 1% relative improvement requires roughly 100x more samples than detecting a 10% improvement. Choose an MDE that represents a meaningful business impact — typically 5-20% relative change for most tests.
What statistical significance level should I use?
The standard is 95% significance, meaning there is only a 5% chance the result is due to random chance (false positive). Use 90% for early-stage or low-traffic tests where speed matters more than precision. Use 99% for critical changes like checkout flow modifications where false positives are costly. Higher significance requires larger sample sizes and longer test durations.
What is statistical power in A/B testing?
Statistical power (typically set at 80%) is the probability that your test will correctly detect a real difference when one exists. A power of 80% means there is a 20% chance of a false negative — missing a real improvement. Higher power (e.g., 90%) reduces false negatives but requires larger sample sizes. The industry standard of 80% offers a good balance between test duration and reliability.
Why should I not stop an A/B test early?
Stopping a test early (also called 'peeking') inflates the false positive rate dramatically. Early in a test, random fluctuations can make one variation appear significantly better. This is known as the multiple comparisons problem. Studies show that peeking at results daily with 95% significance can result in actual false positive rates of 25-30%. Always run the test for the full calculated duration to get reliable results.
What is the difference between relative and absolute MDE?
Ab Test Duration Calculator uses a relative MDE: the target rate is p2 = p1 x (1 + MDE), so a 3% baseline with a 10% MDE means detecting a lift to 3.3%. An absolute MDE of 10 percentage points on the same baseline would mean 13%, a far larger effect needing much less traffic. Because the denominator of the sample-size formula is (p2 - p1) squared, mixing the two conventions can misstate the required sample size by orders of magnitude. Always confirm which definition your tool uses.
Why does the sample size formula use a pooled proportion?
The pooled proportion p-bar is the average of the control and variant rates, and it appears in the Z-alpha term because the null hypothesis assumes both variations share one true conversion rate, so the standard error of the difference is estimated from that single pooled rate. The Z-beta term instead uses the separate variances p1q1 and p2q2, because power is evaluated under the alternative hypothesis where the two rates genuinely differ. Using one variance estimate for both terms would bias the result.
How does testing more than two variations change the required duration?
Sample size per variation stays roughly the same, but daily traffic is now divided among more arms. With three variations instead of two, each arm receives a third rather than a half of visitors, so duration grows by about 50%. You also face a multiple comparisons problem: testing several variants against one control raises the chance that at least one shows a spurious win, so many teams tighten the significance level, for example by dividing alpha by the number of comparisons, which pushes sample size higher again.
Why should a test run in whole weeks rather than the exact calculated days?
Conversion behaviour is cyclical: weekday and weekend traffic often differ in intent, device mix and purchase rate. Stopping after a partial week over-weights whichever days happen to be included and biases the estimate. If the calculator returns 9 days, running 14 covers two complete weekly cycles and averages out day-of-week effects. The extra traffic also adds power. The practical rule is to round the calculated duration up to the next whole number of weeks before you launch.
What is the novelty effect and how does it distort test results?
The novelty effect is a temporary lift caused by returning visitors reacting to a change simply because it looks new, rather than because it is genuinely better. It usually decays within days as the change stops being unfamiliar. In a short test this transient bump can be mistaken for a real improvement. Running for the full calculated duration, and comparing the measured effect in the first and second halves of the test window, helps show whether the lift is holding steady or fading away.
References
Reviewed for accuracy by Daniel Agrici, Founder & Lead Developer · Editorial policy
Related Calculators
🧮Marketing ROI Calculator
Calculate marketing campaign return on investment from spend and revenue generated.
🧮Content Marketing ROI Calculator
Calculate the ROI of content marketing efforts from production costs and generated revenue.
🧮Domain Authority Checker
Check and compare domain authority scores for competitive SEO analysis.
🧮Keyword Density Checker
Analyze keyword density and frequency distribution in any webpage or text content.
🧮Meta Tag Length Checker
Verify that title tags and meta descriptions fit within Google SERP display limits.
🧮Robots.txt Generator
Generate a robots.txt file with crawl directives for search engine bots.
🧮XML Sitemap Size Calculator
Estimate XML sitemap file size and check if it exceeds the 50MB/50K URL limits.
🧮Canonical URL Generator
Generate canonical URL tags to prevent duplicate content issues in search engines.