A/B Test Duration Estimator
Calculate how long A/B tests need to run based on traffic and MDE. Enter values for instant results with step-by-step formulas.
Worked Examples
Example 1: High-Traffic E-commerce
Problem:50,000 daily visitors, 4% conversion, want to detect 10% lift, 80% power, 95% significance, A/B test.
Solution:Sample size: 15,764 per variant. Duration: 1 day (but run at least 7 for day-of-week effects). Test is well-powered for quick decisions.
Result:15,764/variant | ~1 day math | 7 days minimum | Quick test
Example 2: Medium Traffic SaaS
Problem:3,000 daily visitors, 2% conversion, detect 15% lift, 80% power, 95% significance.
Solution:Sample size: 6,958 per variant. Duration: 5 days. Manageable test length. Could detect smaller effects with more time.
Result:6,958/variant | 5 days | Standard duration | Consider 10% MDE option
Example 3: Low Traffic Landing Page
Problem:500 daily visitors, 5% conversion, detect 10% lift, 80% power.
Solution:Sample size: 12,534 per variant. Duration: 50 days. Very long test. Either accept 20% MDE (13 days) or find ways to increase traffic.
Result:12,534/variant | 50 days | Too long | Increase MDE or traffic
Frequently Asked Questions
How long should an A/B test run?
A/B tests should run until they reach statistical significance (usually 95% confidence) with sufficient power (80%+). This depends on traffic, conversion rate, and minimum detectable effect. Never stop early based on preliminary results—wait for the predetermined sample size.
Why can't I stop a test when it shows significance?
Significance fluctuates during a test. Early significance is often false—with enough peeks, random variation will appear significant. Pre-commit to sample size and don't peek. Sequential testing methods exist but require different analysis approaches.
How does baseline conversion rate affect duration?
Lower conversion rates require more samples. At 1% conversion, you need many more visitors to get enough conversions for reliable statistics than at 10% conversion. Tests on low-conversion actions (purchases vs. clicks) take longer.
Should I include weekends in my test?
Yes, always run tests for complete weeks. User behavior differs by day of week. Ending mid-week biases results toward that day's patterns. Minimum one full week; preferably 2-4 weeks for stable results.