Worked Examples
Example 1: E-commerce Checkout
Problem: Baseline 2%, MDE 10% (Relative), Power 80%, Sig 5%
Solution: 2% -> 2.2% (0.2 percentage-point lift). Using the calculator's two-variant frequentist sample-size formula, you need about 80,706 visitors per variation to reliably detect that change.
Result: 161,412 Total Visitors
Example 2: High Traffic Landing Page
Problem: Baseline 15%, MDE 5% (Relative), Power 90%, Sig 5%
Solution: 15% -> 15.75% (0.75 percentage-point lift). At 90% power and 95% confidence, the calculator estimates roughly 48,622 visitors per variation.
Result: 97,244 Total Visitors
Frequently Asked Questions
Why does 80% Power matter?
Power (1 - Ξ²) is the probability of correctly finding a winner if one actually exists. 80% Power means you have a 20% chance of missing a real winner (Type II error). 80% is the industry standard balance between risk and traffic needs.
Can I stop the test early?
No! Stopping as soon as you see significance ('peeking') dramatically increases your False Positive rate. You must commit to the sample size beforehand.
How long should I run the test?
Divide the Total Sample Size by your daily traffic. Ideally, run for full business cycles (e.g., 2 full weeks) to account for day-of-week variances.
How do I get the most accurate result?
Enter values as precisely as possible using the correct units for each field. Check that you have selected the right unit (e.g. kilograms vs pounds, meters vs feet) before calculating. Rounding inputs early can reduce output precision.
Why might my result differ from another tool or reference?
Differences typically arise from rounding conventions, the specific version of a formula (for example, simple vs compound interest), or unit inconsistencies between inputs. Check that both tools are using the same formula variant and the same units. The References section links to the authoritative source behind the formula used here.
Can I use A/B Test Sample Size & Power on a mobile device?
Yes. All calculators on NovaCalculator are fully responsive and work on smartphones, tablets, and desktops. The layout adapts automatically to your screen size.
Background & Theory
The A/B Test Sample Size & Power Calculator applies the following established principles and formulas.
Physics is the fundamental natural science concerned with matter, energy, and the interactions between them. Classical mechanics, founded on Newton's three laws of motion, provides the framework for analyzing the motion of objects. The first law states that an object remains at rest or in uniform motion unless acted upon by a net external force. The second law quantifies this relationship: F = ma, where force equals mass times acceleration in SI units of newtons (N = kgΒ·m/sΒ²). The third law establishes that every action produces an equal and opposite reaction.
Kinematics describes motion without reference to its causes. The four fundamental equations relate displacement s, initial velocity u, final velocity v, acceleration a, and time t: v = u + at, s = ut + Β½atΒ², vΒ² = uΒ² + 2as, and s = Β½(u + v)t. These assume constant acceleration and are foundational for solving projectile motion, free fall, and linear dynamics problems.
Energy conservation underpins much of physics. Kinetic energy is KE = Β½mvΒ², where m is mass in kilograms and v is speed in meters per second. Gravitational potential energy is PE = mgh, where g β 9.81 m/sΒ² near Earth's surface and h is height in meters. The work-energy theorem states that the net work done on an object equals its change in kinetic energy: W = ΞKE.
Electricity and circuits rely on Ohm's law: V = IR, where voltage V is in volts, current I in amperes, and resistance R in ohms. Electrical power is P = IV = IΒ²R = VΒ²/R, measured in watts. Wave mechanics connects frequency f, wave speed v, and wavelength Ξ» through f = v/Ξ», with frequency in hertz (Hz). Pressure is defined as force per unit area, P = F/A, in pascals (Pa = N/mΒ²). The ideal gas law PV = nRT links pressure, volume, moles n, the gas constant R = 8.314 J/(molΒ·K), and absolute temperature in kelvin. Gravitational force between two masses follows Newton's law of universal gravitation: F = Gmβmβ/rΒ², where G = 6.674Γ10β»ΒΉΒΉ NΒ·mΒ²/kgΒ² is the gravitational constant.
History
The history behind the A/B Test Sample Size & Power Calculator traces back through the following developments.
The history of physics spans over two millennia, beginning with the natural philosophy of ancient Greece. Aristotle (384β322 BCE) proposed that all matter consisted of four elements and that objects moved toward their natural place, with heavier objects falling faster than lighter ones. While largely incorrect, his systematic approach to explaining nature dominated Western thought for nearly 2,000 years.
The Scientific Revolution overturned Aristotelian physics. Galileo Galilei (1564β1642) performed groundbreaking experiments on inclined planes and falling bodies, demonstrating that all objects fall with the same acceleration regardless of mass, and established the principle of inertia. His use of mathematics to describe motion was revolutionary.
Isaac Newton synthesized these developments in his landmark Principia Mathematica (1687), laying out the three laws of motion and the law of universal gravitation. Newton's framework unified terrestrial and celestial mechanics, explaining planetary orbits with the same equations governing a falling apple. His calculus provided the mathematical language for expressing rates of change.
The 19th century brought two major theoretical achievements. James Clerk Maxwell formulated his equations of electromagnetism between 1861 and 1862, unifying electricity, magnetism, and optics, and predicting the existence of electromagnetic waves traveling at the speed of light. Thermodynamics was developed by Carnot, Clausius, and Kelvin, establishing the laws governing heat, work, and entropy.
The 20th century produced two revolutions that fundamentally altered the classical picture. Albert Einstein published the special theory of relativity in 1905, showing that space and time are not absolute but relative to the observer, and that mass and energy are equivalent via E = mcΒ². His general theory of relativity in 1915 reinterpreted gravity as the curvature of spacetime. Simultaneously, quantum mechanics emerged from the work of Planck, Bohr, Heisenberg, and SchrΓΆdinger, revealing that at atomic scales energy is quantized and particles exhibit wave-particle duality. These developments culminated in the Standard Model of particle physics, which describes all known fundamental particles and three of the four fundamental forces.