A/B test sample size calculator
How many visitors does an A/B test need to give a result you can trust, and how many weeks will it run? Enter your current rate and the smallest lift you want to detect.
Result
- Visitors per variant
- 21,109
- Total visitors
- 42,218
- Rate to detect
- 2.4%
- Duration (full weeks)
- 3
At 95% confidence (two-sided) and 80% power. Run the test for 3 full weeks and do not read the result before you reach this number.
Formula
- p₁ = current rate · p₂ = p₁ × (1 + relative lift) · p̄ = (p₁ + p₂) ÷ 2
- n = [1.96 × √(2 × p̄ × (1 − p̄)) + 0.84 × √(p₁ × (1 − p₁) + p₂ × (1 − p₂))]² ÷ (p₂ − p₁)²
- duration = 2 × n ÷ weekly visitors, rounded up to a full week
The standard sample size formula for comparing two proportions. Confidence is fixed at 95% (two-sided, z = 1.96) and power at 80% (z = 0.84). n is the number of visitors each variant needs; visitors are split evenly between the two.
Worked example
Say the current conversion rate is 2% and the smallest relative lift you want to detect is 20%: the target rate is 2.4%. The formula gives 21,109 visitors per variant; 42,218 for the two together.
If 20,000 visitors a week enter the test, 42,218 ÷ 20,000 = 2.11; the test runs for 3 full weeks.
How to read the result
- The number is set before the test starts, and the result is not read until you reach it. Looking at interim results and stopping early makes it more likely that you declare a winner where there is no real difference.
- Detecting a smaller difference takes far more visitors: halving the lift you want to detect roughly quadruples the sample you need.
- Days of the week behave differently, so a test runs in full weeks. If the result says months, testing this effect on this page is not realistic; try a bigger change or a page with more visits.
- 80% power means that a real difference of this size can still be missed in one test out of five.
Questions
The questions that come up most about this calculation. If yours is not here, write to us and we will answer within one working day.
How many visitors do you need for an A/B test?
It depends on your current conversion rate and the difference you want to detect. The lower the rate and the smaller the difference, the faster the number grows. For example, detecting a 20% relative lift on a 2% rate at 95% confidence and 80% power takes about 21 thousand visitors per variant.
How long should an A/B test run?
Until you reach the total number of visitors you need, and always in full weeks. Weekday and weekend visitors behave differently, so a test that ends mid-week can mislead. Even with plenty of traffic, it is sound to run for at least one full week and preferably two.
What do 95% confidence and 80% power mean?
95% confidence limits to 5% the chance that the test reports a difference when the two versions do not really differ. 80% power means that if a difference of the size you set really exists, the test will detect it with an 80% probability. Both are values in common use for A/B tests.
Related guides
- Why a fast site sells more: LCP, INP and CLS explainedReading Core Web Vitals as a line in your sales, not a line in a report.
- GA4 setup checklist: from property to DebugView testsFrom the property to consent mode, from internal traffic to testing: the items to work through in order to set up GA4 properly from day one.