Free tool · Testing
A/B Test Significance CalculatorIs your test a winner, or just noise?
Paste your visitors and conversions for both variants. Get the conversion rates, uplift and a proper two-tailed significance check before you roll anything out.
Your numbers
Variant A (control)
Variant B (challenger)
Results
How it works
This runs a standard two-proportion z-test (two-tailed). At 95%+ confidence the difference is very unlikely to be random luck; below 90% you genuinely don’t know yet.
Three rules I hold clients to: decide the sample size before the test, never peek-and-stop the moment it flips green, and run at least one full business cycle (7 days minimum) so weekday and weekend behaviour are both in the data.
How to read your A/B test result
An A/B test, also called a split test, shows two versions of a page or ad to similar audiences and compares how many people convert on each. The difficulty is that two versions will almost never score exactly the same, even when there is no real difference between them. A significance calculator tells you how likely it is that the gap you are seeing is real.
A worked example
Take the numbers pre-filled above. Variant A had 4,200 visitors and 118 conversions, a rate of 2.81%. Variant B had 4,100 visitors and 152 conversions, a rate of 3.71%. That is a relative uplift of 32.0%, and the calculator returns 97.9% confidence. In plain terms, a gap this large would show up by chance only about 2 times in 100 if the two versions were truly equal, so B is a fair winner.
What confidence level should I aim for?
95% is the usual standard, and it is the one I use for landing pages and offers. For low-risk changes such as ad copy, 90% can be enough to move on. Below 90%, treat the result as undecided and keep the test running.
How long should a split test run?
At least seven days, so weekday and weekend behaviour are both in the data, and long enough to reach the sample size you set before starting. Small differences need far more visitors to prove than large ones.
Why not stop as soon as the test turns significant?
Because results swing a lot early on. If you check every day and stop the first time the number crosses 95%, you will call winners that are not real. Fix the sample size first, then read the result once.
What does two-tailed mean?
A two-tailed test checks for a difference in either direction, so it can tell you that the new version is worse as well as better. It is the stricter and safer choice for marketing tests.
Want the full picture, not just the maths?
My AI Ad Audit tears down your actual Meta or Google account and hands you a prioritised fix list.
More free tools
Work with me
Let’s find the money you’re leaving on the table.
Tell me where growth feels stuck. You will get my honest read on what’s leaking, what it’s costing you, and whether I’m the right person to fix it. No pitch deck, no pressure.
Full-stack digital marketer and growth strategist. I connect strategy, paid media, SEO, content, conversion, analytics and AI into one growth system for businesses across the UAE, UK and US.