What test does it use?

A two-proportion z-test (two-tailed), the standard approach for comparing two conversion rates. It flags significance at the common p < 0.05 (95%) threshold.

Why does it say collect more data?

If the p-value is above 0.05 the difference could be chance. Larger sample sizes make true differences easier to detect.

A/B Test Significance & Sample Size Calculator

Q: Why does it say collect more data?

If the p-value is above 0.05 the difference could be chance. Larger sample sizes make true differences easier to detect.

Plan the sample needed per variant, then analyze conversion-rate uplift, p-value and statistical significance.

Variant A VisitorsVariant A ConversionsVariant B VisitorsVariant B Conversions

A Conversion Rate

B Conversion Rate

6.2%

Relative Uplift

24%

P-Value

0.0091

Statistically significant at 95% (p = 0.0091, z = 2.61). Variant B looks like a measurable change.

Two-proportion, equal-allocation planning and two-tailed significance analysis. Sequential peeking, multiple metrics, novelty, traffic mix and experiment design can change the required sample and interpretation.

Loading Tool…

What Is the A/B Test Significance & Sample Size Calculator?

The A/B Test Significance & Sample Size Calculator has two workspaces: plan an equal-allocation two-proportion experiment from baseline rate, minimum detectable lift, confidence and power, then analyze completed visitor and conversion counts with uplift, z-score and p-value.

How It Works

Use Plan Sample Size before launch. After the test, switch to Analyze Results and enter visitors and conversions for control and variant.

When to Use It

Use it when planning or reviewing website, email, product or advertising experiments before deciding whether a measured change is reliable.

Frequently Asked Questions

What test does it use?: A two-proportion z-test (two-tailed), the standard approach for comparing two conversion rates. It flags significance at the common p < 0.05 (95%) threshold.
Why does it say collect more data?: If the p-value is above 0.05 the difference could be chance. Larger sample sizes make true differences easier to detect.

Found an issue? Send Feedback. See our data sources and update policy for tools that depend on changing information.