A/B Test Statistical Significance
Determine if your A/B test results are statistically significant using z-score analysis, conversion rate uplift, and confidence level calculation.
Results
What is it?
Statistical significance tells you whether the difference in conversion rates between Variant A and Variant B is due to a real effect or random chance. The z-score quantifies this difference in standard deviations, which maps to a confidence level. A result is typically considered significant at 95% confidence (z > 1.96).
How to use
1. Enter the number of visitors and conversions for each variant. 2. The calculator computes conversion rates, uplift percentage, z-score, and confidence level. 3. Use the confidence level to decide if Variant B is a genuine winner.
Example scenario
Variant A: 1,000 visitors, 50 conversions (5% CVR). Variant B: 1,000 visitors, 65 conversions (6.5% CVR). Uplift = 30%. Z-score approx 1.76, Confidence approx 92%. This is not yet at the 95% threshold � collect more data before declaring a winner.
Pro tip
Z > 1.65 = 90% confidence, Z > 1.96 = 95%, Z > 2.58 = 99%. Never stop a test early when your confidence crosses 95% � sequential testing bias inflates false positives. Run tests for at least 1�2 full business cycles (typically 2 weeks) regardless of early results. Use a minimum detectable effect (MDE) of 10�20% to calculate the required sample size before launching.