Confidence Interval Builder
For sample means and proportions, with a built-in sample-size calculator for A/B tests. Pick a target margin of error, get the required N.
Inputs
Advertisement
Result
Visualization
A/B testing without statistical nonsense
The hardest part of A/B testing isn't computing the p-value — it's deciding before the test starts how big a sample you need. Stopping "when significant" inflates false-positive rates to 20%+ (see Simmons, Nelson, & Simonsohn, 2011). Use the sample-size calculator to commit to N up-front.
Who uses it
- PMs / growth teams sizing A/B tests before launch.
- Survey researchers estimating population proportions.
- Students doing stats homework without paying for SPSS.
Limitations
- Sample-size formula assumes normal approximation is valid — fine for np ≥ 10 and n(1−p) ≥ 10.
- We don't model unequal variance (Welch's t-test) — for that, use the dedicated tool in your stats package.
- For sequential A/B testing (peeking), use a different methodology (e.g. mSPRT, always-valid inference).
Frequently asked questions
How do I use Confidence Interval Builder step by step without mistakes?
Follow the order given in the usage section: Build confidence intervals for means and proportions, and compute required sample size for an A/B test with a target margin of error. The step-by-step part matters less than the unit check that sits inside it. Confirm every unit matches the thing you are actually trying to measure before you press calculate - that single check catches more wrong answers than any other habit, and it costs about ten seconds.
What are the common mistakes when using Confidence Interval Builder?
Most common mistakes come down to four habits: entering 20 when the field expects 0.2 because the field is a percentage; leaving a field blank and accepting the default without checking what it stands for; changing one input and reading a result that was calculated before the change; and quoting the output as an exact figure when the page describes it as a reference estimate. Press calculate again after any edit, and treat the number as a reference point rather than a final answer.
How do I read the result from Confidence Interval Builder?
Read the final figure first, then come back to this section for the conditions behind it: Build confidence intervals for means and proportions, and compute required sample size for an A/B test with a target margin of error. Keep two things apart: the number itself, which comes from the values you typed, and the assumptions it depends on, which decide whether that number is meaningful. The page sets out what a higher or lower result means for your case, so check that direction as well as the magnitude.
Can I act on the Confidence Interval Builder result directly?
Treat it as one reference sheet before you decide, rather than as the answer itself: Sample-size formula assumes normal approximation is valid — fine for np ≥ 10 and n(1−p) ≥ 10. We don't model unequal variance (Welch's t-test) — for that, use the dedicated tool in your stats package. For sequential A/B testing (peeking), use a different metho The output describes a magnitude under the conditions you entered. It does not include what it actually costs to act, what might change while you act, or how the other side behaves. Before you commit, compare the result against reality and note which assumptions do not hold where you are.
What inputs does Confidence Interval Builder need?
Confidence Interval Builder takes 8 inputs: Type、Confidence level、Sample mean x̄、Sample size n、Std dev σ (or s)、Successes x、Baseline proportion p̂、Minimum detectable effect (MDE, absolute). They are listed in the order they feed the calculation, so an earlier value can change a later one. Do not skip a field you do not recognise - work out what it means first, because a silently guessed value produces a number that looks reasonable and is not. Before typing anything, make sure every figure uses the same unit and the same time period.
What should I check before I run Confidence Interval Builder?
Three checks pay for themselves. First, confirm the question you need answered is really inside the scope of this tool rather than a neighbouring one. Second, confirm that the figures you have are drawn from the same source and the same period - mixing monthly and annual numbers, or two different definitions of the same quantity, quietly distorts the result. Third, keep the key inputs written down, so that later you can tell which set of numbers produced a figure you are about to quote.