Confidence intervals that decision-makers understand

"The average delivery time is 34 minutes" sounds precise, but it came from a sample of 60 orders. A confidence interval (CI) adds the missing honesty: "34 minutes, give or take about 3". Managers can then judge whether the uncertainty matters for the decision.

The idea

estimate  +/-  critical value x standard error

For a 95% CI the critical value is about 1.96 for large samples (normal) or a slightly larger t value for small ones.

Correct interpretation: if we repeated the sampling many times, about 95% of the intervals built this way would contain the true value. It is not "there is a 95% probability the true mean is in this particular interval" - that is a Bayesian statement.

CI for a mean (t-interval)

import numpy as np
from scipy import stats

rng = np.random.default_rng(11)
minutes = rng.gamma(shape=9, scale=3.8, size=60)       # delivery times, Pune

mean = minutes.mean()
sem = stats.sem(minutes)                               # s / sqrt(n)
low, high = stats.t.interval(0.95, df=len(minutes) - 1, loc=mean, scale=sem)
print(f"Mean {mean:.1f} min, 95% CI [{low:.1f}, {high:.1f}]")

CI for a proportion

For conversion rates, repeat purchase rates or defect rates, use a Wilson interval - it behaves well even with small counts:

from statsmodels.stats.proportion import proportion_confint

low, high = proportion_confint(count=87, nobs=1_250, alpha=0.05, method="wilson")
print(f"Repeat rate {87 / 1250:.1%}, 95% CI [{low:.1%}, {high:.1%}]")

The bootstrap: CIs for anything

For medians, ratios or percentiles there may be no simple formula. The bootstrap resamples your data with replacement thousands of times:

res = stats.bootstrap((minutes,), np.median, confidence_level=0.95,
                      n_resamples=5_000, random_state=1)
print(res.confidence_interval)

What changes the width?

Change	Effect on width
Larger sample (4x n)	Roughly halves
Higher confidence (95% to 99%)	Wider
More variable data	Wider
Biased sampling	No change in width - but the interval is wrong

Presenting CIs

• Report the interval, not just the point: "Average bill ₹486 (95% CI ₹462-₹510)".
• In charts, use error bars or shaded bands, and say in the caption what they show.
• Overlapping CIs of two groups do not prove "no difference"; test the difference directly (its own CI).
• Round sensibly - a CI of [33.8126, 36.2291] minutes implies false precision.

Key takeaways

• A CI expresses sampling uncertainty, not bias or data-quality problems.
• Use t-intervals for means, Wilson for proportions, bootstrap for everything else.
• Width shrinks with the square root of n, so precision is expensive.