Chi-square & ANOVA tests

A t-test compares two means. Real business questions often involve more: "Does preferred payment method differ by city?" or "Do three store layouts produce different average bills?" Chi-square tests and ANOVA handle these.

Choosing the test

Question	Data	Test
Are two categorical variables related?	Counts in a table	Chi-square test of independence
Do observed counts match expected shares?	One categorical variable	Chi-square goodness-of-fit
Do 3+ groups have different means?	Numeric outcome, one grouping	One-way ANOVA
Same, but data very skewed or ordinal	Ranks	Kruskal-Wallis

Chi-square test of independence

A payments team counts transactions by city and method:

import pandas as pd
from scipy import stats

table = pd.DataFrame(
    {"UPI": [820, 610, 540], "Card": [240, 300, 180], "Cash": [140, 90, 280]},
    index=["Bengaluru", "Mumbai", "Patna"],
)
chi2, p, dof, expected = stats.chi2_contingency(table)
print(f"chi2={chi2:.1f}, dof={dof}, p={p:.2e}")
print(pd.DataFrame(expected, index=table.index, columns=table.columns).round(0))

• H0: payment method is independent of city.
• The test compares observed counts with expected counts under independence. A tiny p-value means the mix differs by city.
• To see where, compare observed with expected: Patna has far more cash than expected.
• Requirement: expected counts should mostly be 5 or more; merge sparse categories otherwise.

Effect size matters too. Cramer's V (0 = none, 1 = perfect association) tells you whether a significant result is large enough to care about:

import numpy as np

n = table.to_numpy().sum()
cramers_v = np.sqrt(chi2 / (n * (min(table.shape) - 1)))
print(f"Cramer's V = {cramers_v:.2f}")

Goodness-of-fit

Is footfall spread evenly across weekdays?

observed = [410, 380, 395, 402, 450, 610, 580]
print(stats.chisquare(observed))         # expected = equal counts by default

One-way ANOVA

Three layouts are trialled in comparable textile stores:

import numpy as np

rng = np.random.default_rng(8)
layout_a = rng.normal(1450, 300, 60)     # average bill, INR
layout_b = rng.normal(1520, 300, 60)
layout_c = rng.normal(1630, 300, 60)
f, p = stats.f_oneway(layout_a, layout_b, layout_c)
print(f"F={f:.2f}, p={p:.4f}")

ANOVA compares variation between group means with variation within groups. A small p-value says "at least one mean differs" - not which one.

Post-hoc comparisons

Follow a significant ANOVA with Tukey's HSD, which controls the error rate across all pairs:

from statsmodels.stats.multicomp import pairwise_tukeyhsd

values = np.concatenate([layout_a, layout_b, layout_c])
groups = ["A"] * 60 + ["B"] * 60 + ["C"] * 60
print(pairwise_tukeyhsd(values, groups, alpha=0.05))

Running many separate t-tests instead inflates false positives: with three pairs at alpha 0.05, the chance of at least one false alarm is about 14%.

Assumptions and alternatives

• Independent observations (not the same customer counted repeatedly).
• Roughly normal residuals and similar variances - check with stats.levene; if variances differ use Welch's ANOVA.
• Skewed data or outliers: stats.kruskal(layout_a, layout_b, layout_c).

Key takeaways

• Chi-square works on counts, never on percentages or means.
• ANOVA tells you whether means differ; Tukey tells you which.
• Always report an effect size alongside p.