Two of the quietest decisions in any analysis are the level you aggregate at and the number you divide by. Neither feels like a decision, and either one can flip your conclusion. Flip both, on live data, and learn to hear the click.
Sign in to save progress
One dataset, many honest summaries
An average is not a fact about the world. It is a fact about the world plus two choices: which rows you pool together, and what you divide by. Change either choice and the same table can tell the opposite story, with no error anywhere. The classic pooled-vs-group flip is Simpson's paradox, and its causal reading, when should you trust the grouped view, lives in the Simpson's paradox guide. This guide stays on the analyst's side of the desk: aggregation level and denominator as everyday grouping craft, the two questions to ask before you believe any average.
How this playground stays honest: both clinics treat 800 whole patients whose counts follow the fixed recovery odds printed on their cards, and the stores keep their full user and session ledgers on screen. Every rate, gap, tie point, and winner on this page is recomputed from those integer counts on every slider move and toggle. Nothing is scripted.
Northside is the better clinic for mild cases and the better clinic for severe cases. Referrals send it the harder mix. Drag the case mix sliders, toggle between the pooled table and the by-severity table, and find the point where the pooled winner flips even though Northside never stops winning both groups.
Recovery rate, 800 patients per clinic
Pooled verdict: Northside leads by 5.4 pts
| Clinic | Severity | Treated | Recovered | Rate |
|---|---|---|---|---|
| Northside | mild | 560 | 521 | 93.0% |
| Northside | severe | 240 | 175 | 72.9% |
| Riverside | mild | 560 | 487 | 87.0% |
| Riverside | severe | 240 | 166 | 69.2% |
Every rate above is recovered divided by treated from this table, and the table is rebuilt from the sliders on every move.
The referral center. Better with mild cases and better with severe cases.
560 mild · 240 severe · fixed skill: mild 93%, severe 73%
The neighborhood clinic. Worse with mild cases and worse with severe cases.
560 mild · 240 severe · fixed skill: mild 87%, severe 69%
Hunt the reversal point
With Riverside as it is now, the two pooled rates tie near 57% severe at Northside, computed from the fixed skills and Riverside's current pooled rate. Drag Northside past that point: the pooled winner flips while both group winners stay put.
Two online stores sell the same goods. Store A is a browsing app that people open constantly; Store B is an errand people run once. The raw ledger is identical either way you read it. Switch the denominator and watch the conversion crown change heads.
Purchase rate
This framing says: Store A converts better, by 10.0 pts
Per user asks: how likely is a person to buy this month? Per session asks: how likely is a single visit to end in a purchase? Both are honest fractions of the same table below. They answer different questions, so they can crown different winners.
Store A is the app people open on the couch. More browsing means more sessions, while its 60 buyers still buy exactly once each.
100 users × 4 = 400 sessions
The whole dataset
| Store | Users | Buyers | Sessions |
|---|---|---|---|
| Store A | 100 | 60 | 400 |
| Store B | 100 | 50 | 120 |
Every buyer purchases in exactly one session, so purchase sessions equal buyers in both stores. Both rates on the left are computed from this table on every change.
The two questions to ask before trusting any average
Aggregated over what?
Pooling rows mixes their compositions. If the groups being compared carry different mixes, the pooled number rewards the easier mix, not the better performer. Always look at both levels before picking one.
Divided by what?
A rate is a claim about its denominator. Per user, per session, per order, and per dollar are four different questions wearing the same percent sign. Say the denominator out loud in every headline you write.
When the views disagree
A pooled-vs-grouped conflict is a signal, not a bug: some mix variable differs between the things you are comparing. Whether the pooled or the grouped answer is the right one is a causal question; the Simpson's paradox guide takes it from here.