Bias does not wait for the analysis. It walks in with the data, at the moment of collection, and whoever the channel could never reach cannot be recovered by any model downstream. Pick channels, armor bombers, and learn to ask the one question your table cannot answer for you.
Sign in to save progress
The table is not the world
Every dataset is the output of a collection process, and that process has a shape: a phone number dialed, an app installed, a prompt answered, a plane that made it home. Rows that the process could not produce are not missing values you can impute. They are missing people, and no amount of cleaning, weighting, or modeling can conjure information the channel never captured. The estimator math of sampling designs belongs to the sampling methods and bias guide. This guide stays on the craft question that comes before any formula: who could never appear in this data?
How this playground stays honest: the city of 400 residents (seed 20260708) and the fleet of 500 simulated sorties (seed 19440615) are generated once with a seeded generator. Every average, gap, hit count, and survivor count on this page is computed live from those records. Nothing is faked.
A city agency wants the average daily hours residents spend online. The full census below is the truth it will never afford. Every realistic channel reaches a systematically different slice of the city. Switch channels and watch the estimate drift away from a truth that never changed.
The city: 400 people, sorted by age
Dimmed dots are people this channel can never reach. They are not refusing; they are structurally invisible. Nobody. Real collection channels never look like this.
Pick a collection channel
True city average
5.88 h
all 400 residents
Your sample says
5.88 h
n = 400 reached
Gap vs truth
+0.00 h
Everyone. This is the ground truth you almost never get to see.
Channel-only gap vs truth (no non-response layer)
Bars appear as you visit each channel. Compare at least two against the truth. Notice the mall intercept: its gap is small because it mixes students (high) with retirees (low), yet the entire working-age middle is still missing. A small gap on one metric is not a certificate that the channel is unbiased.
Three ways people vanish before the first row is written
Selection at the channel
The channel itself filters who can be observed. A landline survey cannot dial someone without a landline. You just watched this move an estimate above.
Survivorship
You only observe entities that survived long enough to be observed: returning planes, companies still in business, users who did not churn. The scenario below is the classic case.
Non-response
Reachable people who never answer, and whose silence correlates with what you are measuring. The checkbox in section 1 stacks this on top of channel selection.
In World War II, the US military worked with the Statistical Research Group, a wartime team of statisticians based at Columbia University, on questions about aircraft damage. Abraham Wald worked on that problem. As the story is usually told, the instinct was to add armor where returning bombers showed the most bullet holes; Wald pointed at the planes nobody could inspect, the ones that never came back. His actual memoranda built methods for estimating survival probabilities from damage observed only on survivors, and the famous anecdote compresses that work into one inversion: the sparse patches on returners mark the hits that kill.
The fleet below is a seeded simulation built for this exercise, not historical data. You get to make the call Wald made, with the same information he had: hits on returners only.
Hits on the 369 planes that RETURNED
Your move
Armor prevents 70 percent of the losses a hit in that zone would otherwise cause. The re-flight reuses the exact same 500 hits and survival rolls, so the only thing that changes is your armor.
The fix is not a formula. It is a habit: before trusting any number, reconstruct the collection process and list the groups it structurally excludes. Practice on a scenario every product team has met.
A transit agency measures rider satisfaction with a prompt inside its mobile app, shown after each completed trip. The dashboard proudly reports 4.5 out of 5. Which groups could never appear in that number?
Mark every group that is invisible to this channel. Feedback is instant.
Hidden groups found: 0/4 · False alarms: 0