Classification as Bayes rule with a bold independence assumption. Build a spam filter from a ten-message corpus, tune its word likelihoods, then drag points in a 2D Gaussian view and watch the decision boundary refit in real time.
Bayes rule, then one bold shortcut
Bayes rule says the posterior probability of a class is proportional to its prior times the likelihood of the evidence. The joint likelihood of many words is hopeless to estimate from a small corpus, so naive Bayes assumes every word is conditionally independent given the class and multiplies per-word likelihoods instead. That single assumption turns classification into counting.
P(spam | words) ∝ P(spam) * P(w1 | spam) * P(w2 | spam) * ... * P(wn | spam) naive assumption: words are independent GIVEN the class log odds = log P(spam)/P(ham) + Σ log P(evidence_i | spam)/P(evidence_i | ham) posterior = sigmoid(log odds)
Why it is fast
Training is one pass of counting: label frequencies give the priors, word frequencies per class give the likelihoods. The lab below fits its entire model from ten visible messages, and you can audit every count yourself.
Where the assumption bites
In this corpus, winner never appears in spam without free, yet the model counts each as a fresh, independent clue. Correlated features get double counted, which drives posteriors toward 0 or 1 far more confidently than the evidence justifies.
Why it still works
The decision only needs the correct side of 0.5, not a calibrated probability. Even with overconfident posteriors the argmax is often right, which is why naive Bayes filtered spam for decades and remains a strong baseline for text.
Lab 1: a spam filter you can audit
The corpus below is the entire training set. Priors are its label frequencies; each slider starts at that word's Laplace-smoothed corpus estimate. Drag a slider to overrule the corpus and watch the posterior bars recompute through Bayes rule, then toggle words in the incoming message to change the evidence itself.
The training corpus (all ten messages)
Winner! Click to claim your free prize
vocab hits: winner, click, free
Your free trial ends soon, click now
vocab hits: free, click
Winner alert, your free gift is inside
vocab hits: winner, free
Final notice: click to unlock your free reward
vocab hits: click, free
Team meeting moved to 3pm
vocab hits: meeting
Invoice 204 attached, due Friday
vocab hits: invoice
Lunch after the meeting?
vocab hits: lunch, meeting
Please review the invoice before our meeting
vocab hits: invoice, meeting
Are you free for lunch tomorrow?
vocab hits: free, lunch
Click the shared doc link before standup
vocab hits: click
Priors come straight from the labels: P(spam) = 4/10 = 0.40, P(ham) = 6/10 = 0.60.
Per-word likelihoods (start at the corpus estimates, Laplace smoothed)
Compose the incoming message
Toggle which vocabulary words the new message contains. Absent words are evidence too: each contributes its 1 minus likelihood factor.
contains: free, winner, click
Posterior via Bayes rule
Verdict: SPAM (log odds +4.22)
Where the verdict comes from: one additive term per word
Log odds = prior term + sum of per-word log likelihood ratios. Positive terms push toward spam, negative toward ham. This additivity is the independence assumption at work.
| term | evidence | log ratio | push |
|---|---|---|---|
| prior | 4/10 spam | -0.405 | |
| free | present | +1.200 | |
| winner | present | +1.347 | |
| click | present | +0.986 | |
| meeting | absent | +0.507 | |
| invoice | absent | +0.292 | |
| lunch | absent | +0.292 | |
| total | sum of terms | +4.218 | sigmoid → P(spam) = 98.5% |
Lab 2: the same idea with continuous features
Swap word counts for two continuous features and the recipe is unchanged: fit one Gaussian per feature per class, multiply, apply Bayes rule. Independence now means a diagonal covariance, so each class-conditional ellipse stays axis aligned no matter how tilted your point cloud gets. Drag points (or Tab to one and use the arrow keys) and watch means, variances, and the gold P = 0.5 boundary refit from the actual points. Try dragging one cluster into a slanted line: the fitted ellipse cannot tilt to follow it, and that gap is exactly what the naive assumption throws away.
Legend
Fit from the current points
| class | mean | variance |
|---|---|---|
| ham | (2.94, 3.10) | (0.77, 0.72) |
| spam | (7.25, 6.80) | (0.77, 0.70) |
Priors: ham 8/16 = 0.50, spam 8/16 = 0.50. Refit on every move.
Selected point
Click or Tab to a point to see its posterior under the current fit.
Drag any point with the mouse, or Tab to it and use the arrow keys (step 0.25). Means, variances, ellipses, and the gold boundary refit immediately from the points you see.