A trained model is a black box until you interrogate it. Train a real tree ensemble on 60 visible rows, break each feature with a permutation to measure what mattered globally, then decompose one prediction into per-feature pushes that sum exactly to the output.
Sign in to save progress
Two different questions, two different tools
"Which inputs does my model rely on?" is a global question about the whole model. "Why did it predict THIS for THIS row?" is a local question about one prediction. Both are model inspection: they describe what the trained model uses, which is not the same as what causes churn in the world.
Global: permutation importance
Shuffle one column so its values no longer line up with the right rows, then remeasure accuracy. The information channel is cut while the column's distribution stays intact. A big drop means the model was leaning on that feature.
Local: additive attribution
Start from the model's base rate and credit each feature with a signed push, so that base plus pushes equals the prediction exactly. SHAP is the best-known scheme; for trees we can read an exact additive decomposition straight off the decision paths.
A property of the model, not the world
Importance tells you what this particular model uses. Correlated features split credit in arbitrary ways, and a feature can matter in the world yet score zero because the model found a substitute. Never read importance as causality.
Step 1: meet the data, then train a real forest on it
Every row is visible below: a toy churn dataset generated with a fixed seed. By construction the label leans hard on usage, moderately on tenure, lightly on support calls, and ignores the zip digit entirely. The model never sees that recipe; your job is to recover it from the trained forest.
| Row | Usage (hrs) | Tenure (mo) | Calls | Zip digit | Outcome |
|---|---|---|---|---|---|
| 2 | 11 | 2 | 8 | Churned | |
| 34 | 24 | 5 | 3 | Stayed | |
| 37 | 41 | 2 | 9 | Stayed | |
| 27 | 42 | 3 | 5 | Stayed | |
| 37 | 27 | 1 | 7 | Stayed | |
| 11 | 38 | 2 | 8 | Stayed | |
| 22 | 29 | 6 | 0 | Stayed | |
| 20 | 45 | 6 | 6 | Stayed | |
| 11 | 37 | 7 | 7 | Churned | |
| 28 | 26 | 4 | 8 | Stayed | |
| 12 | 14 | 3 | 1 | Churned | |
| 3 | 41 | 2 | 3 | Stayed | |
| 27 | 11 | 3 | 5 | Churned | |
| 27 | 20 | 1 | 5 | Stayed | |
| 29 | 34 | 3 | 2 | Stayed | |
| 29 | 14 | 1 | 6 | Stayed | |
| 7 | 18 | 7 | 0 | Churned | |
| 28 | 39 | 7 | 4 | Stayed | |
| 7 | 39 | 2 | 6 | Churned | |
| 11 | 22 | 2 | 5 | Stayed | |
| 6 | 36 | 2 | 7 | Stayed | |
| 26 | 10 | 0 | 3 | Stayed | |
| 18 | 48 | 0 | 1 | Stayed | |
| 30 | 29 | 3 | 7 | Stayed | |
| 39 | 17 | 2 | 7 | Stayed | |
| 35 | 3 | 0 | 6 | Stayed | |
| 14 | 20 | 7 | 9 | Churned | |
| 15 | 41 | 4 | 7 | Stayed | |
| 19 | 45 | 2 | 8 | Stayed | |
| 14 | 7 | 6 | 6 | Churned | |
| 11 | 16 | 1 | 1 | Churned | |
| 34 | 41 | 3 | 4 | Stayed | |
| 15 | 23 | 5 | 0 | Churned | |
| 15 | 44 | 5 | 2 | Stayed | |
| 5 | 14 | 5 | 9 | Churned | |
| 1 | 9 | 0 | 7 | Churned | |
| 3 | 7 | 5 | 7 | Churned | |
| 36 | 33 | 4 | 2 | Stayed | |
| 31 | 15 | 1 | 4 | Stayed | |
| 31 | 38 | 0 | 0 | Stayed | |
| 18 | 42 | 5 | 8 | Stayed | |
| 5 | 29 | 3 | 2 | Churned | |
| 9 | 30 | 2 | 9 | Stayed | |
| 9 | 23 | 1 | 0 | Stayed | |
| 16 | 30 | 4 | 2 | Churned | |
| 8 | 31 | 4 | 8 | Churned | |
| 9 | 38 | 7 | 8 | Churned | |
| 36 | 36 | 0 | 5 | Stayed | |
| 22 | 15 | 0 | 9 | Stayed | |
| 6 | 23 | 6 | 7 | Churned | |
| 29 | 26 | 3 | 0 | Stayed | |
| 15 | 36 | 7 | 5 | Churned | |
| 10 | 25 | 7 | 2 | Churned | |
| 8 | 14 | 0 | 5 | Churned | |
| 23 | 30 | 5 | 6 | Stayed | |
| 20 | 14 | 1 | 3 | Stayed | |
| 30 | 11 | 6 | 4 | Churned | |
| 15 | 15 | 6 | 0 | Churned | |
| 18 | 41 | 5 | 5 | Churned | |
| 12 | 33 | 5 | 1 | Churned |
The model
A bagged ensemble of 12 CART trees, depth at most 3, at least 4 rows per leaf, each trained on a deterministic bootstrap of these 60 rows. Leaves store the mean churn label, so the forest outputs a probability. Training is real and runs in your browser; the fixed seed means retraining rebuilds the same forest.
0 of 12 trees built
Step 2: break features one at a time
Shuffle a column and the table in Step 1 updates: the values are really permuted (a fixed, seeded shuffle) and the accuracy bar is recomputed by running all 60 rows back through the forest. Shuffle a column the model relies on and accuracy craters; shuffle the noise column and nothing happens.
Monthly usage (hrs)
hours of product use per month
Tenure (mo)
months as a customer
Support calls
support tickets in the last quarter
Zip last digit
last digit of the zip code, pure noise by construction
Train the forest in Step 1 first; without a model there is no accuracy to break.
Accuracy on the 60 rows, recomputed live
Waiting for a trained forest.
Measured permutation importance
Shuffle at least two different columns above to unlock the per-feature measurement. Try one column the model should care about and one it should not.
Step 3: which feature does YOUR forest lean on most?
Answer from the evidence you just generated: the correct option is whichever feature has the largest measured accuracy drop for the forest you trained, not a hardcoded answer key.
Train the forest first; the answer is measured from it.
Step 4: explain one prediction, exactly
Global importance says what the model uses on average. For a single customer we can do better: walk each tree's decision path and credit every split to the feature that made it. The result is a SHAP-style waterfall whose bars sum exactly to this forest's prediction for that row.
Train the forest first: attributions are read off the trained trees.
The waterfall appears once you have a trained forest and a selected row.