Training learns parameters; you choose hyperparameters. Every cell of this 10 by 10 grid is a real decision tree trained in your browser on the same 240-point dataset. Explore the validation landscape, then spend a budget of 12 trials with grid, random, and adaptive search to find the best configuration.
Sign in to save progress
The settings space: a landscape you can only sample
A decision tree learns its splits from data, but its depth limit and minimum leaf size are set by you before training starts. Each combination produces a different model with a different validation score, so tuning is a search over a landscape you cannot see in advance. Evaluating one point of that landscape costs a full training run, which is why real tuning always happens under a budget. Here the playground is small enough to cheat: all 100 trees are trained up front (in a few milliseconds), so you can compare what your budget found against the true optimum.
Grid search
Lay a lattice over the space and evaluate every crossing. Predictable coverage, but each hyperparameter only ever gets a handful of distinct values, and useless rows and columns burn trials.
Random search
Draw configurations uniformly. When only some hyperparameters matter, random search tries more distinct values of each one than a grid does, which is why it often wins (Bergstra and Bengio, 2012, JMLR 13).
Adaptive search
Spend a few trials probing, then concentrate the rest near the best score found so far. This greedy hill-climb is the simplest member of the family that includes Bayesian optimization and Hyperband.
The dataset: 240 points in two interleaved crescents, split 160 train / 80 validation, with 8 percent of training labels deliberately flipped so that an unconstrained tree overfits. Every accuracy on this page is measured on these points by the code running in your browser; nothing is scripted.
Step 1: walk the landscape
Each cell is one trained tree: brighter teal means higher accuracy. Click or arrow-key through at least three configurations and watch the decision boundary change shape, from one straight cut at depth 1 to a jagged, noise-chasing partition at depth 10 with min leaf 1.
min samples per leaf (columns), tree depth (rows)
Flip to training accuracy and look at the bottom-left corner: deep trees with tiny leaves reach 100 percent on the training set while their validation score sinks. That gap is overfitting, measured on real trees, not asserted.
Selected: depth 3, min leaf 8
Filled dots: training points. Hollow rings: validation points. Shaded rectangles: the tree's actual leaf regions (blue predicts class A, pink predicts class B).
Train acc
85.0%
Val acc
87.5%
Leaves
7
Overfit gap
-2.5 pts
Step 2: spend 12 trials wisely
Now pretend the landscape is invisible, as it always is in practice. You can afford 12 training runs out of 100 possible configurations. Run all three strategies and compare what each one finds against the full-sweep optimum.
Run a search to place 12 numbered trial chips on the validation landscape.
| strategy | trials | best val acc | found on trial | gap to sweep best |
|---|---|---|---|---|
| Grid | 12 | not run yet | ||
| Random | 12 | not run yet | ||
| Adaptive | 12 | not run yet | ||
| Full sweep (reference) | 100 | 92.5% | 42 | 0.0 pts |
The full sweep row is what evaluating all 100 configurations would cost and find; your budget buys 12 of them. Seeded runs are deterministic: hit New seed to see how much luck is involved, then run the same strategy again.
Step 3: the budget question
Random search draws 12 of the 100 grid cells without replacement. How likely is it that at least one draw lands in the 10 best cells?