Master modern resampling methods: build confidence intervals without assuming a particular distribution, test significance by shuffling data, and evaluate models with K-fold cross-validation. Each demo animates live, so you can watch the distributions emerge.
Sign in to save your progress.
Resample with replacement 1,000 times to build the sampling distribution of the mean.
Few assumptions, not none. Bootstrap avoids assuming a particular population distribution, but it does assume your sample is representative (i.i.d. draws), and it can struggle with very small samples or extreme statistics like the maximum.
Test if the difference between two groups is statistically significant. No normality assumption needed, only that observations are exchangeable between groups under the null.
Key idea: Under the null hypothesis (no group difference), any assignment of labels to observations is equally likely.
Watch how each fold serves as the held-out validation set in turn, building an error estimate from data the model never trained on.
Folds are assigned round-robin here (point i goes to fold i mod K), so each segment of this strip represents a fold's points, not a contiguous slice of the dataset. You can see the interleaving in the scatter plot colors below.
Every point is held out exactly once, and the average validation MSE estimates generalization error. Treat the ± std with care, though: the K fold errors are correlated because their training sets overlap, so this spread understates the true uncertainty of the estimate.
Error versus model complexity (a validation curve; a learning curve would instead plot error versus training-set size). Understand when your model is too simple (high bias) or too complex (high variance).
Error from wrong assumptions. A linear model fitting quadratic data has high bias.
Sensitivity to fluctuations in training data. A high-degree polynomial has high variance.
Noise inherent in the data. Cannot be reduced by any model, no matter how good.