NeuroNomixer logoNeuroNomixer
  • Home
  • Blog
  • Visual Guides
  • Authors
  • Contact
Sign InSign Up
NeuroNomixer logo
HomeBlogAuthorsContactPrivacy Policy

© 2026 NeuroNomixer — Built with Next.js & Tailwind CSS

Visual Guides/Bootstrap, Permutation Tests & Cross-Validation
Unit 15 · Resampling & Modern Methods

Bootstrap, Permutation Tests & Cross-Validation

Master modern resampling methods: build confidence intervals without assuming a particular distribution, test significance by shuffling data, and evaluate models with K-fold cross-validation. Each demo animates live, so you can watch the distributions emerge.

Guide Progress0 / 3 sections complete
Bootstrap CI
Permutation Test
Cross-Validation

Sign in to save your progress.

Section 1

Bootstrap Confidence Intervals

Resample with replacement 1,000 times to build the sampling distribution of the mean.

Original Sample (n=30)x̄ = 50.86
mean3542.55057.565
0 / 1000
Bootstrap Distribution of MeansNo data yet

Results

Sample Mean50.857
Bootstrap SE—
95% CI Lower—
95% CI Upper—

How Bootstrap Works

  1. 1.Start with your original sample of n observations.
  2. 2.Draw n values at random WITH replacement from the sample.
  3. 3.Compute the statistic (e.g., mean) on this resample.
  4. 4.Repeat 1,000 times to get a distribution of the statistic.
  5. 5.Take the 2.5th and 97.5th percentiles as the 95% CI.

Few assumptions, not none. Bootstrap avoids assuming a particular population distribution, but it does assume your sample is representative (i.i.d. draws), and it can struggle with very small samples or extreme statistics like the maximum.

Section 2

Permutation Test

Test if the difference between two groups is statistically significant. No normality assumption needed, only that observations are exchangeable between groups under the null.

40506070
Control (Group 1)x̄ = 49.66
40506070
Treatment (Group 2)x̄ = 54.73
Observed Difference (Group 2 − Group 1)
+5.065
0 / 5000
Null Distribution (permuted differences)No data yet

Results

Group 1 Mean49.660
Group 2 Mean54.725
Observed Diff+5.065
p-value (two-tailed)—

How Permutation Test Works

  1. 1.Compute observed difference between group means.
  2. 2.Pool all observations into one combined dataset.
  3. 3.Randomly shuffle and re-split into two groups of same size.
  4. 4.Compute the difference for this random permutation.
  5. 5.Repeat 5,000 times to build the null distribution.
  6. 6.p-value = (k + 1) / (N + 1), where k = permutations with |diff| ≥ |observed|. Both tails count, and the +1 (the observed labeling itself) keeps p from ever being exactly 0.

Key idea: Under the null hypothesis (no group difference), any assignment of labels to observations is equally likely.

Section 3

K-Fold Cross-Validation

Watch how each fold serves as the held-out validation set in turn, building an error estimate from data the model never trained on.

Number of Folds (K)5
210
Data Split: 5 Folds (50 points)
1
2
3
4
5
Current validation foldPending

Folds are assigned round-robin here (point i goes to fold i mod K), so each segment of this strip represents a fold's points, not a contiguous slice of the dataset. You can see the interleaving in the scatter plot colors below.

Data & Fit
0123450510
Fold
Train MSE
Val MSE
1
—
—
2
—
—
3
—
—
4
—
—
5
—
—

Key Insight

Every point is held out exactly once, and the average validation MSE estimates generalization error. Treat the ± std with care, though: the K fold errors are correlated because their training sets overlap, so this spread understates the true uncertainty of the estimate.

Section 4

Bias-Variance Tradeoff & the Validation Curve

Error versus model complexity (a validation curve; a learning curve would instead plot error versus training-set size). Understand when your model is too simple (high bias) or too complex (high variance).

UnderfittingSweet SpotOverfittingOptimalLowMediumHighModel Complexity →ErrorTraining ErrorValidation Error
↘

Underfitting

  • •High bias, low variance
  • •Model too simple
  • •Both train & test error high
  • •Solution: increase complexity
✓

Sweet Spot

  • •Balanced bias and variance
  • •Good generalization
  • •Low test error
  • •Use cross-validation to find it
↗

Overfitting

  • •Low bias, high variance
  • •Model memorizes noise
  • •Low train, high test error
  • •Solution: regularize / simplify

Bias-Variance Decomposition

MSE=Bias²+Variance+Irreducible Error
Bias²

Error from wrong assumptions. A linear model fitting quadratic data has high bias.

Variance

Sensitivity to fluctuations in training data. A high-degree polynomial has high variance.

Irreducible Error

Noise inherent in the data. Cannot be reduced by any model, no matter how good.

← All GuidesNext Guide →