An LLM is a neural network trained on vast text to predict the next token. Behind that simple idea lies billions of parameters and capabilities that surprised even their creators.
Sign in to save progress
LLMs predict the most likely next token given everything before it. Play the game to see this in action.
Round 1/3: complete the sentence
“The cat sat on the ___”
“Large” is the key word. More parameters enable more capable representations.
Bars scaled logarithmically so smaller models remain visible.
GPT-2
2019
Basic generation
GPT-3
2020
In-context learning
PaLM
2022
Chain-of-thought
Llama 3.1
2024
Complex reasoning
Training happens in three stages. Click each step to reveal it.
These capabilities were never explicitly trained; they appeared as models scaled up, surprising researchers. The tiers below are illustrative: abilities emerge gradually and unevenly, there are no agreed parameter thresholds, and some researchers argue apparent “jumps” are partly an artifact of how benchmarks are scored.
At inference time the model generates one token at a time. Each new token becomes part of the input for the next.
0 / 7 tokens generated
Key Insight
LLMs don't “understand” language the way humans do; they model statistical patterns. Yet these patterns encode surprisingly deep structure about the world.