After producing logits, LLMs use sampling parameters to shape outputs. Adjust them below and watch the probability distribution reshape in real time.
Sign in to save progress
Prompt: "The sky is ___", raw softmax at T=1.0
P(i) = exp(logit_i / T) / Σ exp(logit_j / T): higher T flattens the distribution.
Turquoise = in Top-K pool · Gray = excluded
Only sample from the K highest-probability tokens. Excluded tokens have their mass redistributed.
Probabilities are renormalized across included tokens.
Pick the smallest set of tokens whose cumulative probability ≥ P. Adapts to distribution shape.
Using 4 tokens
Cumulative: 90.6%
Five deterministic samples from the current T + Top-K distribution.
Real-world settings
Major LLM APIs (OpenAI, Anthropic) default to T=1.0 with no top-p cutoff, and what consumer apps like ChatGPT use internally is not published. In practice, developers often dial down to T≈0–0.3 for code and extraction tasks, and up to T≈0.9–1.1 for creative writing.
Up Next
Context Windows: What the Model Can See
Slide context length and test information retrieval at different positions.