You pay per token, but users feel milliseconds. Price a real workload, split its wait into TTFT and generation, race streaming against blocking, and compute exactly when prompt caching pays for itself.
Sign in to save progress
The two meters every LLM product runs on
What you pay
Every request bills tokens twice: the prompt you send in and the answer that comes back. Output tokens usually cost several times more per token, but prompts are often 10 to 50 times longer, and you resend them on every call. The bill is frequently dominated by input.
What users feel
Users never perceive your $/MTok. They perceive two numbers: how long until the first token appears (TTFT) and how fast text flows after that (TPOT, time per output token). Total wait = TTFT + tokens × TPOT.
This guide stays on the product side of the ledger: what a request costs, why it feels slow, and when caching pays. The systems mechanics underneath these numbers (prefill vs decode, batching, KV memory) belong to the LLMs category.
Price a workload
This is a live calculator, not a screenshot: every number below is recomputed from the sliders. Move at least two knobs to price a workload that looks like yours. Prices are editable assumptions, not facts.
Editable price assumptions
Defaults are illustrative placeholders as of July 2026, not any vendor's price list. Type in your provider's real rates and everything on this page recomputes.
Cost of one request
$0.030
Per day
$150.00
5,000 requests
Per month (30 days)
$4,500
150,000 requests
With these numbers the prompt is 80% of every request's bill, even though each output token costs more than each input token. The reason: you resend the whole prompt on every call. That resent share is exactly what prompt caching (last section) attacks.
Decompose the wait
The same request has a latency anatomy: nothing appears until TTFT, then tokens arrive one every TPOT milliseconds. Drag both knobs and watch which segment actually owns the wait.
Output length comes from your workload above: 400 tokens. Total = TTFT + tokens × TPOT.
Total wait
10.5s
TTFT share
5%
Throughput
40 tok/s
TTFT bundles everything before the first token: network, queueing, and prompt prefill. How prefill and decode work inside the model is the LLMs category's story; on the product side you measure both knobs and decide which segment to attack. Long answers are usually generation bound, so trimming output length often beats faster hardware.
Perceived speed: streaming vs blocking
Both responses below take exactly the same total time, computed from your numbers above. The only difference is when the user first sees something.
Streaming
first content at 500ms
Reading starts at TTFT (500ms); the rest of the answer arrives while the user is already reading.
Blocking
first content at 10.5s
Same request, same 10.5s of compute. The user just stares at a spinner until the last token exists.
Both lanes use the exact numbers from your waterfall above. Run the race to see why streaming is the cheapest latency optimization you will ever ship.
Prompt caching: pay once to stop repaying
If a prefix of your prompt is identical across requests (system prompt, tool definitions, shared documents), you can pay a one-time write premium to store it, then a deep discount every time you reread it. The rates are editable multipliers; the savings are computed exactly from your workload above.
The write and read rates are editable multipliers of your input price, defaults illustrative as of July 2026: writing the prefix costs a premium once, rereading it gets a deep discount. Cached prefix here: 6,000 of 8,000 prompt tokens.
Input cost, no cache
$0.024
every request
First request (write)
$0.029
+$0.0045 premium
Later requests (read)
$0.0078
saves 67.5% of the input bill
Find the break-even
The first request pays the write premium, so caching starts out MORE expensive. Move the slider to the smallest number of requests at which the cached total drops below the no-cache total, then lock it in.
no cache $0.030 vs cached $0.035: caching costs $0.0045 more here