Send every query to the cheapest model that can handle it, escalating only when the model is not confident. Route a simulated query stream through a three-tier cascade, drag the confidence threshold to trace the cost-quality frontier, then break the whole thing with miscalibration and watch quality fall while the dashboard stays green.
Sign in to save progress
The cascade: three tiers, one gate
Every query starts at the Small tier. The tier answers and reports a confidence; if that confidence clears the escalation threshold, the answer ships. If not, the query escalates and the next tier is also paid for. The final tier always answers. Below, a seeded stream of 400 queries (60% easy, 25% medium, 15% hard) is routed live with your current settings: every count is computed from the stream.
Small
1.0 units/query219 / 400 answered (55%)
85.8% of its answers correct
400 queries reached this tier
Medium
6.0 units/query94 / 400 answered (24%)
88.3% of its answers correct
181 queries reached this tier
Large
30.0 units/query87 / 400 answered (22%)
74.7% of its answers correct
87 queries reached this tier (always answers)
first 10 queries of the stream, routed live
400 total| # | difficulty | answered by | confidence | outcome |
|---|---|---|---|---|
| 1 | 0.35 easy | Small | 0.75 | ✗ wrong |
| 2 | 0.64 medium | Medium | 0.70 | ✗ wrong |
| 3 | 0.52 medium | Medium | 0.85 | ✗ wrong |
| 4 | 0.10 easy | Small | 0.96 | ✓ correct |
| 5 | 0.81 hard | Large | final tier | ✗ wrong |
| 6 | 0.80 hard | Large | final tier | ✓ correct |
| 7 | 0.27 easy | Small | 0.86 | ✓ correct |
| 8 | 0.75 hard | Large | final tier | ✓ correct |
| 9 | 0.13 easy | Small | 0.95 | ✓ correct |
| 10 | 0.47 medium | Medium | 0.90 | ✓ correct |
The tiers
Each tier has a cost per query (relative units) and a capability midpoint: the difficulty at which it is 50% likely to be correct. The defaults are illustrative, chosen to echo the roughly 5x to 30x price spread between small and frontier models; drag them to match your own stack.
Small
Medium
Large
Always-big baseline
30.0 units, 93.3%
Always-small baseline
1.0 units, 61.5%
The cost-quality frontier
Every gray point is the cascade evaluated on the same stream at one threshold value; the pink point is your current threshold. Drag the slider and trace the frontier. The shaded corner is the win region: cheaper than always-big at near-equal quality. Push the threshold to 1.0 and note the cascade costs MORE than always-big: you pay the lower tiers for answers you never use.
Accept a tier's answer only when its confidence is at least this value; otherwise escalate.
Shaded corner: the win region (cheaper than always-big, accuracy within 1 point). Pink dot: your current threshold, computed at 10.24 units per query and 84.0% accuracy.
Quality degraded
10.24 units per query at 84.0% accuracy, versus always-big at 30.0 units and 93.3%. Cheap, but accuracy is 9.3 points below always-big. Raise the threshold.
Break it: miscalibrated confidence
The router never sees correctness, only the model's self-reported confidence. In this simulation, confidence = true probability of being correct plus an overconfidence term that grows as calibration decays, and grows most on exactly the queries the model is weakest at. Miscalibration never blocks answers the model was truly sure of; it only lets bad ones through. Drag calibration down and compare the two panels: this is a model swap, a quantized deployment, or a domain shift that nobody re-measured.
1.00 = confidence equals the true probability of being correct. 0.00 = confidence is inflated toward certainty on everything.
What your dashboard sees
Cost down, confidence high. Every number here looks like good news.
Ground truth (invisible without evals)
The gap between reported confidence and this column is the calibration error. No alert fires, because nothing the router measures has changed for the worse.
Mini challenge
The Question
Two weeks after launch, your router dashboard reads: cost per query down, average accepted-answer confidence at an all-time high, zero alerts.
What do you actually know about answer quality?
Pick an answer to get feedback computed from the simulation.