A quality check that lives in someone's head protects nothing. Here you turn a data contract into executable rules, run corrupted batches through a three-stage pipeline, and watch exactly what one switched-off rule costs the dashboard at the end.
Sign in to save progress
From quality talk to quality gates
The six dimensions of data quality name what good data means. This guide is about enforcement: each expectation becomes a small executable predicate (type, range, not_null, unique, relationship) that runs against every incoming batch and quarantines whatever fails, the same idea behind tests in tools like dbt or Great Expectations. The difference between a documented standard and a validation rule is that the rule cannot forget to check. It sits inside the pipeline as a gate, and the pipeline does not move data past it.
How this playground stays honest: the three batches (24 rows, 6 of them defective) are fixed fixtures you can read cell by cell in section 1. Every caught and escaped count comes from executing your enabled rules against those rows, and every bar on the dashboard is a live sum over the rows that actually passed your gate. Nothing is faked.
An orders feed lands three times, once per day. The data dictionary says what every column must look like; the batches say what actually arrived. Inspect both. Some cells break the contract, and finding them by eye is the last time anyone should have to.
Data dictionary: the orders contract
order_id : string
Order key, format ORD-1xxx. Must be unique: one row per order.
customer_id : string
Required. Must exist in the customers reference table.
amount : number
Order value in dollars. Valid orders are 5 to 500.
quantity : integer
Items in the order, 1 to 10.
Customers reference table
These 6 ids are the only customers that exist. A relationship rule checks incoming customer_id values against this table.
The raw batches, exactly as they will arrive
| order_id | customer_id | amount | quantity |
|---|---|---|---|
| ORD-1001 | C-103 | 42.5 | 1 |
| ORD-1002 | C-101 | 129.99 | 2 |
| ORD-1003 | C-105 | 88 | 1 |
| ORD-1004 | C-102 | N/A | 1 |
| ORD-1005 | C-104 | 260.75 | 3 |
| ORD-1006 | C-106 | 19.99 | 1 |
| ORD-1007 | C-101 | 250000 | 2 |
| ORD-1008 | C-103 | 310.4 | 4 |
Read each batch against the dictionary on the left. Some cells break the contract: a value with the wrong type, a number outside its range, a missing key, a replayed order, an id no reference table has heard of. Nothing here is highlighted for you, because nothing will be highlighted in production either. The rules you write in section 2 are what does the noticing.
Translate the dictionary into rules. Each one is a real predicate: when the pipeline runs, every enabled rule executes against every row, and one failure is enough to quarantine the row. Rules are deliberately narrow. A range check shrugs at text, a type check shrugs at nulls, so coverage comes from the set, not from any single rule.
Rule builder
The value must match the column's declared type. Nulls pass; missingness is not_null's job.
Your schema: 0 rules, 0 enabled
No rules yet. The gate is wide open: every row, valid or not, will sail straight into the warehouse. Run the pipeline once like this to see what that costs, then come back and write the contract down as rules.
This is how pipeline testing tools like dbt think about quality: each expectation from the data dictionary becomes one small executable test, and the set of tests is the schema. The OFF switch exists because the most common failure in real teams is not a missing rule; it is a rule someone silenced during an incident and never turned back on.
Extract, validate, load: the three batches flow through the gate in order, and whatever survives your rules becomes the revenue dashboard on the right.
Start unguarded: run the pipeline in section 3 before writing a single rule, and look closely at what reaches the dashboard.
Each run rebuilds the warehouse from scratch with your current rule set, like a scheduled pipeline job. Caught means a defective row was quarantined; false alarms means a contract-valid row was quarantined by mistake; escaped means a defective row passed every enabled rule.
Stage 1 · Extract
The three fixture batches from section 1, replayed in order.
Stage 2 · Validation gate
0 rules armed
Nothing in the gate yet. Every enabled rule runs against every row of each batch; one failed rule is enough to quarantine a row.
Stage 3 · Warehouse and dashboard
0 rows in the warehouse
Dashed gold line: revenue from contract-valid rows only, the number this chart should show. Bars turn orange when they drift from it.
Quarantine · 0 rows
Empty. Rows land here when an enabled rule catches them, tagged with the rule that fired.