Measured, not asserted
How accurate is it?
Every figure below comes from running the real pipeline against a dataset whose faults were planted on purpose, so the right answer is known exactly. Nothing here is typed by hand — the page reads the benchmark’s own output, including the parts it fails.
- Dataset
- retail_orders.csv
- Rows
- 6,583
- Planted defects
- 199
- Measured
- 11 August 2026
What these numbers cover
The benchmark runs with no language model configured. Everything below therefore describes ingestion, cleaning, statistics, and dual-engine verification — and none of it describes model interpretation and written narrative. That is the honest division: the arithmetic is where a wrong answer is silent and dangerous, and it is exactly the part that needs no model.
Verification
Each published figure is computed twice — DuckDB reading the canonical Parquet, and Polars computing it independently — and shipped only if the two agree. The number that matters is the last one: anything above zero means a claim reached a reader without being checked.
What it found, and what it missed
The defects were planted, so recall is a fraction rather than a judgement. These are reported as found over planted because the near misses are the interesting part — a cleaner that finds most of the duplicates is a different product from one that finds all of them, and rounding both to “detected” throws away the only distinction that matters.
| Planted defect | Found | Recall |
|---|---|---|
| Duplicate rows | 116 of 118 | 98.3% |
| Missing values in a measure (net_sales) | 47 of 47 | 100% |
Rows, safety and reproducibility
How to check this yourself
The benchmark is a script in the repository that runs the same pipeline a customer file goes through, then writes the figures this page renders. It runs the fixture twice and compares the two runs, because a figure that is stable when watched is not the same as a figure that is stable — if they ever disagree, this page says so rather than picking one.
You can also just read the finished analyses and click any number to see both engine results and the rows behind it.