We compute every figure twice. Here is what happens when the two answers differ.

A language model generating a number produces it the way it produces every other token: by predicting what should come next. That process has no arithmetic in it. It has a very good sense of what a plausible number looks like in context, which is a different thing and, for reporting, a worse one.

The consequence is specific. When such a model is wrong about a figure, it is wrong confidently and unremarkably. There is no hedge in the output, no widened error bar, no change of tone. A revenue total that is 4% low reads exactly like one that is correct. If you had an independent way to notice the 4%, you would not have needed the tool.

Compare this with how the same problem shows up in spreadsheets, where at least the errors have signatures. When two people get different totals from the same file, the cause is nearly always structural, a subtotal row counted as data, a filter left on, an export that ran twice, and each of those leaves a trace you can go and look for. The model failure leaves none. That is what makes it worth engineering against.

Every figure that reaches a report is computed by two independent engines. Independent is doing real work in that sentence: different implementations, not the same routine called twice, because the same routine called twice reproduces the same mistake and reports agreement.

The two results are compared. If they agree, the figure is published. If they disagree, the figure is dropped, not averaged, not flagged for review, not published with a caveat. It does not appear.

Dropping is the unusual choice and it is deliberate. The alternatives all look more helpful and are worse:

• Average them. You now have a number that neither engine computed and no one can defend.
• Pick the first. You have discarded the information that something is wrong and kept the thing that might be broken.
• Publish with a warning. Warnings are read by the person who built the report and by nobody downstream. A caveated number in a deck is an uncaveated number by the second meeting.

A figure that two independent computations disagree about is not a figure with a known uncertainty. It is a figure whose provenance is unknown, which is not a weaker version of a fact.

This is also why a report from the pipeline can be shorter than you expected. A gap is a decision, not an oversight.

Disagreement is rarer than you might guess, because the two engines agree readily about arithmetic. Arithmetic is not where files break. Disagreement clusters where the two implementations had to interpret something, and interpretation is exactly where the risk lives:

Type coercion at the boundary. A column that is mostly numeric with a scattering of text entries. One implementation coerces and skips, the other coerces and fails. The disagreement is the useful output, it tells you the column is not what it claims to be.

Date parsing under ambiguity. 03/04/2026 is two different dates and both are defensible. Two engines resolving it differently is a signal that the column needs a stated convention rather than a guess.

Null versus zero. Whether a blank cell is excluded from a mean or counted as zero changes the answer materially, and the two conventions are both common. This is the same disagreement that makes two dashboards built from the same source show different numbers.

Boundary conditions on ranges. Whether the last day of a period is inside or outside it. Small, silent, and the reason a monthly figure can be wrong by exactly one day's trade.

None of these is an arithmetic bug. All of them are places where a single implementation would have produced one confident answer and never mentioned that another confident answer was available.

We publish the benchmark rather than describing it, and it is regenerated by running the actual pipeline against a file whose defects were planted, so "did it find them" has an exact answer. Nobody types a number onto that page; the build fails if anyone tries. The current run, measured 16 August 2026 on retail_orders.csv:

• 6,583 of 6,583 rows read. The row count is reported as read-versus-expected rather than as a total, because silent truncation is the error that invalidates everything after it.
• 199 planted defects.
• 6 of 6 engine checks agreed. 6 findings published, 0 published without agreement.
• Duplicate rows: 116 of 118 found. That is 98.3% recall. It misses two.
• Missing values in `net_sales`: 47 of 47.
• 13 of 13 prompt-injection attempts blocked, and 1 of 1 sensitive column withheld from the models entirely. More on that under security.
• Two independent runs produced identical results.

The duplicate figure is the one worth dwelling on. It is a miss, it is published with the same weight as the passes, and it stays on the page. A benchmark that only reports what it passes is an advertisement with a chart on it. The first question any competent sceptic asks is what was left out, and a page that has already answered it is the only kind worth reading.

There is a scope limit on all of this that we state rather than bury. The benchmark runs with no language model configured. It covers ingestion, cleaning, statistics and verification. It excludes model interpretation and narrative generation, the layer that turns verified figures into sentences. We measure the layer that can be measured exactly, and we do not extend the claim to the layer that cannot.

Two engines agreeing is a strong statement about computation and a weak one about meaning. Both can be right about the arithmetic and wrong about the question.

If your export dropped a region before the file ever reached us, both engines will faithfully compute the wrong total. If a column changed meaning in March, the same header now recording something else, both will happily aggregate across the change. If the metric definition is contested inside your own business, agreement between two implementations of one definition tells you nothing about whether it is the definition anyone wanted.

Verification narrows the problem to one thing: whether the number in front of you is the number the data supports. That is a smaller claim than "this analysis is correct", and it is the claim we can actually stand behind. Deciding whether a verified figure is a real finding is still a judgement, and it is still yours.

Practically nobody in this category publishes an accuracy figure at all. That is not an accident of modesty. Publishing one commits you to a number that can go down, to a page that reports your misses next to your hits, and to a benchmark that some competitor will eventually run better than you.

It is worth it anyway, because the alternative is asking people to upload their financial records on the strength of an adjective. Every tool in this space says it is accurate. The word costs nothing and carries none. A fraction, 116 of 118, can be checked, argued with, and beaten. We would rather compete on the version that can be checked.

The current run is here, and it changes when the pipeline changes. If you want to see what it does to one of your own files, you can upload one and read the headline finding before there is anything to pay.