Your invoice extraction system has a 7% error rate. The CFO has proposed adding an independent verification pass — a second Claude instance reads the source document and the extraction output, with no prior commitment to the original's reasoning, and flags any discrepancies. Verification doubles per-invoice cost. At 50,000 invoices/month and ~$0.50 per extraction, that's ~$25,000/month additional spend. Independent verification absolutely would catch most errors. The question is whether to verify all 50,000.
What's the most defensible approach?
Why did you pick that answer? Two or three sentences. The act of articulating it is what builds the judgment — not the click that follows.
Verification budget should be spent where errors actually live, not uniformly. A 7% aggregate error rate almost certainly isn't uniform — small invoices on standard PDF templates may be 99% accurate, while photo receipts from one-off vendors may be 65% accurate. Verifying everything spends $25K/month; stratifying lets you spend a fraction of that to learn the distribution, then concentrate verification on the strata that actually need it. Stratification before action is a recurring lesson: it applies to deciding which cases to automate, which to measure carefully, and — here — which to spend verification budget on.
"Financial data justifies the cost" sounds responsible but treats verification as an absolute requirement rather than a cost-benefit decision. If 80% of the volume is on documents with a 1% error rate, you're spending $20K/month verifying invoices that didn't need it — and that money was better spent fixing the strata that did.
Improving the extraction prompt is the right long-term move — but "improve without measuring" leaves you optimizing blind. Without stratified data on where errors concentrate, prompt improvements are guided by guesses. Verification on a stratified sample produces the data that tells you which improvements matter.
The same extraction instance retains the same biases that produced the original errors. Self-review systematically validates its own decisions — that's why independent review (a fresh instance) beats self-review structurally, regardless of cost. Saving 50% of the cost by paying that bias is a bad trade.