A finance team is evaluating Claude for two tasks. Task 1: Summarize 10-page quarterly earnings reports into one-paragraph briefs. Claude does this beautifully — coherent, accurate, captures the right nuances. Task 2: Compute the weighted average of 12 financial figures from the same report. Claude consistently gets arithmetic wrong — sometimes off by orders of magnitude, sometimes plausible-but-wrong, almost never reliably correct. The team is confused: how can the same model handle complex prose flawlessly but fail at arithmetic that any pocket calculator gets right?
What's the right framing?
Why did you pick that answer? Two or three sentences. The act of articulating it is what builds the judgment — not the click that follows.
Jagged intelligence is the most important LLM behavior to internalize for production AI work. LLMs are spiky: extraordinary at some things, surprisingly bad at others, and the boundary doesn't correspond to human intuitions about difficulty. Multi-step arithmetic is a textbook weak spot — not because the model can't "do math," but because token-by-token generation isn't the right computation for it. The architectural answer isn't to find a smarter model; it's to route arithmetic to a calculator tool and let the model do what it's actually good at.
This is the genuinely tempting wrong answer because reasoning models do narrow the arithmetic gap somewhat. But they don't eliminate jagged intelligence — they shift the boundary, not erase it. They're also slower and more expensive on the prose tasks the team is already getting right, which means you're paying more for worse latency to fix one weak spot. The tool-use approach (route arithmetic to a calculator, leave prose to the model) is structurally simpler and more reliable than buying a stronger model for the weakness.
The reports aren't ambiguous — the same documents that produce excellent prose summaries also produce wrong arithmetic. Cleaning the source doesn't address the underlying capability gap.
Abandoning the use case forfeits the value of the prose summarization, which is genuinely excellent. The right response to jagged intelligence is targeted tool use for the weak areas, not blanket avoidance.