A consulting team's research agent is asked the same question five times — "Should we recommend market entry into Vietnam for a mid-sized US fintech?" Each response is slightly different: same overall recommendation (Yes, with conditions), but different supporting points emphasized, different risks foregrounded, different framings of the regulatory landscape. The team is debating whether this is a problem.
What's the right framing?
Why did you pick that answer? Two or three sentences. The act of articulating it is what builds the judgment — not the click that follows.
Not all variance is equal — measure variance on the dimensions that affect decisions. If the recommendation is the same and the key risks identified are the same, prose variance is inert: the user makes the same decision either way. If the recommendation flips between "Yes" and "No" across runs, that's a serious consistency problem. The right consistency test is: same query × N runs × extract decision-relevant fields × measure variance on those fields. Variance on framing is normal; variance on outcomes is a bug.
"Unavoidable, accept it" is what teams say when they haven't measured. You can't reason about whether the variance matters until you separate decision-relevant from framing-only variance. Measure first.
Temperature 0 reduces but does not eliminate variance — multi-turn agentic systems still vary based on tool-result ordering, race conditions in retrieval, and other non-deterministic factors. And temperature 0 produces flatter, less useful outputs for many tasks. The fix is measuring the right kind of variance, not eliminating all variance.
You can't meaningfully "average" three prose recommendations into a "consensus" recommendation. Averaging is for numerical estimates of the same quantity computed by the same method — and even then, averaging across methodologically different sources can be meaningless. Recommendations require picking one, with attribution and reasoning.