Your code review agent flags issues in pull requests. The team is increasingly frustrated because it produces false positives at a high rate — flagging stylistic patterns that match the team's actual conventions, complaining about defensive null checks the team explicitly wants, and sometimes contradicting itself across files.
The current prompt says: "Review this pull request and report any issues. Be conservative and only report high-confidence findings."
What's the most effective revision?
Why did you pick that answer? Two or three sentences. The act of articulating it is what builds the judgment — not the click that follows.
"Be conservative" and "high confidence" are vague instructions that don't define what counts as a valid finding. The model's calibration of "confidence" doesn't necessarily match yours. Replacing vague guidance with explicit categorical criteria — what to report and what to skip — gives the model the actual decision boundary you want. This is one of the most common and high-leverage prompt engineering moves: replace adjectives with categories.
Adding emphasis ("EXTREMELY") to vague instructions doesn't make them less vague. The model still doesn't know which categories you care about. This is a common mistake — escalating tone to compensate for missing structure.
Model self-rated confidence is poorly calibrated, especially for false positives — by definition the model doesn't know it's wrong, or it wouldn't be reporting the finding in the first place. Filtering by self-reported confidence often filters out correct findings while leaving false positives untouched.
Temperature affects randomness in token selection, not which findings the model considers valid. Lowering temperature might make the same wrong findings appear more consistently, which is the opposite of what you want.