05
Domain 05
Context Management & Reliability
Lost-in-the-middle, structured errors, honoring explicit requests, stratifying before automating.
15 questions
- Q21What's the best architectural fix?A 12-turn customer case where critical numbers from turn 1 have been compressed away by turn 12.
- Q22Which response is most useful?A subagent times out — what should it return to the coordinator?
- Q23Should the agent surface the resolution first, or escalate immediately?A frustrated customer asks for a human, mid-conversation — and the agent has the resolution one click away.
- Q24What should the synthesis subagent do?Two credible sources cite different market growth numbers — what does the synthesis agent do?
- Q25What's your most defensible position?97% accuracy on high-confidence cases. The VP wants to cut human review next month.
- Q46What's the best fix?An order-lookup tool returns 40 fields when the agent uses 5 — and the context is suffering.
- Q47What's the right behavior?Three different John Smiths matched the lookup. The agent picks the most recent one and proceeds.
- Q48What's the right behavior?A frustrated customer hasn't asked for a human. The agent has the resolution one click away.
- Q49What's the right pattern?Two robust subareas, three thinly-sourced ones — all presented in identical confident prose.
- Q50What's missing?A 2024 report says 12% growth, a 2025 study says 8%. The synthesis says "between 8% and 12%."
- Q71Which produces the most useful summary?You've decided to use a discovery subagent. Now: what do you ask it to return?
- Q72What's the right structural fix?A 3,000-token ticket history with the most decision-critical fact buried in turn 4 — and the agent treats this as a routine first-time issue.
- Q73What's the right escalation pattern?A customer requests a competitor price match. Your written policy is silent on the case.
- Q74What's the right measurement approach?50,000 invoices monthly, 90% high-confidence. The team can't review 45,000 manually — but the error rate has to be measured.
- Q75What's the architectural fix?A research synthesis combines a 2018 figure with a 2024 figure without surfacing dates, and reviewers are flagging it as misleading.