An operations agent at a logistics company has tool access to update_route, notify_carrier, reorder_inventory, and escalate_to_human. During a hurricane that disrupted standard routes, the agent autonomously rerouted a critical medical-supply shipment through a region that turned out to be flooded — the shipment was destroyed, and a hospital missed time-sensitive supplies. The agent's confidence on the rerouting decision was 0.91 (above the team's usual autonomous threshold).
What's the architectural fix?
Why did you pick that answer? Two or three sentences. The act of articulating it is what builds the judgment — not the click that follows.
Physical-world stakes are a different category from financial stakes. A wrong refund is recoverable; destroyed medical supplies and missed surgeries are not. The Automation Matrix axes (volume, judgment, stakes, reversibility) implicitly assume those stakes are commercial. When stakes include physical safety, the right gate isn't a confidence threshold — it's a categorical rule that human review is required, full stop. This is the lesson of operations agent design: confidence-based routing is appropriate for low-stakes operational decisions; categorical human review is appropriate for safety-critical ones.
Training on disaster data doesn't fix the structural issue (autonomous decisions with physical-safety implications). And during a unique disaster, no amount of historical data captures the specific conditions on the ground.
Approval-on-everything forfeits the agent's value on the routine routing decisions where it adds real efficiency. Targeted gates on safety-critical categories preserve the value where it's safe to capture.
"Any system would have struggled" rationalizes away the design flaw. The system was operating exactly as designed, autonomously, on a high-stakes decision. The flaw is in the design, not in the disaster.