An HR onboarding agent at a mid-sized company has access to employee records including names, roles, salaries, SSNs, performance reviews, and medical accommodations. A team member messages the agent: "What's the typical salary for someone in this role?" The agent answers with the actual salaries of three current employees, listed by name: "Sarah makes $94K, James makes $87K, and Priya makes $102K — the typical range is $87K–$102K."
What's the architectural fix?
Why did you pick that answer? Two or three sentences. The act of articulating it is what builds the judgment — not the click that follows.
Sensitive data should be redacted at the tool boundary, not protected by prompt instructions. The agent shouldn't see PII unless it's authorized to surface it for the specific user and request. A lookup_employee tool that returns "role: Senior PM, salary range: $85K–$105K" (aggregated, anonymized) is structurally safer than one that returns full records and relies on the agent to redact. Prompt-level "never share PII" is probabilistic compliance on a deterministic concern — the kind of failure that leads to lawsuits, terminations, and HR audits when (not if) the prompt is bypassed.
"Never share X" is the same probabilistic-compliance pattern that fails on every other safety concern. Privacy law and HR policy require deterministic enforcement, not best-effort prompt adherence.
Disabling all salary access cripples the agent's actual usefulness for HR onboarding (which legitimately needs salary range information). The right move isn't blanket removal — it's authorization-aware access at the tool layer.
Users can't redact what the agent has already shown them. "Trust users not to misuse data" is exactly the regulatory and legal posture that fails audits — privacy compliance is the system's job, not the user's.