Session 7 — Failure Modes

When RAG Goes Wrong

Three scenarios every knowledge agent must handle

The Setup

"What is CloudSync's return policy?"

Retrieved Documents

refund-policy-2024.md

Current

Updated: November 15, 2024

CloudSync offers a 14-day return window. Customers receive store credit for returns within 14 days of purchase. Original payment refunds require manager approval. All returns require receipt or order confirmation.

OUTDATED

returns-policy-2022.md

Outdated

Updated: March 20, 2022

CloudSync guarantees a full refund within 30 days of purchase. No questions asked. Return items in original packaging for a complete refund to original payment method.

The Problem

Without date-awareness

With date-awareness

Agent Response

"CloudSync offers a generous 30-day full refund policy. Simply return items in original packaging for a complete refund to your original payment method. No questions asked!"

⚠️ WRONG — Used outdated 2022 policy

This Already Happened

In 2024, Air Canada was held legally liable when its customer service chatbot quoted an outdated bereavement fare policy. The airline had to honor the incorrect policy. Cost: $812 per affected customer.

Air Canada v. Moffatt, 2024

What went wrong?

The agent treated all documents equally regardless of dates. In production, this means outdated policies get quoted as current, creating legal and business risk.

How to fix it

Add date metadata and prioritize recent documents during retrieval.

 Python
# Add date metadata to documents
def search_with_date_priority(query, documents):
    results = search(query, documents)
    # Sort by date — newest first
    results.sort(key=lambda d: d['date'], reverse=True)
    # Warn about outdated docs
    for r in results:
        if is_older_than(r['date'], months=12):
            r['warning'] = 'OUTDATED - verify current policy'
    return results

The Setup

"What is the maximum discount we can offer enterprise clients?"

Retrieved Documents

sales-procedures-q4.md

SOP-A

Enterprise discounts are capped at 25% off list price. Exceptions require VP approval for deals over $100K. Standard enterprise discount is 15-20%.

enterprise-pricing-guide.md

SOP-B

For enterprise accounts, sales reps can offer up to 40% discount on annual contracts. Volume discounts of 10-30% are automatically applied for 100+ seats.

The Problem

Without conflict detection

With conflict detection

Agent Response

"Enterprise clients can receive discounts of up to 40% on annual contracts, with standard discounts ranging from 15-25%. Volume discounts of 10-30% are automatically applied for large orders."

⚠️ BLENDED — Mixed contradicting policies into one answer

What went wrong?

The agent blended information from contradicting sources instead of flagging the conflict. Users receive confidently wrong information.

How to fix it

Use the LLM itself to detect contradictions before generating an answer.

 Python
# Detect contradictions in retrieved documents
def check_for_conflicts(results, query):
    if len(results) > 1:
        response = client.messages.create(
            model="claude-sonnet-4-20250514",
            messages=[{
                "role": "user",
                "content": f"Do these documents contradict each other about '{query}'?\n\n" +
                           "\n---\n".join([r['text'] for r in results])
            }]
        )
        if "contradict" in response.content[0].text.lower():
            return "CONFLICT DETECTED - escalate to human"
    return None

The Setup

"What are our competitor's pricing plans?"

Retrieved Documents

Company Knowledge Base

Contains: refund-policy.md, sales-procedures.md, product-catalog.md, employee-handbook.md, customer-faq.md

No competitor data in the knowledge base

The Problem

Without guardrails

With guardrails

Agent Response

"Based on market analysis, your main competitors typically price their products 10-15% below your list price. CompetitorA offers a basic plan at $29/month, Pro at $79/month, and Enterprise at $199/month. CompetitorB uses a per-seat model starting at $12/user/month."

⚠️ HALLUCINATED — None of this data exists in the knowledge base

Agent Confidence 92%

Inappropriately high confidence for fabricated data

What went wrong?

Without retrieval guardrails, the agent generates plausible-sounding information from its training data, not from your documents. This is the classic hallucination problem.

How to fix it

Use relevance scores to decide whether the agent should answer at all.

 Python
# Confidence-based guardrails
def answer_with_guardrails(query, search_results):
    if not search_results or max_relevance_score(search_results) < 0.3:
        return {
            "answer": "I don't have information about that in our knowledge base.",
            "confidence": "low",
            "action": "Suggest alternative sources"
        }
    # Only answer from retrieved documents
    return generate_answer(query, search_results,
                          instruction="ONLY use the provided documents. "
                                      "If the answer isn't in the documents, say so.")