Multi-Agent Pipeline — Homework Guide

1

2

3

4

Task 1 — Extend

Add a 4th Agent to Your Pipeline

Your pipeline has Researcher → Analyst → Editor. Add a 4th agent. Choose one:

Researcher

→

Analyst

→

Editor

→

Your 4th
Agent

Choose your 4th agent

✅

Fact-Checker

Cross-references claims against known data. Place between Analyst and Editor.

📋

Summarizer

Produces a 1-paragraph executive brief. Place after Editor.

🎯

Domain Expert

Adds industry-specific context (e.g., healthcare, finance). Place between Researcher and Analyst.

Starter code

Define your 4th agent’s prompt and wire it into the pipeline.

extend_pipeline.py

# Choose your 4th agent and define its prompt
FOURTH_AGENT_PROMPT = """You are a specialized [ROLE] Agent.
Your job: [DESCRIBE WHAT IT DOES]
Input: [WHAT IT RECEIVES]
Output format:
[DEFINE STRUCTURED OUTPUT]
IMPORTANT: [KEY CONSTRAINT]"""
# Modify run_pipeline() to include the 4th agent
def run_pipeline_v2(query):
    # Stage 1: Researcher
    researcher_output = call_agent(RESEARCHER_PROMPT, query, "Researcher")
    # Stage 2: YOUR 4TH AGENT (if it goes here)
    # fourth_output = call_agent(FOURTH_AGENT_PROMPT, ..., "Your Agent")
    # Stage 3: Analyst
    analyst_output = call_agent(ANALYST_PROMPT, ..., "Analyst")
    # Stage 4: Editor
    editor_output = call_agent(EDITOR_PROMPT, ..., "Editor")
    return {...}
# Test with 3 business questions
questions = [
    "Impact of AI on supply chain management",
    "Remote work trends and productivity data",
    "Sustainable business practices ROI"
]
for q in questions:
    result = run_pipeline_v2(q)
    print(f"Query: {q}")
    print(f"Quality Score: [check editor output]")
    print("---")

→ 3 color-coded sections. Agent prompt (cyan) — define your new agent’s role. Pipeline logic (green) — wire in the 4th stage. The yellow line is where your agent goes. Test harness (magenta) — run on 3 questions.

# Choose your 4th agent and define its prompt
FOURTH_AGENT_PROMPT = """You are a specialized [ROLE] Agent.

Your job: [DESCRIBE WHAT IT DOES]

Input: [WHAT IT RECEIVES]

Output format:
[DEFINE STRUCTURED OUTPUT]

IMPORTANT: [KEY CONSTRAINT]"""

# Modify run_pipeline() to include the 4th agent
def run_pipeline_v2(query):
    # Stage 1: Researcher
    researcher_output = call_agent(RESEARCHER_PROMPT, query, "Researcher")

# Stage 2: YOUR 4TH AGENT (if it goes here)
    # fourth_output = call_agent(FOURTH_AGENT_PROMPT, ..., "Your Agent")

# Stage 3: Analyst
    analyst_output = call_agent(ANALYST_PROMPT, ..., "Analyst")

# Stage 4: Editor
    editor_output = call_agent(EDITOR_PROMPT, ..., "Editor")

return {...}

# Test with 3 business questions
questions = [
    "Impact of AI on supply chain management",
    "Remote work trends and productivity data",
    "Sustainable business practices ROI"
]

for q in questions:
    result = run_pipeline_v2(q)
    print(f"Query: {q}")
    print(f"Quality Score: [check editor output]")
    print("---")

Deliverable

4th agent prompt defined with clear role, input, output, and constraints
Pipeline updated with the 4th agent wired into the correct position
3 business questions run through the extended pipeline
Quality comparison — compare scores with and without the 4th agent

💡 Placement matters

Where you insert the 4th agent changes the pipeline’s behavior. A Fact-Checker between Analyst and Editor catches errors before editing. A Domain Expert between Researcher and Analyst enriches the analysis. Think about information flow.

1

2

3

4

Task 2 — Analyze

Document Information Distortion

Run your pipeline 5 times on the SAME question. Document how outputs differ. This is the “telephone game” in action.

Record your results

Use this table format in your notebook. Fill in each row after each pipeline run.

Run #	Key Facts Found	Quality Score	Unique Claims	Consistent Claims
1	fill in...	fill in...	fill in...	fill in...
2	fill in...	fill in...	fill in...	fill in...
3	fill in...	fill in...	fill in...	fill in...
4	fill in...	fill in...	fill in...	fill in...
5	fill in...	fill in...	fill in...	fill in...

Automation snippet

distortion_test.py

# Run the same question 5 times and collect results
test_question = "Impact of AI on supply chain management"
results = []
for i in range(5):
    print(f"\n{'='*50}")
    print(f"Run {i+1} of 5")
    print(f"{'='*50}")
    result = run_pipeline(test_question)
    results.append(result)
# Compare: What changed between runs?
for i, r in enumerate(results):
    print(f"\nRun {i+1}:")
    print(f"  Editor output: {r['editor'][:200]}...")

Analysis questions

Answer each of these in your submission:

Which facts appeared in ALL 5 runs? These are reliable.
Which facts appeared in only 1–2 runs? These are unstable.
What was the quality score range? (e.g., 7.5–9.0)
What safeguards would improve consistency?

⚠ The point is variation, not perfection

If every run produces identical output, try setting temperature=1 or using a more open-ended question. LLMs are non-deterministic — the question is how different, not whether they differ.

1

2

3

4

Task 3 — Design

Design a Multi-Agent System for Your Industry

Propose a multi-agent system for your target industry (the one you’ll use for your final project). Think like an architect — agents, handoffs, human gates, and cost.

Design Template

1.

Industry

[YOUR INDUSTRY] — the one you’ll use for your final project

2.

Use Case

[SPECIFIC PROBLEM TO SOLVE] — what multi-step process would benefit from agents?

3.

Agent Roles

For each agent: Role, Input, Output, System prompt summary (2–3 sentences)

4.

Handoff Logic

Agent 1 → Agent 2: What data passes? What format?

5.

Human Gates

Where should a human review before proceeding? What decisions are too important for full automation?

6.

Cost Estimate

API calls per run, estimated tokens, cost per run at ~$3/M input, ~$15/M output

Agent design template

design_template.txt

Industry: [YOUR INDUSTRY]
Use Case: [SPECIFIC PROBLEM TO SOLVE]
Agent 1: [ROLE]
  - Input: [What it receives]
  - Output: [What it produces]
  - System prompt summary: [2-3 sentences]
Agent 2: [ROLE]
  - Input: [What it receives from Agent 1]
  - Output: [What it produces]
  - System prompt summary: [2-3 sentences]
Agent 3: [ROLE]
  ...
Handoff Logic:
  - Agent 1 → Agent 2: [What data passes, what format]
  - Agent 2 → Agent 3: [What data passes]
Human Gates:
  - [Where should a human review before proceeding?]
  - [What decisions are too important for full automation?]
Cost Estimate:
  - API calls per run: [NUMBER]
  - Estimated tokens per run: [NUMBER]
  - Cost per run at ~$3/M input, ~$15/M output: $[ESTIMATE]

Example design

☁ CloudSync — Customer Support Pipeline

Industry:

SaaS / Cloud Computing

Use Case:

Automated customer support ticket resolution

Agent 1:

Triage Agent — Classifies incoming tickets by severity and category (billing, technical, account). Routes to the right team.

Agent 2:

Research Agent — Searches knowledge base and past tickets for relevant solutions. Pulls account context.

Agent 3:

Response Agent — Drafts a customer-facing reply using the research. Matches brand voice and tone.

Agent 4:

QA Agent — Checks accuracy, tone, and compliance. Flags responses that need human review.

Human Gate:

All billing-related and account-deletion responses require human approval before sending.

Cost:

4 API calls × ~2K tokens each = ~8K tokens/run ≈ $0.14 per ticket

✅ Any architecture works

The pipeline is the baseline, but feel free to propose a debate pattern (two agents argue), an orchestrator pattern (manager agent delegates), or a hybrid. Just explain why your architecture fits the use case.

1

2

3

4

Task 4 — Read

Claude Code’s Team Feature

Read about Claude Code’s multi-agent “team” feature and the orchestrator pattern. This is how real production systems coordinate multiple AI agents.

Reading Assignment

Read the Claude Code Subagents documentation on Anthropic’s site.
Also review the Building Effective Agents guide — focus on the orchestration section.
Think about how these patterns connect to what you built in Session 8’s lab.

Deliverable

Write one paragraph (5–8 sentences) explaining how Claude Code’s team feature implements the orchestrator pattern from Session 6. Include:

Who is the orchestrator? Which component decides what each agent does?
How are tasks distributed? How does the system break work into sub-tasks?
How do agents communicate? What’s the message-passing mechanism?

📝 Keep it concise

One paragraph is enough. The goal is to show you understand the orchestrator pattern and can recognize it in real products — not to write an essay. Connect what you read to the architectures we discussed in Session 6 (Pipeline, Debate, Orchestrator).

Troubleshooting

FAQ & Common Issues

Click any question to expand the answer.

📝 How long should my 4th agent’s prompt be? ▶

Similar to the others — 10–20 lines. Focus on role, input format, output format, and key constraints. A great agent prompt is specific enough that the LLM knows exactly what to do, but flexible enough to handle different inputs.

🎲 My pipeline gives different results every time — is that a bug? ▶

No! That’s the point of the Telephone Game exercise. LLMs are non-deterministic. The question is: how different? Use temperature=0 for more consistency. The goal of Task 2 is to measure and document this variation, not eliminate it.

💰 How do I estimate API costs? ▶

Count your agents × average tokens per call. With claude-sonnet-4-20250514 at ~$3/M input and ~$15/M output, a 3-agent pipeline with ~2K tokens each costs roughly $0.10 per run. A 4-agent pipeline costs ~$0.14. For your design in Task 3, multiply agents × tokens × expected daily runs.

🔄 Can I use a different architecture (debate, orchestrator) instead of pipeline? ▶

Absolutely! The pipeline is the baseline for Tasks 1 and 2, but for Task 3’s design exercise, feel free to propose any architecture that fits your use case. Debate patterns work great for adversarial checking (e.g., code review), orchestrators for complex routing (e.g., customer support). Just explain why you chose it.

Key Takeaways

Agent teams mirror org charts. Division of labor, specialization, and coordination costs apply to AI agents just like human teams. Design your agent system like you’d design a department.

Watch for the telephone game. Information degrades across agents. Citation chains and confidence propagation are your safeguards against compounding errors.

Multi-agent isn’t always better. Each agent adds latency, cost, and complexity. Use the decision framework: multi-agent for complex, multi-expertise tasks. Single-agent for everything else.

The pipeline pattern is your workhorse. Research → Analysis → Review is the most common multi-agent pattern. Master it and you can adapt it to any domain.

4

agents in pipeline

5

distortion runs

1

industry design

3

architectures learned

Back to Session 8 Open in Colab

Extend Your Multi-Agent Pipeline

Add a 4th Agent to Your Pipeline

Choose your 4th agent

Fact-Checker

Summarizer

Domain Expert

Starter code

Deliverable

Document Information Distortion

Record your results

Automation snippet

Analysis questions

Design a Multi-Agent System for Your Industry

Agent design template

Example design

☁ CloudSync — Customer Support Pipeline

Claude Code’s Team Feature

Reading Assignment

Deliverable

FAQ & Common Issues

Key Takeaways