Week 4 · Build, debug, iterate

Independent project development

You've got the parts. Today it's a hackathon: you build your own agent for real. But real agents break — so first you'll learn to debug them (spot the three failure patterns fast), then build in structured phases with check-ins so nobody stays stuck.

Week 4 of 5 · theory 20 min · practical 100 min

What you'll walk away with

The three ways agents fail — hallucinations, tool errors, prompt issues — and how to catch each one.
Two habits that keep agents honest: validation (check outputs, keep a human in the loop) and logging (record every Thought → Action → Observation).
A working MVP with at least two tools, built in timed phases with class check-ins.
A reusable agent template — tool registry, logging, and try/except error handling.

Part 1 · Theory (20 min)

Debugging agentic systems

An agent has more moving parts than a chatbot — the model, the tools, and the prompts wiring them together — so there are more places for things to go wrong. The good news: almost every bug falls into one of three patterns. Learn to name them and you'll fix them fast.

🌀 Hallucinations

The agent invents a fact or an answer instead of using a tool to find out. Sounds confident, but it's made up.

🔧 Tool errors

A tool crashes, returns junk, or gets called with the wrong arguments — e.g. a date passed where a number belongs.

💬 Prompt issues

Vague instructions → wrong behavior. "Summarize this" with no length or focus gives a rambling, off-target answer.

✓

The move

When something's off, ask: did it make it up, did a tool break, or was the prompt unclear? That names your fix.

⚠️ The most dangerous bug: a confident hallucination

A crash is obvious — you see the red error. A hallucination is sneaky: the agent answers smoothly and looks right, so it slips past you. That's exactly why the next two habits matter — they force the agent to prove its answer instead of just asserting it.

Validation — don't trust, verify

Before you believe an agent's answer, check it. Three cheap habits catch most problems:

Check tool outputs — did the tool actually return what you expected, or an empty list / error string?
Sanity-test the result — "average age = 320" is impossible; a real number would be ~30. Add quick guard checks.
Keep a human in the loop — for anything important, a person reviews before the answer is trusted or acted on.

Logging — make the invisible visible

An agent loops silently inside your code. If you can't see what it did, you can't fix it. So log every step: print or record each Thought → Action → Observation. Then when the answer is wrong, you scroll back and spot the exact step where it went off the rails.

a logged trace

# with logging, the failure is obvious:
Thought   → "I need the average age from the dataset."
Action   → mean_age("Age")
Observation → 320.0        # 👈 impossible — a bug lives here
Thought   → "That looks wrong. Re-check the column."

why it mattersWithout the log you'd only see the final wrong answer. With it, you can point at the exact Observation that broke — that's the whole game in debugging.

🔑 The one idea

Every agent bug is a hallucination, a tool error, or a prompt issue. Log the loop so you can see it, validate the output so you can trust it.

Part 2 · Practical (100 min)

The structured hackathon

Now you build your own agent. It's a hackathon, but a structured one — timed phases with class check-ins so momentum never dies and nobody stays stuck alone. Start from the reusable scaffold in the Week 4 notebook (agent template with a tool registry, logging, and try/except error handling built in).

Phase 1 — Foundation (30 min)

Get the smallest thing working end to end: the core agent loop, your first working tool, and your primary data source. Don't polish — just get one full Thought → Action → Observation cycle to run and log.

Check-in & pattern recognition (15 min)

Pause the build. 2–3 students share their screen and their logs. As a class, spot and solve one common integration challenge (loading data, calling a tool, reading a key) — whatever most teams are hitting. One fix, everyone benefits.

Phase 2 — Expansion (40 min)

Now make it real. Add a second tool or data source, wire in basic error handling (so one bad tool call doesn't kill the run), and get the full end-to-end workflow answering your project's question.

robust_tool_call.py

def run_tool(name, args):
    log(f"Action → {name}({args})")     # log every call
    try:
        result = TOOLS[name](**args)      # tool registry lookup
    except Exception as e:
        result = f"TOOL_ERROR: {e}"       # don't crash the loop
    log(f"Observation → {result}")
    return result

the patternWrap each tool call in try/except and log both sides. A broken tool becomes an Observation the agent can react to — not a crash that ends the run.

Debugging clinic (15 min)

Everyone submits their top blocker. The class votes on 2, and we solve them together on the big screen — reading logs, naming the failure type, and fixing it live. You'll leave with a template for how to debug the next one yourself.

Checkpoint

Your agent confidently reports "the average passenger age was 320." Nothing crashed. What kind of bug is this, and what catches it?

A tool error — the code threw an exception. A bad result that slips through — a sanity-check on the output catches it. Nothing's wrong; trust the agent's answer.

No exception fired, so try/except won't help here — the tool "succeeded" but returned nonsense (or the agent hallucinated it). This is why validation matters: a quick sanity check ("is an age of 320 even possible?") plus a logged trace lets you catch and locate an impossible answer that looks confident.

⚡ Energizer · Bug Hunt

Right before the build, spend 12–15 minutes hunting bugs — it primes everyone to spot the exact failures they're about to hit in their own code.

Run it · 12–15 min · teams of 3–4

Find it, classify it, fix it

Each team gets a short buggy agent transcript with planted problems: a hallucinated fact, a tool called with a wrong argument, a vague prompt, and an unhandled error. Teams race to find each bug, classify it (hallucination / tool error / prompt issue), and propose a one-line fix. Round 1 is an easy transcript; Round 2 is trickier. The debrief maps each bug type to a fix habit — verify, validate tool inputs, tighten the prompt, add try/except + logging.

Coach: open the printable card → Bug Hunt (green energizer sheet, ready to print).

Homework & deliverables

MVP — a working agent with at least two working tools and an end-to-end workflow that answers your project's core question. Bring your logs.
Differentiated support — advanced? stretch toward a multi-agent setup or a simple UI. Stuck? use the pre-built tool templates in the notebook and pair-program with a teammate.
Come to Week 5 ready — a running MVP is the raw material you'll polish and present next week.

Recap. Agents fail in three ways — hallucinations, tool errors, prompt issues. You beat them by logging the loop (so you can see it) and validating outputs (so you can trust it). Then you ran a structured hackathon — Foundation, check-in, Expansion, debugging clinic — to build an MVP with at least two tools. Next week: polish it and present.

← Back Week 3 · Data analysis agents & scoping Next → Week 5 · Polishing & presentation