Han's Generative AI Quest

Agentic Workflows [1]: Honeyguide: who maintains the understanding when agents write the code?

Coding agents make code easier to write and change, but what an agent works out about a repository usually stays in one private conversation. Honeyguide keeps that understanding as a checkable map meant to be shared: every claim cites its source lines, a separate Checker reads those lines against the claim, and a refresh keeps the map current as the code changes. On the repo I tested, the map didn't measurably help an agent answer more accurately. This post shows how the checking and the upkeep work and what they cost. The shared team workflow isn't built yet, and whether it saves engineering time is open.

Pepper & Carrot AI-powered flipbook · Part 20 — Rebuilding the Red-Teamer on LangGraph Deep Agents: Same Rules, a Batteries-Included Harness

Part 20 of the Pepper & Carrot AI flipbook series. Post 19 built an AI agent that attacks my own reading app, hand-rolled from scratch. This post rebuilds it on LangChain's Deep Agents, which hands you planning, subagents, and human-in-the-loop for free. But the project lives by one rule: the attacker never grades its own attack. A framework makes that rule easy to break, since the obvious move is to give the model a tool for everything — including the judge. The fix takes one line: compute the verdict inside the tool the agent calls, so it can read a pass/fail but never produce one. Then we make the attacker much smarter, without letting it become its own scorekeeper.

Pepper & Carrot AI-powered flipbook · Part 19 — Agentic Red-Teaming: How an AI Agent Hunts Prompt Injection, Hallucination, and Spoiler Leaks

Part 19 of the Pepper & Carrot AI flipbook series, and the discovery half of evaluation. Post 18 built a deterministic evaluator that grades the reading companion against a frozen test set; its blind spot is that it can only catch failures someone already wrote a test for. This post builds the complement: an agentic red-teamer, an AI agent handed the same two MCP tools and a mission ("make it spoil," "get it to invent lore," "talk it out of its rules") that decides its own attacks, adapts across a multi-turn conversation, and reports what broke. It's written for someone brand-new to agentic workflows: every term (agent, tool call, oracle, prompt injection, red-teaming) is defined from zero. The throughline is one rule it inherits, explore agentically and judge structurally: the agent decides what to try, but a separate checkable oracle, never the attacker model, decides whether it won. Every confirmed failure is written back as candidate gold for the deterministic harness. Find once, guard forever.