Lab · case study · 7 min read

Agent Review

When an agent finds a problem, the right fix isn’t always obvious. Agent Review shows the trade-offs, brings a person in for the judgment, and keeps the reasoning for the next run.

  • What I built: Agent Review, a prototype for knowing when a coding agent should proceed, ask a person to decide, or wait for approval, without anyone reviewing every action.
  • What it plays: two simulated runs. In one the agent finds a problem with two defensible fixes and can’t tell which trade-off people can live with; in the other, it needs permission.
  • Where it came from: an experiment I ran first, giving Claude Code the same UI job twice, with written design rules and without.

Try the prototype

The prototype’s decision card, in its dark theme. The agent’s read: Recommended, separate the meanings, because the same red stands for a failure and a replaced value; and What I can’t determine, whether people who use the billing activity screen rely on the red to notice a plan change during time-sensitive work, which could change the recommendation, with what it inferred rather than checked and a Request evidence button. Under it, “Which trade-off is acceptable?” and two cards of the same shape: Separate the meanings, recommended, where failures stay red and the replaced plan turns neutral with a “Replaced” label; and Keep the red for now, where both stay red and the plan gains the same label. Each shows how it would look and lists three benefits and three risks, with its own Choose button.
One red means “payment failed” on one screen and “plan replaced” on another. The agent proposes two fixes, recommends one, and says what it can’t know.
Role
Product strategy and design · Interaction design · Design-system architecture · Accessibility · Experimentation · Implementation
Built with
React, TypeScript, Storybook, Vitest with axe, Playwright, Claude Code
Size
14 components · 258 stories, each a tested state · 12 design rules for the agent · 3 versions of one feature
Status
An interactive prototype playing two simulated runs, informed by a real coding-agent experiment. The prototype, the Storybook, and the source and experiment record

A solo project, begun October 1, 2026 and still in progress: every commit in the public repository is from my account, co-authored by Claude Code.

Feature walkthrough

Shown below: Agent Review’s report on one design job handed to an AI coding agent.

Agent Review in its default dark theme, in a simulated run: “Bring the billing screens onto the token layer” with Dana’s request open, the results reading “7 changes made and checked. 2 decisions need you.”, and the first decision, “Separate the two meanings of red, or keep the familiar signal?”: a replaced plan, the old plan struck through in red with an arrow to the new one, hard-coded from the old billing app, beside “Payment failed” in the same red; the checks folded to one line; the agent’s read, recommending it separate the meanings and saying it can’t determine whether people rely on the red to notice a plan change; and two directions side by side, separate the meanings or keep the red for now, each with how it would look and three benefits and three risks. Under it, a second decision, the run’s checks and the completed work.
The same screen with keep the red for now chosen, its card marked and its button pressed, the recommended card beside it: under them, what the agent will do if it is applied, in five steps; Dana’s reason, in her words, that billing support scans the feed for red during renewal week; and “Use this decision for similar cases” checked, with the rule as it will read and the three waiting changes it settles now.
The same run after the decision: the notice says it was decided, applied and kept as a rule that settled three waiting changes; one close case still asks; and the decision record: what the agent did, from keeping the red and adding the label to writing the exception down and opening a review, then Dana’s reason in her words, the trade-off accepted, the risk that remains, the open follow-up, and the rule it became.
Skip the walkthrough

One job, from delegation to decision

The agent found the problem. Choosing the fix took a person.

The checks can’t choose between the two fixes. What decides it is how people use the screen, and that isn’t in the code.

The checks can’t choose between the two fixes. What decides it is how people use the screen. The simulated run, in five lines:

  • The job: move 36 hard-coded values in Relay’s billing screens onto design tokens. Seven changes needed no one.
  • One red, two meanings: a replaced plan borrowed the red of “Payment failed”.
  • Two defensible fixes: separate the meanings, so the plan turns neutral with a label, or keep the red for now, with a label and a written exception.
  • What it can’t know: whether people rely on the red to spot a plan change. It recommends separating, and says so.
  • The decision: Dana keeps the red, for a reason only she has. The record keeps it, and three more changes follow it.

A second run, below, stops for approval instead.

  1. 1The job

    Move 36 values onto the design system. Change nothing anyone can see.

    Dana, a product designer, asks Claude Code to swap Relay’s hard-coded colors and sizes for design tokens. Seven changes needed no one. Relay is made up, and the run is simulated.

  2. 2The problem

    One red, two meanings.

    The old plan is red because it was replaced; “Payment failed” is red because something went wrong. The plan’s red is a copy of the failure’s, from the old billing app.

  3. 3What it can’t know

    It recommends a fix, and says what could change its mind.

    Separating the meanings is cleaner. Whether people rely on the red to spot a plan change isn’t in the code, so the agent asks rather than guesses.

  4. 4The trade-off

    Two defensible fixes, weighed the same way.

    Each lists three benefits and three risks, and the checks can’t choose between them. Neither is the wrong answer.

  5. 5The decision

    The person picks the trade-off, and says why.

    Dana keeps the red, against the recommendation, with context the agent couldn’t have: in this simulated run, billing support scans the feed for red at renewal time. The plan shows what the agent will do.

  6. 6The work

    Either choice is real work.

    The agent adds the label, writes the exception down and opens a review task. Kept as a rule, the decision settles three of the four waiting changes; the fourth is close but different, so it asks.

  7. 7The record

    The reasoning stays with the work.

    The record keeps Dana’s reason, the trade-off she accepted, the risk left and the follow-up. Each change that follows it names it. A second run, below, stops for approval instead.

The two reds are just one example

The same problem appears whenever an agent can build something correctly but can’t determine whether it’s the right decision. Agent Review tells three kinds of work apart:

Proceeds on its own
The agent handles it.
Clear instructions, valid permissions and passing checks: it proceeds and records what happened.
In the prototype The walkthrough’s seven merged changes.
Needs judgment
A person decides.
More than one fix is defensible and the choice turns on context the agent doesn’t have, as with the two reds, so it recommends one and asks. That isn’t a failure.
Proposed A new interaction. A confirmation step works technically. Does the product actually need it?
Needs approval
A person authorizes.
The agent knows how to make the change, but it’s beyond its authority, so the right owner approves it. That isn’t a technical problem.
In the prototype A shared component. Fixing one screen changes forty others. Should the agent proceed?

The prototype plays two of these stops. The same stop can come up around an ambiguous requirement, a shared design-system change, an accessibility trade-off, conflicting patterns, an architectural choice, unwritten product intent, a consequential workflow change, or an exception to a rule. None of those is built; the two runs are what the prototype demonstrates.

Agent Review is designed for the space between passing checks and making the right call. Routine work continues. Unresolved decisions go to the right person. Useful answers become guidance for the next run. That holds as agents improve: autonomous work needs an explicit way to tell what can be finished, what needs judgment, and what needs authorization.

Try the second run

The prototype’s second run, in its dark theme: a decision marked Needs approval, “The fix is in a table 40 screens share. Change it for all of them?”, with what the agent was asked and what it proposes, the one-line change to the shared Table, the forty screens by area and the Table’s owner, what was checked beside what wasn’t (12 of the 40 screens have a visual baseline; 28 have none), “So this isn’t a technical problem. It’s a permission decision.”, and three answers: this screen only, recommended; all 40 screens, approved once; or more evidence first.
The prototype’s second run, also simulated. Each of its three answers ends differently.

The design decisions behind it

My first version was a queue of changes waiting for review. As agents get more capable, approving every change becomes the bottleneck, so I redesigned around completed work: what happened, what was verified, and what still needs a person.

Two more decisions make that safe:

  • Job boundaries, not confidence scores. A few sentences per delegation say what the agent may do on its own, and every change cites the one it acted under. Design rules say how to build; job boundaries say what this agent may decide.
  • Decisions that keep their reasons. When a person decides, the record keeps why, what they accepted and what’s left to do. It becomes a rule only if they keep it, scoped and revocable. A wider permission, like changing a shared component, has to come from that component’s owner.
A decision record in the prototype: keep the red for now, decided by you at 14:26 on the billing activity. What the agent did: kept the red on --color-danger, added a “Replaced” label and icon, wrote the exception into the design system’s list, opened a task to review it, recorded the decision, and applied it to the three changes the rule covers. Then the reason, in the person’s words; the trade-off accepted; the risk that remains; an open follow-up; and the reusable guidance, kept as a rule that three later changes followed.
A decision, recorded. The reason, the trade-off and the follow-up stay with the work, and each change that follows it names it.

The experiment

Before the prototype, I built a React component system, wrote twelve design rules for an agent to read before touching it, and compared three versions of one feature, bulk actions on the customer table:

  • Run 1: Claude Code, with the system and the twelve design rules, told to read the rules first.
  • Run 2: the same request, with the rules and the project’s documents removed.
  • An intentionally flawed version, to test the checks: four defects the rules forbid, planted by hand. None of its failures is an agent’s.

I ran the same checks on all three (a token lint, accessibility tests, a layout check and visual comparisons) and reviewed each myself. Each run happened once, as a subagent of my own Claude Code session with only the repository to read: a small experiment, not a representative study.

The ArchiveConfirmation story: three customers selected, and the selection toolbar asking “Archive 3 customers?” with Cancel and a red Archive 3 customers button.
Every component state is a story, and every story is a test. The component system’s Storybook; the agent wrote this story in Run 1.

What the two runs showed

Both runs stayed inside the component system.
Each built the feature from the existing components, with the design rules or without them, and neither hard-coded a value.
The design rules mattered where the system was silent.
They settled what components could not: changing a shared one, what an emptied data table shows, and where a new pattern goes.
The checks found no defect in either agent’s version.
On the intentionally flawed version, they found what I planted: 17 hard-coded values and text at 2.87:1 contrast.
Review changed the system. No run has tested it since.
The overflow test had passed a toolbar 58px too wide, so I rewrote it, and four design rules gained wording.

Neither the design rules nor the checks removed the need for judgment. It moved to the questions the checks left open:

  • Was the shared change a real fix, or a shortcut?
  • Is the new confirm-before-archive step one the product wants?
  • Is a changed screen the feature, or a break? The visual comparison flagged four on every version and couldn’t tell.

From one decision to a better system

A person’s decision, with its reason, can become guidance for the next run, scoped to where it applies. Over time, the goal is fewer interruptions, not less oversight.

  1. SystemComponents, tokens, stories
  2. AgentBuilds the change
  3. ChangeA branch to inspect
  4. Automated checksLint, accessibility, layout, visual. Routine work merges from here
  5. Human judgmentOnly what the checks can’t settle, decided or approved by whoever owns it
  6. Stronger systemThe next agent inherits it

What I’d validate next

I haven’t tested Agent Review with anyone yet. Next, I’d put it in front of designers and engineers supervising real agent work, and measure four things:

  • Understanding: can they say what the agent did, from the account alone?
  • Catching: do they spot a decision the agent got wrong on its own?
  • Deciding: do they weigh both directions, or take the recommendation by default?
  • Guiding: can they set up guidance without spending more time reviewing?

The same test would show whether the three kinds of work hold up beyond these two runs. The question is whether a team can delegate more and stay just as informed and in control.