What does a person need to supervise an AI agent?

Key idea 1 of 7

A recommendation is one decision; an agent is a run of them

Most of what I have written about AI here is about one decision. A system recommends and a person judges, with the interface in between. Second Look, my dispatch prototype, is that shape exactly: one load, one truck, one press.

An agent is a different shape. It is given a goal rather than a question. It takes a run of steps: read the failed order, find the missing field, look up the partner’s rule, draft the fix, resubmit, move on.

Nobody is watching each step. That is the point of it and also the problem. The unit a person has to be able to judge is no longer the answer. It is the run.

Take an agent that works the overnight queue of failed orders. By morning it has touched forty. The specialist who arrives at seven does not need forty recommendations to review. They need to know:

• Which of the forty it was sure about
• Which it was not
• What it did when it was not
• Whether any of that can still be undone

Key idea 2 of 7

Draw the line where a mistake gets expensive, not where it gets possible

The first design decision is made off the screen. It is the line between what the agent may do on its own and what it has to ask about. Draw it where a mistake becomes expensive to reverse, not where a mistake becomes possible.

Reading an order costs nothing to get wrong. Resubmitting it to a trading partner costs a phone call. Changing the rule that translates every order from that partner costs a week.

1. Readthe failed order; nothing to undo
2. Look upthe partner’s rule; nothing to undo
3. Draftthe correction; a review to undo
4. Resubmitthe order; a phone call to undo
5. Change the ruleevery future order; a week to undo

Second Look draws this line in one place. Assign stages a choice that Undo reverses. The page says that where Assign sent a driver, a confirmation would earn its place. That is the same line, drawn for a system that only recommends.

Autonomy is not a setting on the agent. It is a boundary in the workflow, and engineering has to enforce it where the copy describes it. The person supervising the agent has to see where the line is and move it, for this partner or this kind of order, without opening a ticket.

Key idea 3 of 7

Approval is a question with the evidence attached

When the agent reaches the line, it asks. How it asks is most of the interface. A question that says “Resubmit order 4471?” teaches people to answer yes, because there is nothing in it to judge.

The question that works carries its evidence:

• This order failed on a missing customer identifier.
• The same field is missing on eleven others from this partner since Tuesday.
• Here is the corrected document beside the original, with the change marked.
• Here is the last time this correction was made, and what happened.

That is Second Look’s card asked in the other direction: the same factors and the same record, arriving as a question rather than a suggestion. An approval without the evidence is a rubber stamp, and the rubber stamp is what the agent was supposed to replace.

Two things belong on the question and are usually missing:

• What happens if nobody answers. At three in the morning nobody will, so the agent has to hold the order or skip it and say which it did.
• What the answer is worth: whether a yes here applies to this order alone or teaches the agent to stop asking about this kind of case.

A person who cannot tell the two kinds of yes apart will answer for one order and be surprised when the agent stops asking.

Key idea 4 of 7

The run has to be readable afterward, by someone who was not there

Whatever the agent did on its own has to be readable afterward by a person who was not in the room. Readable is a stronger word than logged. The specialist at seven needs the run as a list of forty orders, each with:

• What the agent decided
• The step at which it decided
• What that step stood on
• Whether the effect can still be reversed

The three that hit the line and waited are at the top. The one it skipped because the partner’s rule did not cover it is next. The thirty-six it corrected and resubmitted are a group, opened only if something in the morning looks wrong.

The log

• Every call, in the order it was made
• What the model returned, in full
• Timestamps to the millisecond
• Written for the engineer who built the agent

The record

• Every decision, grouped by how sure the agent was
• What each one stood on, and how old that was
• Which are still reversible, and until when
• Written for the person who has to answer for it

Second Look writes only half of this. Its page says so. The override reason is saved. What the system recommended, on which figures, at what time, is not. A recommendation can get away with half a record for a while because a person was there for every press.

An agent that acts has to write both halves at every step, or the person supervising it is supervising a memory. The record is the interface. If a decision cannot be reconstructed from it, the agent was not supervised. It was trusted.

Key idea 5 of 7

Stopping is an interface, and so is picking up where it stopped

Stopping an agent sounds like one button, and it is three:

• Stop before the next step. It is cheap and always available.
• Stop and undo the last step. It is only possible if the last step was reversible and the record says so.
• Stop and hand the queue back, half done, to a person. It is the one nobody designs and everyone needs the first time the agent fails at order twenty-two.

What the person needs then is the state of the work: which orders are done, which are touched and not finished, and which are untouched. They also need a way to take the queue from there without the agent’s help.

That is the question an operations floor asks when a transaction stalls: what happened, what it affects, what can I safely do next. Here it is asked of the agent instead of the system.

The failure halfway through is the ordinary case. The interface for it is the interface for the whole thing, because it is the one the person actually uses.

Second Look’s rule for this is that if the recommendation vanished tomorrow, the screen would still be a dispatch screen. An agent’s version is that a queue the agent abandoned is still a queue.

Key idea 6 of 7

The instruction is a design surface, and the test is the other half of it

The agent does what its instructions say. They are written by someone who is not an engineer: the specialist, the dispatch manager, the customer-service lead. That makes the instruction a design surface, and most are a text box.

The interface for an instruction needs what a good form needs:

• What this instruction changes
• Which cases it applies to
• An example of it applied
• What it conflicts with

Then it needs a way to try the instruction before it runs on real orders: last night’s forty, replayed under the new instruction, with every decision that changed marked. That is the test set. The test is where the person learns whether the change did what they meant.

I wrote the same loop into Second Look’s study. It used matched scenarios, a plausible wrong answer and a stale figure, run before shipping and again after every change. For an agent the same loop is the control panel. A person who can change the instruction but cannot test it is not in control of the agent. They are editing it and hoping.

I have tried a small version of this with a different kind of agent. In Agent Review a coding agent reads twelve written rules before it changes an interface. Each rule names the command that checks it, so an instruction and its test arrive together.

Run once with the rules and once without, the same feature request stayed inside the component system both times. The rules changed what the agent did only at the edges, such as whether it touched a shared file. One request and two runs is a small sample. It suggests the instruction is a smaller lever than it looks. The test is where the control actually is.

Key idea 7 of 7

Measure what the person caught, not what the agent finished

An agent’s dashboard measures what the agent finished: orders cleared, hours saved, the share resubmitted without a person. Those are the vendor’s numbers. The supervising person’s numbers are the other ones:

• How many of the agent’s decisions a person caught before they cost anything
• How many got through
• How long the queue took to hand back the night it failed
• Whether the instruction changes made the next night better than the last

1

press to overrule the recommendation, with the reason asked afterward and optional

3

matching override reasons on one customer in a month before a person reviews what the ranking weighs

0

actions the prototype takes without a press; the version that acts is the one this article is about

That last measure, whether the next night was better, is the one Agent Review taught me to take seriously. After one review there, one of four findings changed a test and one became a line in the rules. Two changed nothing. The lint already caught one. The other already had a token and a rule that had simply not been used.

Deciding which is which keeps the instructions short enough to be read before every run.

Second Look’s measure is the decision, not the acceptance. An agent’s is the catch, not the completion. An agent that finishes everything and is never caught is either perfect or unsupervised, and the dashboard cannot tell you which.