Writing · 5 min read

What does a person need to supervise an AI agent?

More than an approve button. A person supervising an agent needs to know what it is allowed to do on its own, to be asked before the step that is expensive to undo, to read afterward what it did and what each step stood on, to stop it and pick the work up where it stopped, and to change its instructions and test the change before the next run. The scenario below is illustrative, and the prototype it draws on recommends rather than acts; that difference is the subject.

A large sage boom barrier drawn flat from the side, its post at the left, its arm raised at forty-five degrees and running off the top of the frame, and a solid black counterweight on the arm’s short tail.

Key idea 1 of 7

A recommendation is one decision; an agent is a run of them

Everything I have written about AI on this site is about one decision. A system recommends, a person judges, and the interface is what sits between the two. The dispatch cockpit is that shape exactly: one load, one truck, one press.

An agent is a different shape. It is given a goal rather than a question, and it takes a run of steps to reach it: read the failed order, find the missing field, look up the partner’s rule, draft the fix, resubmit, move to the next one. Nobody is watching each step, which is the point of it and also the problem. The unit a person has to be able to judge is no longer the answer. It is the run.

Take an agent that works the overnight queue of failed orders. By morning it has touched forty. The specialist who arrives at seven does not need forty recommendations to review. They need to know which of the forty it was sure about, which it was not, what it did when it was not, and whether any of that can still be undone. Every section below is one of those needs.

Key idea 2 of 7

Draw the line where a mistake gets expensive, not where it gets possible

The first design decision is not on a screen. It is the line between what the agent may do on its own and what it has to ask about, and the place to draw it is where a mistake becomes expensive to reverse, not where a mistake becomes possible. Reading an order costs nothing to get wrong. Drafting a correction costs a review. Resubmitting the order to a trading partner costs a phone call. Changing the rule that translates every order from that partner costs a week.

One run, and where the line falls in it
  1. Readthe failed order; nothing to undo
  2. Look upthe partner’s rule; nothing to undo
  3. Draftthe correction; a review to undo
  4. Resubmitthe order; a phone call to undo
  5. Change the ruleevery future order; a week to undo

The cockpit draws this line in one place. Assign stages a choice that Undo reverses, and the page says that where Assign sent a driver, a confirmation would earn its place. That is the same line, drawn for a system that only recommends. Autonomy is not a setting on the agent. It is a boundary in the workflow, and engineering has to enforce it where the copy describes it. The person supervising the agent has to be able to see where the line is and move it, for this partner or this kind of order, without opening a ticket.

Key idea 3 of 7

Approval is a question with the evidence attached

When the agent reaches the line, it asks. How it asks is most of the interface. A question that says “Resubmit order 4471?” is a question people learn to answer yes to, because there is nothing in it to judge.

The question that works carries its evidence. This order failed on a missing customer identifier; the same field is missing on eleven others from this partner since Tuesday; here is the corrected document beside the original with the change marked; here is the last time this correction was made, and what happened. That is the cockpit’s card asked in the other direction: the same factors and the same record, arriving as a question rather than a suggestion. An approval without the evidence is a rubber stamp, and the rubber stamp is what the agent was supposed to replace.

Two things belong on the question and are usually missing. What happens if nobody answers, because at three in the morning nobody will, so the agent has to hold the order or skip it and say which it did. And what the answer is worth: whether a yes here applies to this order alone or teaches the agent to stop asking about this kind of case. A person who cannot tell those apart will answer the first one and be surprised by the second.

Key idea 4 of 7

The run has to be readable afterward, by someone who was not there

Whatever the agent did on its own has to be readable afterward by a person who was not in the room, and readable is a stronger word than logged. A log is what the engineers get. The specialist at seven needs the run as a list: forty orders, each with what the agent decided, the step at which it decided it, what that step stood on, and whether the effect can still be reversed. The three that hit the line and waited are at the top. The one it skipped because the partner’s rule did not cover it is next. The thirty-six it corrected and resubmitted are a group, opened only if something in the morning looks wrong.

What the log holds, and what the person needs from it

The log

  • Every call, in the order it was made
  • What the model returned, in full
  • Timestamps to the millisecond
  • Written for the engineer who built the agent

The record

  • Every decision, grouped by how sure the agent was
  • What each one stood on, and how old that was
  • Which are still reversible, and until when
  • Written for the person who has to answer for it

This is the part the cockpit does not do yet, and its page says so: the override reason is saved, and what the system recommended, on which figures, at what time, is not written down. An agent that acts has to write both, at every step, or the person supervising it is supervising a memory. The record is the interface. If a decision cannot be reconstructed from it, the agent was not supervised. It was trusted.

Key idea 5 of 7

Stopping is an interface, and so is picking up where it stopped

Stopping an agent sounds like one button, and it is three. Stop before the next step, which is cheap and always available. Stop and undo the last step, which is only possible if the last step was reversible and the record says so. And stop and hand the queue back, half done, to a person, which is the one nobody designs and everyone needs the first time the agent fails at order twenty-two.

What the person needs then is not an error message. It is the state of the work: which orders are done, which are touched and not finished, which are untouched, and a way to take the queue from there without the agent’s help. That is the question an operations floor asks when a transaction stalls, what happened, what it affects, what can I safely do next, asked of the agent instead of the system. The failure halfway through is the ordinary case, and the interface for it is the interface for the whole thing, because it is the one the person actually uses.

The cockpit’s rule for this is that if the recommendation vanished tomorrow, the screen would still be a dispatch screen. An agent’s version is that a queue the agent abandoned is still a queue.

The sage boom barrier from the side with its arm lowered level across the picture, a black road band along the bottom of the frame under it, one small cream card on the road directly under the arm, and a small yellow lamp on the post.
The agent has stopped at the line it was given, and what it wants to do next is on the card, waiting for someone to read it.

Key idea 6 of 7

The instruction is a design surface, and the test is the other half of it

The agent does what its instructions say, and the instructions are written by someone who is not an engineer: the specialist, the dispatch manager, the customer-service lead. That makes the instruction a design surface, and most of them are a text box.

The interface for an instruction needs what a good form needs: what this instruction changes, which cases it applies to, an example of it applied, and what it conflicts with. Then it needs a way to try it before it runs on real orders: last night’s forty, replayed under the new instruction, with every decision that changed marked. That is the test set, and the test is where the person learns whether the change did what they meant. I wrote the same loop into the cockpit’s study: matched scenarios, a plausible wrong answer, a stale figure, run before shipping and again after every change. For an agent it is not a study. It is the control panel. A person who can change the instruction but cannot test it is not in control of the agent. They are editing it and hoping.

Key idea 7 of 7

Measure what the person caught, not what the agent finished

An agent’s dashboard measures what the agent finished: orders cleared, hours saved, the share resubmitted without a person. Those are the vendor’s numbers. The supervising person’s numbers are the other ones: how many of the agent’s decisions a person caught before they cost anything, how many got through, how long the queue took to hand back the night it failed, and whether the instruction changes made the next night better than the last.

What the cockpit prototype already holds itself to, and what an agent would add
1
press to overrule the recommendation, with the reason asked afterward and optional
3
matching override reasons on one customer in a month before a person reviews what the ranking weighs
0
actions the prototype takes without a press; the version that acts is the one this article is about

The cockpit’s measure is the decision, not the acceptance. An agent’s is the catch, not the completion. An agent that finishes everything and is never caught is either perfect or unsupervised, and the dashboard cannot tell you which.