Lesson 8 of 12 · 3 min read
How do you run a design sprint for an AI product?
Keep the five days, and add the questions a classic sprint never had to ask: is the data there, what role does the AI play, where does its authority stop, and what would make a careful person trust it. Those decisions shape the prototype more than any sketch.
A classic sprint assumes the software does what it’s told
In a usual sprint, the prototype’s behavior is whatever the team drew. With AI in the product, behavior depends on data the team may not have, and it will sometimes be wrong.
Bansi Mehta shared the workbook her team at Koru UX, a design firm, uses with healthcare clients, where a wrong answer has a patient on the other end. It adds eight tools to the week.
Problema canvas of four questions
Datawhat exists, and whether it’s usable
Rolewhat the AI does, and where its authority stops
Decisionsprompts across six dimensions
Patternsseven interaction cards to sketch from
Trusta sixteen-question checklist
Metricsseven categories, each with a threshold
Feedbackhow use improves the next answer
Draw the boundary before anyone sketches
Of the eight, I’d draw the autonomy boundary on day one, because every sketch depends on it.
In Dispatch Cockpit, my prototype for truck dispatchers on made-up data, the line came before the screen: it recommends a truck and never assigns one. Drivers short of legal hours are ruled out before any ranking.
A classic sprint asks
- What’s the problem?
- What could the screen be?
- Can people use it?
An AI sprint also asks
- Is the data there to answer it?
- What may the AI do alone?
- Do people trust it the right amount?
On Friday, test the wrong answer too
A sprint prototype usually shows only what goes right. So put one wrong recommendation in Friday’s prototype, and don’t tell the participants. The test counts how many catch it.
For the cockpit, I’d plant a top truck that looks right, but its driver’s hours were read 45 minutes ago.
A dispatcher who checks the hours passes on it. One who trusts the screen assigns a driver who may be out of hours.
Count the other direction too. Someone who throws out a good recommendation and redoes the work by hand has also failed the task.
When the AI is right
- Accepts it: the product worked
- Rejects it: too little trust, and the work gets done twice
When the AI is wrong
- Catches it: the product worked
- Accepts it: too much trust, and the error ships
The cockpit hasn’t been in front of a working dispatcher, so I can’t say how many would catch it. That’s the session I’d run first.
If nobody in the test catches the wrong answer, find out why. The prototype may have taught them to stop looking.
Check yourself
Could your team say, in one sentence, what the AI may do without asking?
If not, that’s the first hour of the sprint. Every sketch after it is guessing at the answer.
Does your test plan include a moment where the AI is wrong on purpose?
Trust shows in what people do with a bad answer. Count who catches it.
The idea: Bansi Mehta, Koru UX, AI design sprint workbook. The example: Dispatch Cockpit, on synthetic data, on this site.