Writing · 4 min read
Enterprise AI should help people decide, not just answer
A useful AI feature helps someone judge a recommendation and act on it appropriately. That means designing the evidence beside the answer, the uncertainty around it, controls that have a real effect, and the path for when the suggestion is wrong. Acceptance is not the measure of success; the quality of the decision is. The scenario below is illustrative, not an account of a project already delivered.
Key idea 1 of 6
A suspected cause is a place to start, not permission to act
AI earns its place in an enterprise workflow when it helps someone make a better decision or finish a real task. Producing an answer is one part of that. The person still has to understand what the answer rests on, what is still uncertain, and what happens if they act on it.
Take a proposed feature that groups failed orders and suggests a likely cause.
The system notices that twelve rejected orders appear to share a missing customer identifier, and suggests a problem in the rules that translate the partner’s documents into the receiving system’s format.
That is a good place to start an investigation.
It is not yet permission to change the rule or resubmit twelve orders. The design work lives in that distance.
Key idea 2 of 6
Finding, summarizing, suggesting, drafting and acting are five different products
Before choosing between a conversational interface and a recommendation card, I would settle what role we are giving the system.
Is it finding the relevant records? Summarizing the evidence? Suggesting a cause? Drafting a correction? Taking an action? Each of those is a different product, with a different cost when it is wrong.
- Findthe relevant records
- Summarizethe evidence
- Suggesta cause
- Drafta correction
- Actunattended
For this example a useful first release might group related failures, name the suspected field, and help a specialist open the affected documents. A later one might support a reviewed correction. Executing it unattended is its own conversation about reliability, permissions, consequences and recovery.
I would make those calls with product, engineering, architecture and whoever owns the model. A designer should not imply that an action is safe because it was easy to draw as a button.
I would also ask whether the task needs AI at all. A known validation rule deserves a clear deterministic message, not a generated explanation of itself.
Key idea 3 of 6
Put the evidence beside the recommendation
For the failed orders, I would keep the recommendation next to the records it is about.
The person should be able to open the affected orders, see the field or the validation result in question, and know when it was last checked. A suspected cause should look visibly different from a confirmed one.
I would work with the technical team to be sure the supporting evidence is drawn from the records rather than generated alongside the answer, because an explanation that sounds right and cites nothing is worse than no explanation.
The wording carries the claim. “These orders share a missing field” and “changing this mapping will resolve these orders” are not the same sentence.
Key idea 4 of 6
Give people control that does something
A specialist might accept the grouping and reject the suspected cause. Someone else might recognize a partner-specific exception the system has never been told about.
The interface has to have somewhere for those responses to go. I would design ways to inspect, correct, dismiss or escalate without losing the task in progress. For a more autonomous feature I would make its authorized scope and its stopping conditions explicit, with engineering enforcing the boundary rather than the copy describing it.
Stopping a pending action and reversing one that already happened are different promises, and I would not label both of them undo.
Feedback needs the same honesty. Dismissing a suggestion might hide it, flag it for review, or feed a later retraining cycle. The interface should say which, instead of implying that the system just learned something.
Key idea 5 of 6
Prototype the wrong answer
I would deliberately build several outcomes: a correct suggestion, a plausible but wrong one, one with incomplete evidence, and a case that changes while the person is reviewing it.
Then watch. Can they detect the mismatch? Do they know how to dig further? Can they keep working when the system cannot help?
Build
- A correct suggestion
- A plausible but wrong one
- One with incomplete evidence
- A case that changes while the person is reviewing it
Watch for
- Can they detect the mismatch?
- Do they know how to dig further?
- Can they keep working when the system cannot help?
- Do they notice that the case moved?
Simulated output is enough to test the interaction before a model is ready, as long as everyone stays clear that a successful prototype test says nothing about whether the model is accurate.
Key idea 6 of 6
Measure the decision, not the acceptance
I would not make acceptance rate the definition of success. A high rate can mean the suggestions are good. It can equally mean people are clicking through them.
Correct decisions, unnecessary interventions, successful recoveries, and whether the underlying problem takes less work to fix: those are the measures, read alongside the AI team’s own evaluation of the model.
The opportunity here is larger than adding a chat window. It is to give someone better evidence, a better starting point, and the right amount of help, while keeping the consequences of the decision legible to the person who owns it.