In an AI design proposal, read the prototype plan first

Design proposals for AI products tend to arrive in a familiar shape: discovery workshops, personas, user journeys, high-fidelity screens, a design system, a usability test. Every item on that list is legitimate, and a proposal can contain all of them while missing the one problem that makes an AI product different.

Conventional software is designed on the assumption that the system is correct and the user might not be. An AI component is fast, usually right and sometimes wrong, and the interface has to make checking cheap, doubt obvious and error safe. A proposal either shows that its author knows this or it does not, and you can usually tell within a few pages.

Start with the prototype plan

The most revealing section of any proposal is how the prototype will be built and tested.

Does the prototype contain wrong answers? A prototype that only shows good outputs tests nothing that matters. What you need to learn is whether people notice when the system is wrong, how long that takes them and what they do next. That requires realistic errors placed in the prototype on purpose.

Who is it tested with? People who do the job — not stakeholders, and not a recruited panel that has never done the task. Managers judge whether a product is impressive; the people doing the work judge whether it is faster.

What is measured? Review time, errors caught, errors missed, and the comparison with the current way of working. If the plan measures satisfaction — whether people liked it — it is measuring the wrong thing, and people almost always like AI demos. A good proposal also says what happens to the results: who will read them and which decision depends on them.

Then the order of work

Is the current task watched and timed first? Without a baseline the product has nothing to beat, and any prototype looks like an improvement.

Is the interaction model settled before screens are drawn? Where the system proposes, where the person decides, how disagreement is expressed. If the proposal goes from workshops straight to wireframes, the screens will make that decision by accident.

Is the design system at the end, if it is there at all? A design system built before the interaction has been tested is decoration on an unproven idea. It has its place, and that place is after the product is known to work.

Infographic

Two proposals for the same product

Familiar shape

• –Workshops, personas, user journeys
• –High-fidelity screens early
• –Prototype with good answers only
• –Tested for satisfaction
• –Design system up front

Built for uncertainty

• →Current task watched and timed
• →Interaction model before screens
• →Prototype with realistic errors
• →Tested for review time and misses
• →Design system later, if needed

Smaller signals worth noticing

How confidence is shown. A proposal that plans to display a confidence percentage has not yet discovered that users read percentages as a precision they almost never have. Structure — what the answer rests on, what was ambiguous, what the system declined to answer — usually works better.

Where the sources go. Showing what an answer was based on, within one click, is one of the cheapest ways to speed up review. If provenance is not mentioned, nobody will design it.

What happens to corrections. When a person fixes an output, does that fix go anywhere, or is it thrown away? A proposal that is silent on this will produce a product that asks for the same correction every day.

Honesty about fit. A good provider will tell you if what you really need is a visual refresh from a design studio, or if the task underneath is not yet defined well enough to design for.

What to ask in the meeting

Ask to see the failures from a previous prototype test — not the summary, the findings where people missed errors or gave up. A provider who has done this work will have them and will not be embarrassed by them. One who only has success stories has probably only tested success.

Ask how long one workflow takes. A specific answer — a few weeks for an interaction model and a tested prototype — suggests experience. An open-ended one suggests the work has not been scoped around the decision you need to make.

Finally, ask who from their side will sit in the test sessions. If it is only a junior researcher and the designers read about it in a report, the revisions will be exactly as good as that report.

How we work

We built AI-native apps design to answer these questions the way we would want them answered. An interaction model and a tested prototype for one workflow is typically three to five weeks. We often work alongside an existing design team: they know your product and your users, and we add the design language for uncertainty.