The workflow test: does this job need an AI tool?

Most AI projects that disappoint were never good fits. The model was fine. The job was wrong.

Before we build anything, for us or for a partner, we ask five questions about the work. None of them is about which model to use. That choice comes last.

1. Is the input messy but the output clear?

Language models shine when the input is loose and the output has a fixed shape. Loose input means emails, PDFs, call notes or photos of forms. A fixed shape means a filled-in record, a category or a draft in a set format.

Good sign: “We read forty supplier emails a day. We copy the quantities, dates and SKUs into a sheet.”

Warning sign: “We want AI to tell us what our strategy should be.” If nobody can say what a right answer looks like, nobody can check one.

2. Can a person check the result faster than they could make it?

This is the most important question.

Say a draft takes thirty seconds to review and ten minutes to write from scratch. The tool pays for itself, even if it’s often wrong. The reviewer catches the misses.

But if checking takes as long as doing the work, the tool adds a step. It doesn’t remove one.

3. Does the job happen often enough to matter?

A task that comes up twice a month seldom earns a new tool, however tedious it is. We look for work that happens every day or every week, across several people. Then small gains add up.

Here’s a rough sum: how often × minutes saved each time × people doing it. If the number is small, a checklist or a template is the better bet.

4. Is the cost of a mistake contained?

Every model makes mistakes. So ask what a mistake costs, and whether someone catches it in time.

• Contained: a ticket gets the wrong tag, and a person moves it within the hour.
• Not contained: a message goes to a customer with the wrong price, and nobody checked it.

When a mistake would cost a lot, we don’t drop the idea. We change the design. The tool drafts, a person approves, and the system records who approved what.

5. Do we have real examples to test with?

We won’t ship an AI feature without a test set. That’s a batch of real inputs, used with permission, each with a known right answer. Fifty good examples are enough to start.

Without them, “it seems to work” is the only quality bar. And that bar moves every time the model changes. If the examples don’t exist yet, collecting them is step one. It comes before any screen design.

When the answer is “not AI”

Sometimes the work fails the test. That’s a useful result too. Here’s what we suggest instead:

• A form, not free text. If the mess comes from people typing whatever they like, add structure to the input.
• A simple automation. If the steps never change and the data is tidy, a rules-based setup is cheaper and faster. It’s also easier to trust.
• A shared template. If the problem is that everyone does it their own way, a good template solves most of it.

How we use this test

When someone brings us a task for our co-build programme, this is our first conversation. It takes about thirty minutes. Either way, you leave with a clearer picture of the job.

Want to run your own work through it? Book a conversation.