These are the six layers we hold every AI system against, whether we build it or assess it. They're not complicated and you need no technical background — you only have to ask them and listen to what comes back.
Every AI incident you've heard of, from the chatbot that promised a discount that didn't exist to the newspaper with invented quotes, traces back to exactly one of these missing layers.
Six questions. Put them to the system you're considering, and note what comes back for each layer.
The sources
Where does this answer come from? Show me, per answer, which document it rests on.
What you should hear
Every answer points at its source. If it isn't there, the system says “I don't know” instead of making something plausible up.
The red flag
“The model is trained on your documents.” Then there's no source to point at — only a vague memory of the text.
The limits
What is the model allowed to do itself? Who calculates, pays, publishes and deletes?
What you should hear
Calculating, paying, publishing and deleting sit outside the model, in fixed rules. The model proposes; the rules execute.
The red flag
“The model can do that too.” A model allowed to settle a bill can also wander while settling it — and you'll only find out afterwards.
The sign-off
Who signs off before anything goes out, and how does that person see there's something to check?
What you should hear
A human signs off on the substance. The system shows where it's unsure, so there's genuinely something to judge.
The red flag
A single “Approve” button under a finished answer. After twenty good ones, nobody reads the twenty-first. That's clicking away, not checking.
The record
Six months from now, can you retrieve what the system answered, on which source, and who approved it?
What you should hear
Recorded per answer: which source, what was answered, who approved. Exportable without the supplier having to be involved.
The red flag
“It's in the logs.” Logs are plumbing, not a record. Ask for an export before you sign, not when you need one.
The data
Where does our data live, who can reach it, and is it trained on? Where is that written down?
What you should hear
Where the data lives, what happens to it, and that it isn't trained on — in the contract, not in the pitch.
The red flag
A verbal promise, or “we only train on anonymised data”. Ask where that's written and what “anonymised” means here.
The people
What does this do to my team? Do they keep judging the work, or do they end up ticking it off?
What you should hear
It takes over the routine work and leaves the judgement with the team. The supplier can name what people keep doing.
The red flag
“Your people won't need to know any of it.” In five years nobody can weigh whether the answer is right.