What is hallucination in AI?

In practice

The controls that work in combination: retrieve the source material rather than relying on memory, require citations and show them, constrain the output to a structure that can be validated, check figures against the system that owns them, and put a person in front of anything binding. Each one narrows the space in which a fabrication can survive to the end.

Where it fits in a build

Tolerance depends entirely on the consequence. A wrong suggestion in a draft costs seconds. A wrong figure in a tender submission or a wrong clause in advice to a client costs considerably more, and those workflows should never be designed on the assumption that the model is right.

Common mistakes

• Presenting model output to a customer without a human or a validation step in between.
• Judging accuracy from a handful of impressive demos rather than a measured sample.
• Hiding uncertainty. A system that shows what it was unsure about is used properly; one that sounds certain every time is trusted until it is not.

Related terms

• Retrieval-augmented generation (RAG)
• Human-in-the-loop
• Audit trail

Working on something that involves this?

ASTACKRA designs and builds AI systems, automation and custom software for businesses that need technology shaped around their own workflow. If this term turned up in a proposal and you want a straight answer about whether it applies to your situation, ask us — no obligation, and we will tell you if the answer is no.