Why your AI makes things up, and 6 ways to stop it

You ask an AI assistant a simple question. It answers confidently, with a name, a number or a reference that does not exist. This is called a hallucination, and it is the most common reason people stop trusting generative AI.

The good news: most hallucinations are predictable, and you can reduce them a lot with a few simple habits.

Why it happens

A large language model does not look facts up. It predicts the next most likely word, based on patterns it learned during training. Most of the time, the likely answer is also the true one. But when the model does not really know, it still produces something that sounds right, because sounding right is what it was trained to do.

Hallucinations are most likely when:

• The question is about something recent, niche or private (your company, your documents)
• You ask for exact details: figures, dates, quotes, links, citations
• The prompt pushes for an answer at any cost ("Just tell me the number")
• The conversation is very long and early details get lost

1. Give it the facts (grounding)

The strongest fix is to stop asking the model to remember and start asking it to read. Paste the document, policy or data into the prompt and tell it to answer only from that.

Answer using only the text below. If the answer is not in the text,
say "Not in the document."

[paste your text here]

In apps and agents, this pattern is called retrieval-augmented generation (RAG): the system first searches your content, then hands the relevant pieces to the model.

2. Give it permission to say "I don't know"

Models try to be helpful. If "I'm not sure" is not an acceptable answer, they will guess. Say it explicitly:

If you are not confident, say so and explain what you would need to check.

3. Ask for sources, then check them

Ask the model to quote the exact sentence from your document that supports each claim. Made-up claims usually cannot produce a matching quote. For web answers, use a tool with live search and actually open the links. A citation you have not opened is not a citation.

4. Break big questions into small steps

"Write a full market report" invites invention. "List the five facts you found, then summarise them" keeps the model honest at each step, and lets you spot the weak link.

5. Lower the creativity for factual work

If you build with an API, a lower temperature setting (for example 0 to 0.3) makes output more consistent and less inventive. Keep higher settings for brainstorming and creative writing.

6. Add a checker for anything that matters

For agents that take actions or produce reports, add a second step that reviews the first: a separate prompt or model that checks each claim against the source, or simple code that validates numbers, dates and links. Humans should still review anything high-stakes.

Quick checklist

1. Did I give it the source material?
2. Did I allow "I don't know"?
3. Did I ask for quotes or citations, and check them?
4. Is the task broken into steps?
5. Is a person or checker reviewing the result?