What are Hallucinations in LLMs?

What is it

A hallucination in an LLM is when the model generates factually incorrect, fabricated, or inconsistent information, presented with the same confidence and fluency as correct information. The model does not "lie" (it has no intent) — it simply produces plausible text based on statistical patterns, without a concept of objective truth.

Hallucinations can be:

• Factual: The model invents facts, quotes, dates, or sources.
• Faithfulness: The model ignores prompt instructions or contradicts context.
• Self-consistency: The model contradicts itself within the same response.

Mental model

Imagine a librarian who has read millions of books but has no memory of which ones are real. When asked a question, they respond by combining information from all books, but without being able to distinguish between an encyclopedia and a work of fiction. They will give answers that sound authoritative but may be completely made up.

How to mitigate

No perfect solution exists, but there are effective strategies:

# 1. Lower temperature for more factual responses
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}],
    temperature=0.1
)
 
# 2. RAG system: provide verified context
context = retrieve_from_database(user_query)
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": f"Answer based SOLELY on:\n{context}"},
        {"role": "user", "content": user_query}
    ]
)
 
# 3. Prompting that forces verification
def safe_prompt(query):
    return f"""Answer only if you are 100% sure.
    If unsure, say 'I don't have enough information'.
    Question: {query}"""

When to worry / when not to

Worry about hallucinations when:

• The application impacts critical decisions (medicine, legal, finance).
• Generated content is published without human review.
• Factual accuracy is essential.

Less concerning when:

• The use is creative or brainstorming.
• Human oversight is in the loop.
• The domain is closed and controlled (internal documentation).

Always implement a factual verification system for applications requiring accuracy. Combine RAG with explicit citations and an automatic validation step. Don't rely on the model to self-regulate its hallucinations.

Hallucinations are inherent to LLMs, not a bug that can be fully fixed. Any system using LLMs must be designed assuming it will occasionally generate false information. Mitigation is the system's responsibility, not the model's.