Which bay holds capsule A?
Stipulated p: 95%
Answer score 0.95 · abstain 0.00
Rule: answer. Fixed answer outcome: correct.
Machine learning · extension
Hallucination and uncertainty
A fluent answer can still be false. Change how a toy exercise scores wrong answers and abstentions, then trace individual claims back to their evidence.
Standalone lesson. No coding required. Results are illustrative and stay on this page; checks do not save to the core course notebook.
Plausible text is not a fact check
Here, hallucination means generated factual content that is false or unsupported by the relevant evidence. A false statement alone does not establish deliberate deception. These labels describe different questions: whether the claim is supported, and whether there is evidence of intentional misleading behaviour.
A language model assigns probabilities to next tokens given preceding text. A high token probability means a continuation is likely under that distribution; it is not a calibrated probability that the resulting factual claim is true. Smooth phrasing and precise numbers do not supply evidence.
01 · change the scoring rule
When does answering earn more?
Expected score is the average score over hypothetical repetitions. If a correct answer earns 1 and both a wrong answer and abstention earn 0, even a low chance of correctness gives answering a positive expected score. Add a wrong-answer penalty or credit for abstention and that comparison can reverse.
Start at zero penalties, then raise the wrong-answer penalty to 2. Finally raise abstention credit to 1. Watch coverage as well as accuracy.
Illustrative · computed scoring experiment
A correct answer earns 1 point. These six fictional questions have stipulated probabilities of a correct answer, not token probabilities or measured model confidence. Sample outcomes are fixed; moving a slider never reruns them.
Answer expected score = p × 1 − (1 − p) × penalty. Abstention score = credit. The rule answers only when its expected score is higher; ties abstain.
How many samples arrived?
Stipulated p: 80%
Answer score 0.80 · abstain 0.00
Rule: answer. Fixed answer outcome: correct.
Which dock received capsule B?
Stipulated p: 65%
Answer score 0.65 · abstain 0.00
Rule: answer. Fixed answer outcome: incorrect.
Which courier carried it?
Stipulated p: 50%
Answer score 0.50 · abstain 0.00
Rule: answer. Fixed answer outcome: correct.
At what minute did it arrive?
Stipulated p: 30%
Answer score 0.30 · abstain 0.00
Rule: answer. Fixed answer outcome: incorrect.
What was its launch temperature?
Stipulated p: 10%
Answer score 0.10 · abstain 0.00
Rule: answer. Fixed answer outcome: incorrect.
Expected total 3.30
Realised total, fixed sample 3.00
Coverage: answered / all 6/6 · 100%
Accuracy among answers 3/6 · 50%
Expected score averages hypothetical repeated trials under the stipulated probabilities. Realised score uses this one fixed sample. High conditional accuracy with low coverage does not mean every question was answered correctly.
Research connection: Kalai et al., Why Language Models Hallucinate (2025) analyse statistical errors and scoring incentives that favour guessing over abstention. This exercise isolates one incentive; it does not explain every hallucination or simulate real training.
02 · check the evidence
Supported, contradicted or missing?
Compare each claim with the source sentences. A missing detail calls for a narrower statement or abstention. It is different from a source explicitly contradicting the claim.
Illustrative · fictional source exercise
The station log
- S1. Capsule A is stored in bay 3.
- S2. Capsule A arrived on Monday.
- S3. The capsule contains four rock samples.
Classify each claim against this log only. These are authored examples, not a model transcript.
A citation helps only when its source supports the claim. This exercise tests support within a fictional log; real sources also need checks for reliability and recency.
What transfers to real systems
Evaluate incorrect claims, abstentions and coverage separately. Check whether citations support the actual wording. Retrieval can supply evidence, but an irrelevant or misread source can still produce unsupported claims. Calibrating confidence requires measurements against correct outcomes; a confident tone is insufficient.