Correct answer
56Correctness 1.0 + agreement 0.0 = 1.0 points
Machine learning · extension
Sycophancy: agreement versus correctness
When a response changes to match a user’s stated view despite conflicting evidence, agreement can replace accuracy. Hold one arithmetic fact fixed and change only the rewards.
Standalone lesson. All outputs below are authored toy candidates, not model transcripts. Checks do not save to the core course notebook.
Agreement is not always an error
If the user’s factual statement is correct, agreement can be correct too. Sycophancy becomes visible when responses systematically align with the user’s position at the expense of truth or independent assessment. Adapting tone, acknowledging feelings and respecting subjective preferences are different from changing a verifiable fact.
Sharma et al., Towards Understanding Sycophancy in Language Models report sycophancy in five tested assistants across four tasks. Their preference-data analysis and optimisation experiments provide evidence that preference judgements can contribute. This does not establish the same effect in every model or task.
01 · hold the fact fixed
Which candidate earns the most?
Verify 7 × 8 by adding seven groups of eight: 8 + 8 + 8 + 8 + 8 + 8 + 8 = 56. Set the user assertion to 54, then increase correctness reward above agreement reward. Reverse the user assertion and compare the paired evaluation below.
Illustrative · computed candidate scores
Fixed fact: 7 × 8 = 56
There are exactly three authored candidate policies. Each earns correctness reward if its answer is 56, plus agreement reward if it matches the user’s assertion. Abstention earns 0. Highest score wins; ties use the displayed order.
Repeat user assertion
54Correctness 0.0 + agreement 2.0 = 2.0 points
Abstain
No answerCorrectness 0.0 + agreement 0.0 = 0.0 points
Highest-scoring policy: Repeat user assertion. Output: 54.
Bars share a 0–8 point scale. Scores are not LLM probabilities. This is enumeration of three rules, not reinforcement learning or preference training.
Reverse the assertion, keep the fact
An evaluation needs both prompts. Agreement with a correct assertion alone cannot reveal this failure.
| User says | Winning output | Correct? |
|---|---|---|
| 54 | 54 | No |
| 56 | 56 | Yes |
02 · evaluate both assertions
A correct answer once is not enough
The repeat-user policy returns 56 when the user says 56 and 54 when the user says 54. Testing only the first prompt misses the failure. A paired evaluation keeps the factual question fixed while reversing the expressed position. Check correctness as well as consistency: a system can be consistently wrong.
A scoring illustration, not a training recipe
Real preference optimisation uses datasets and model updates, with many interacting signals. This page enumerates three fixed policies. It illustrates how rewarding agreement can favour an incorrect response, but cannot predict a model’s behaviour or quantify a real training effect. Useful evaluation includes factual checks, changed user positions and examples where disagreement is appropriate.