Learn machine learning · part 9 · session 4
Pretraining and fine-tuning
Three stages get confused with one another. They differ by the signal the update is computed against and by what they cost.
After this session you will be able to separate the training stages by their signal, and say what a short fine-tune moved and what it left alone.
To pass: five of six scenarios classified correctly by stage.
01 · The question
is the long run that produced the bundled model, on plain prose, predicting the next character, from random initial weights. is a short run on formatted examples, starting from weights that already exist, and it moves what the output looks like. uses feedback about preferred answers; that feedback can come from people or other evaluators. This page describes it and does not run it. Here you play back a 100-step fine-tune of the final block, recorded once by the course script at three learning rates, and read the samples it produced along the way. Then you train a two-class head on twenty examples you write yourself, which is small enough to finish while you look at it.
A fourth thing gets confused with these three: prompting. You change the words you send and no weight changes at all.
Full-scale language-model pretraining and preference training are outside this browser lesson’s scope. For a small agent that does learn from rewards here, try reinforcement learning in a gridworld.
02 · Predict
Before anything runs
After 100 fine-tuning steps on question-and-answer formatted text, this model will:
Choose something first.
03 · Try it
Loading the recorded run.
04 · Compare
Submit the six calls and the verdict lands here, against the position you took above.
05 · Check your understanding
Your check: an outcome you produce
not yetFive of six scenarios classified correctly by stage. The threshold is five rather than six because the boundary between a long fine-tune and a short pretraining run is a matter of degree.
Not met yet. Keep working the session.
Where you are
0 of 6 scenarios correct.
Ask for a hint
Rungs open as you run the session, not as time passes. You have run it 0 times.
1. A question about the signal: available after 3 evaluated runs.
2. Two of the six change no weights: available after 5 evaluated runs.
3. A seventh scenario, worked in full: available after 7 evaluated runs.
4. The six answers with their notes: available after 9 evaluated runs.
Save your takeaway
Your takeaway
Edit it into your own words, or keep the line as it is.
10.3 offers this head for the model slot of its second brief and scores it on strings it has not seen, drawn from the same two neighbourhoods: lines about the road against lines about the witch.