Learn machine learning · deep dive D11 · optional
Top-k, top-p and temperature
Three controls sit between a model's distribution and the text you read. They are usually described together and they do not do the same thing, so this dive measures each one instead of judging it by eye.
This is optional. It opens once 9.2 Next-token prediction is done, and nothing in the track waits on it. The gate is a setting that reaches a distinct-trigram ratio of at least 0.55 with a longest repeat of at most 20 characters, in one batch at seed 7.
01 · The question
9.2 put you inside one step of the choice. This dive is about the whole run: what the text looks like after two hundred of those steps, and what changes when each of the three controls moves.
Two measures do the judging. The distinct-trigram ratio is the number of distinct three-character sequences in a sample divided by the number of three-character sequences in it, so text that goes round in circles scores low. The longest repeat is the longest piece of text that appears twice inside one sample. Neither is a quality score, and the page never calls a sample good.
The model is the bundled character model, trained offline. The sampling runs here, at a printed seed.
02 · Predict
Before anything runs
Which of the three controls changes how many candidates are left to draw from at each step, in a way that depends on the step?
Choose something first.
03 · Try it
The bench
Move one control at a time and generate a batch. The measures are computed over the whole batch; the first two samples are printed in full and the rest are behind a disclosure. Everything the model produces at these settings is what the model produces: nothing is filtered for presentability.
Three controls, one batch
trained offline by the course scriptLoading the bundled character model.
Three settings worth running
- Temperature 0.2 with the other two off. Read the longest repeat before you read the text.
- Temperature 1.6 with the other two off. Read the in-vocabulary word fraction.
- Top-k 1. Read whether the samples are identical, and what that does to the ratio across the batch against the ratio within a sample.
Ask for a hint
Rungs open as you run the session, not as time passes. You have run it 0 times.
1. A question: available after 3 evaluated runs.
2. The answer: available after 5 evaluated runs.
04 · Compare
05 · Check your understanding
Your check: two measures on the batch you generate
not yetOne batch that reaches a distinct-trigram ratio of at least 0.55 within a sample, with a longest repeat of at most 20 characters. At least six samples of a hundred characters, so the measures mean something.
Not met yet. Keep working the session.
No batch has cleared both measures yet.
Save your takeaway
Your takeaway
Edit it into your own words, or keep the line as it is.