Learn machine learning · part 8 · session 1
Embeddings
Two thousand words, thirty two numbers each, learned from which words appear near which other words. Close here means occurred in similar company, and the job of this session is to find a pair where that is all it means.
After this session you will be able to pull nearest neighbours out of an embedding space and say what closeness there does and does not mean.
The gate is a pair of words that are mutual neighbours here, plus the right account of what that closeness is.
01 · The question
Each of these 2,000 words is 32 numbers, learned from which words appear near which other words in a pile of old books. That row of 32 numbers is the word's . Nothing else went in. No definitions, no grammar, no facts. So when two words come out close together, the honest reading is that they occurred in similar company, and sometimes that matches meaning and sometimes it does not.
Below you can pull neighbour lists, do the arithmetic on one of them, and try the vector sums people like to quote. Your job is to find a pair the space calls close that you would not. Thirty five minutes.
Vectors trained offline by scripts/ml/train-embeddings.mjs at seed 7 on public-domain text published before 1929: loading the credit
02 · Predict
Because each word is 32 numbers, you can subtract one word's numbers from another's and add a third. People quote king minus man plus woman as landing on queen. On this space, trained on one 1900 novel, where will it land?
Before anything runs
The famous vector sum is king minus man plus woman. On this space, trained on pre-1929 public-domain prose, where will the nearest result land?
Choose something first.
03 · Try it
Loading the 2,000 word vectors.
Ask for a hint
Rungs open as you run the session, not as time passes. You have run it 0 times.
1. A question: available after 3 evaluated runs.
2. A narrowing: available after 5 evaluated runs.
3. A worked fragment, on a two-dimensional toy: available after 7 evaluated runs.
4. The answer: available after 9 evaluated runs.
04 · Compare
05 · Check your understanding
Your check: an outcome you produce
not yetTwo parts. A word pair where both words are in the vocabulary and each is in the other's top ten, and the right account of what that closeness is.
Not met yet. Keep working the session.
What closeness here means
This check confirms the two words are mutual neighbours here and that you picked the co-occurrence explanation. It cannot judge whether you find them related in meaning. That judgement is yours and it is the point of the exercise.
Save your takeaway
Your takeaway
Edit it into your own words, or keep the line as it is.