Learn machine learning · part 7 · session 1
Order is information
Two lines, the same four words, opposite meanings. This classifier only counts how many times each word appears. It never sees which word came first, so 'not good' and 'good not' arrive as the same thing, and you are going to build the pair that proves it.
After this session you will be able to build two lines a word-counting classifier cannot tell apart, and say what counting throws away.
The gate: one submission in which all four checks hold, with equality tested on the scores rather than on the probabilities.
01 · The question
Two lines, the same four words, opposite meanings. "not good" against "good not" is the smallest version of the problem. In Part 1 you turned rows into columns and it cost you nothing. Text is a , and it does not survive that treatment.
What the counter actually sees
illustrativethe two word counts, drawn for the lessonBelow is a classifier that counts words and drops where they sit, fitted here on 24 short phrases I wrote for this session. Your job is to build the pair of lines it cannot separate, using only its own vocabulary. Twenty five minutes, and the gate is a behaviour rather than a number.
02 · Predict
Before anything runs
Reverse the word order of a phrase this classifier scores. What happens to its output?
Choose something first.
03 · Try it
1 · read the training set
The 24 phrases this classifier is fitted on
syntheticwritten for this sessionI wrote these lines for one exercise about word order. They are not drawn from anywhere and they are not sentiment data. The label says which side of the exercise a line was written for and nothing else.
| # | phrase | label |
|---|---|---|
| 1 | this meal was very good | 1 |
| 2 | this meal was not good | 0 |
| 3 | the room is clean and warm | 1 |
| 4 | the room is dirty and cold | 0 |
| 5 | a lovely day | 1 |
| 6 | a terrible day | 0 |
| 7 | the service was quite fast | 1 |
| 8 | the service was very slow | 0 |
| 9 | the film is really great | 1 |
| 10 | the film is really dull | 0 |
| 11 | the book was fine | 1 |
| 12 | the book was poor | 0 |
| 13 | that trip was always good | 1 |
| 14 | that trip was never good | 0 |
| 15 | the price is quite fine | 1 |
| 16 | the price is quite awful | 0 |
| 17 | it was a good meal | 1 |
| 18 | it was a bad meal | 0 |
| 19 | not bad | 1 |
| 20 | not great | 0 |
| 21 | the day was warm and lovely | 1 |
| 22 | the day was cold and dull | 0 |
| 23 | clean enough but not great | 1 |
| 24 | not clean enough and too cold | 0 |
The 8 held-out phrases
| # | phrase | label |
|---|---|---|
| 1 | the meal was great | 1 |
| 2 | the meal was terrible | 0 |
| 3 | a fast service | 1 |
| 4 | a slow service | 0 |
| 5 | this book is quite good | 1 |
| 6 | this book is not good | 0 |
| 7 | the room was lovely and clean | 1 |
| 8 | the room was poor and dirty | 0 |
2 · fit the classifier
200 full-batch gradient steps at learning rate 0.5 over the 24 phrases above. The input is one count per vocabulary word and nothing else: where a word sits in the line is not part of it.
3 and 4 · score a phrase, then reverse it
Fit the classifier first.
5 · build the pair
computed in this browserchecked in this browserTwo lines, both built from the 40 words below, with the same word counts and opposite intended labels. The engine checks four things in order and names the first that fails.
The 40 words you can use
a · always · and · awful · bad · book · but · clean · cold · day · dirty · dull · enough · fast · film · fine · good · great · is · it · lovely · meal · never · not · poor · price · quite · really · room · service · slow · terrible · that · the · this · too · trip · very · warm · was
Fit the classifier first.
Ask for a hint
Rungs open as you run the session, not as time passes. You have run it 0 times.
1. A question: available after 3 evaluated runs.
2. A narrowing: available after 5 evaluated runs.
3. A worked fragment, on another vocabulary: available after 7 evaluated runs.
4. The answer, on this session's vocabulary: available after 9 evaluated runs.
04 · Compare
05 · Check your understanding
Your check: an outcome you produce
not yetAll four checks hold in one submission: both lines use only vocabulary words, the word counts are identical, the intended labels differ, and the two scores come out the same to twelve decimals. The scores are compared rather than the probabilities, because two probabilities can agree to printing precision while the scores differ.
Not met yet. Keep working the session.
The two lines must come out with the same score. I allow a difference of 0.000000000001 because adding the same numbers in a different order leaves a speck of rounding dust.
Save your takeaway
Your takeaway
Edit it into your own words, or keep the line as it is.
Fit the classifier above before adding it to the shelf, so the weights on the shelf are the ones you saw.