Learn machine learning · part 2 · session 2
Impurity and splits
Twelve rows, three columns, and one measure. Compute the impurity yourself, rank the candidates, then watch the search enumerate and score the same ones.
After this session you will be able to compute Gini by hand and say exactly what a split search enumerates.
The gate is three unseen variants of this slice: call each one before it runs, and two of three passes.
01 · The question
Twelve rows, three columns, small enough to check every number with a pencil. The measure is Gini : one minus the sum of the squared class proportions, so a node with only one class scores zero and an even mix scores 0.5.
A makes two nodes with different numbers of rows in them, so you average their Gini in proportion to their size: (6/12 × left) + (6/12 × right). That average is what "weighted" means everywhere below.
You will compute a child node yourself, then rank the three candidate splits, then watch the greedy sweep score the same candidates in the same order. There is no insight step in the middle. The search enumerates every candidate, scores each one, and keeps the lowest weighted impurity.
The convention it enumerates under: a categorical column splits multi-way, one child per level; a numeric column splits at the midpoint between adjacent sorted distinct values. That is where thirteen candidates come from on twelve rows.
The slice
syntheticseed 7| # | income_band | has_collateral | years_employed | repaid |
|---|---|---|---|---|
| 0 | high | yes | 12 | yes |
| 1 | mid | yes | 8 | yes |
| 2 | high | yes | 21 | yes |
| 3 | low | yes | 3 | yes |
| 4 | mid | yes | 15 | yes |
| 5 | high | yes | 6 | yes |
| 6 | low | no | 2 | no |
| 7 | mid | no | 9 | no |
| 8 | low | no | 1 | no |
| 9 | high | no | 18 | yes |
| 10 | low | no | 4 | no |
| 11 | mid | no | 11 | yes |
Eight repaid, four not. Parent Gini is 0.444444.
Where the thirteen candidates come from
syntheticseed 7 · rows 0 to 1102 · Predict
Before anything runs
Before any arithmetic: which column makes the best root split on these twelve rows? Best at the top.
Put these in the order you expect
Focus an item, press Enter to lift it, move it with the arrow keys, then press Enter again.
03 · Try it
A split is a table cut in two
syntheticseed 7 · rows 0 to 11Twelve rows, one pile, impurity 0.444.
To three decimals or better, within 0.002.
Attempts: 0
To three decimals or better, within 0.002.
Attempts: 0
Rank the three columns, best first
Candidate columns, best root split at the top
Focus an item, press Enter to lift it, move it with the arrow keys, then press Enter again.
Submitted answers: 0
Ask for a hint
Rungs open as you run the session, not as time passes. You have run it 0 times.
1. A question: available after 2 evaluated runs.
2. A narrowing: available after 3 evaluated runs.
3. A worked fragment, on a different example: available after 5 evaluated runs.
4. The answer: available after 7 evaluated runs.
04 · Compare
Run the sweep above and the verdict states what it found, against what you ranked.
05 · Check your understanding
Your check: predict results on new examples
not yetThree variants of the same twelve rows, each with one documented change. Call each one before you run it; two of three passes. Every reveal re-runs the real sweep on the real variant.
Variant 1 of 3
Variant A. A column called application_ref is added: a categorical column of 12 distinct values, one per row. What happens to the sweep?
Call it before you run it.
Variant 2 of 3
Variant B. A column called collateral_posted is added: an exact copy of has_collateral. What happens to the sweep?
Call it before you run it.
Variant 3 of 3
Variant C. Row 9 changes from repaid to not repaid, and nothing else moves. What happens to the root split?
Call it before you run it.
Called 0 of 3 variants so far.
Save your takeaway
Your takeaway
Edit it into your own words, or keep the line as it is.