Learn machine learning · part 2 · session 5
Tree audit
The last session of the part asks for a shippable thing: a tree that meets a cost target, and the exact path behind one refusal.
After this session you will be able to ship a tree against a cost target and reconstruct the exact path behind one refusal.
The gate is both: total cost at or below 260 synthetic units on the audit set, and every node of the path cited correctly, in order.
01 · The question
The cost model is the one carried from 1.5: 10 units for each approved application that is not repaid, 1 unit for each refused application that would have been repaid. Synthetic cost units, invented for this exercise.
This is the part's last session and it asks for a shippable thing. Build a that meets the cost target on the audit set, and set the it refuses at.
Then one refused application is pulled out and you reconstruct the path it took: which column, which comparison, which value, at each node, in order. The engine checks your reconstruction against the real path through your own tree. An explanation that does not match the model is not an explanation, however reasonable it sounds.
Data: the 1.5 audit set. 1,000 synthetic rows, 950 repaid against 50 not repaid, seed 7. The tree is fitted on the 1,200 training rows and scored on the audit set, which it has never seen.
syntheticseed 7 audit set
02 · Predict
In 1.5 you could only move a threshold on a score someone else produced; the cheapest that approach can get on this audit set is 381. Now the score itself is yours.
Before anything runs
With the score itself under your control, and not just the threshold on it, the best cost will be:
Choose something first.
03 · Try it
Step 1 · configure and fit
The leaf's score is the share of its training rows that did not repay.
Fits: 0
Step 2 · check the cost by hand, opens once you have fitted a tree.
Step 3 · ship it
Shipping freezes the configuration and pulls one refused application out of the audit set.
Ask for a hint
Rungs open as you run the session, not as time passes. You have run it 0 times.
1. A question: available after 3 evaluated runs.
2. A narrowing: available after 5 evaluated runs.
3. A worked fragment, on a different example: available after 7 evaluated runs.
4. The answer: available after 9 evaluated runs.
04 · Compare
The verdict appears once the tree is shipped and the path is reconstructed. Taking the answer rung of the ladder also opens it, and records the session as done with the answer shown.
05 · Check your understanding
Your check: reach the stated target
not yetBoth conditions. Total cost at or below 260 synthetic units on the audit set, and every cited column and value matching the real path through your own tree, in order, with a tolerance of 0.02 on numeric thresholds and an exact match on categorical values. Approving everything costs 500, and the fixed score in 1.5 reached 381, so both of those miss.
Not met yet. Keep working the session.
Save your takeaway
Your takeaway
Edit it into your own words, or keep the line as it is.