Learn machine learning · part 1 · session 1

What a prediction is

Two thousand synthetic loan applications, five columns you can read, and a baseline that is already right seven times in ten. Beat it by hand, then say what you expect the rows you have never seen to do.

After this session you will be able to beat a baseline with rules you wrote by hand, and say why the score on rows you have not seen is the honest one.

rule.handSession 1.1About 35 minutesYour check: demonstrate the idea yourself

This gate checks that you committed to a direction before seeing the answer. Being wrong about the size is expected.

01 · The question

Here is a table of 2,000 loan applications. Seven of every ten were repaid, so guessing "repaid" every time is right about seventy per cent of the time. That is your , and it is not nothing. is just: out of every row, how many did you get right. Your job in this session is to beat the baseline by hand: write up to three if/else rules over the columns you can see, and watch the training score move.

1,400 of the rows are yours to work with. The other 600 I keep back and do not show you, the . They stand in for the applications that have not arrived yet. Before I show them to you, you will write down what you expect your score on them to be. The gap between those two numbers is most of what this track is about.

syntheticseed 7 · 2,000 rows

Eight training rows of the synthetic loan table
income_band	has_collateral	years_employed	debt_ratio	loan_amount	outcome
mid	no	0	0.55	14200	did not repay
mid	yes	16	0.16	19700	repaid
low	yes	3	0.47	23000	did not repay
low	no	0	0.66	7600	did not repay
high	yes	7	0.39	37800	repaid
low	no	14	0.66	14300	did not repay
mid	yes	8	0.45	27300	did not repay
mid	no	3	0.49	13600	did not repay

Synthetic loan applications, generated in this browser from seed 7. No real lending data is used anywhere in this track.

One row, one guess, one tick

syntheticseed 7 · six rows of the table above

Step 1 of 4

One row, its answer covered.

Accuracy is that counter: how many of the covered answers you guessed right.

02 · Predict

Before anything runs

You are about to write rules and see how often they are right on the 1,400 rows you can see. Now guess: on the 600 rows I kept back, will your score be higher, lower, or the same? Then put a number on it.

Your held-out prediction0.50 to 1.00

One line on why, if you want one

03 · Try it

Step one: read the baseline

Predicting repaid for every row is the number to beat. It is honest, it is free, and it is not nothing.

Step two: write up to three conditions

computed in this browser

Conditions are read from the top. The first one that matches a row sets its prediction, and a row no condition matches takes the majority class.

No conditions yet, so the rule set is the majority rule. Add one to move the score.

Training accuracy on 1,400 rows

computed in this browserseed 7

0.698

977 of 1,400 rows marked correctly. The majority baseline is 0.698.

Write at least one condition before locking.

Refuse means your rule predicts this application will not be repaid. Approve means it predicts repaid.

Your rules mark 977 of 1,400 training rows correctly. What is the training accuracy?

To three decimals, within 0.002.

Attempts: 0

Write a condition first. The check needs a rule set to count.

Step four: spend the held-out rows

The held-out rows are scored once, after the rules are locked and after your prediction is recorded. That order is the whole point of the session.

This button is off because the rules are not locked yet.

04 · Compare

The verdict appears once you have locked the rules and scored the held-out rows.

05 · Check your understanding

Your check: an outcome you produce

not yet

Three things have to hold: the rules you locked score above the majority baseline on the training rows, a held-out prediction was recorded before the reveal was available, and that prediction sits below your training score. The size of the gap is recorded and never graded.

Not met yet. Keep working the session.

not yet Above the baseline: 0.698 against 0.698.
not yet Committed before the reveal.
not yet Prediction below the training score.

This gate checks that you committed to a direction before seeing the answer. Being wrong about the size is expected. Scores on rows you practised on are almost always flattering, so a sensible guess is a little lower than your training score. If you disagree, that is a fine thing to be wrong about here.

Save your takeaway

Ask for a hint

Rungs open as you run the session, not as time passes. You have run it 0 times.

1. A question: available after 3 evaluated runs.

2. A narrowing: available after 5 evaluated runs.

3. A worked fragment, on a different example: available after 7 evaluated runs.

4. The answer: available after 9 evaluated runs.

Your takeaway

Edit it into your own words, or keep the line as it is.

next up

1.2 Features: the model only sees columns Play now

Open the notebook