All field notes

AI · Retrospective

Scoping a durian-grading model before writing any training code

A white durian beside a stack of photo cards and a small scale with a blue pan.
Concept illustration · AI-generated

My Trivechain work this month included adding asset issuance to the TriveAsset SDK on 6 July. On 12 July I started something smaller and more personal: a public repository called durian3, with a README and no code. It is a hobby project, and so far it is a plan written down before any code.

Four questions I keep being asked

As a durian lover in Malaysia, I get asked the same four things:

  1. Where is a durian stall near me?
  2. What price should I expect when I get there?
  3. Is this durian really the variety the seller says it is?
  4. Is it a good one?

They sound like one problem. They are two. The first two are about collecting and averaging information, and an app can do that without any machine learning: people record the stalls they visit and the prices they see, stall owners update their own prices, and averaging those prices gives a rough idea of what a durian will cost and how the price has moved. The last two are where a trained model might help.

Why the data comes first

So the README splits the work into phases. Phase 1 is the app: create a stall, update the price of a variety at that stall, upload photos tagged with the variety and with whether they show the seed, the flesh or the husk, and view average and historical prices. Phase 2 uses those photos to train a deep-learning model, and then relates its predictions to prices.

The order is deliberate. A model that identifies varieties needs labelled photos, and I do not expect to find a ready-made, well-labelled collection of photos taken at Malaysian stalls. Phase 1 is how the labels would get made. It also exposes the weakness of that plan early: crowd-sourced tags are cheap and noisy. A buyer may not know the variety. A seller has a reason to name the more expensive one. Before any training, I would want a way to judge how far to trust each label, for example by checking whether several people tagged the same stall's fruit the same way.

What people look at

When people judge a durian, they look at the colour of the thorns, the shape of the thorns, the gaps between them, the overall shape of the fruit and the colour of the shell, and some go by its weight. Different grades sell in different price ranges, so a grade estimate is worth having.

Writing those features down made the gaps obvious:

  • Weight is not in a photo. It has to be entered, which means a scale and an honest number.
  • Colour depends on the light. A husk under a stall's fluorescent tube and the same husk in afternoon sun will not match. The model has to learn the fruit, not the lighting.
  • Size depends on distance. Thorn spacing in pixels means nothing without something of known size in the frame, or a feature that does not depend on scale, such as a ratio.
  • Variety and grade are different targets. A variety is a label that can, in principle, be checked. A grade is a judgement, and two experienced sellers may not agree. A model trained on disagreeing labels will learn the disagreement.

How I would check it

Before calling anything accurate, I would hold back photos from stalls the model never saw in training. Several photos of one fruit, or of one stall's fruit under the same light, are close to duplicates, and testing on them would flatter the result. I would also look at which varieties get confused with each other, not only at the overall score, because the costly mistake is calling a cheaper durian a premium one.

My own deep-learning research, at Multimedia University, was on predicting network faults, and it was published in IJET in 2018. Network faults and durian husks have little in common, but the order of work carries over: decide what is being predicted, find out where the labels come from and how good they are, and only then choose a model.

A plan, not a product

GPT-3 demos have been circulating widely this month: a language model trained on a very large collection of text, writing fluent paragraphs from a short prompt. My durian plan sits at the other end of the scale. It has a narrow question, a set of photos that does not exist yet, and labels that have to be earned one stall at a time.

Neither the app nor the model exists. What exists is a README that separates the price problem from the grading problem, puts data collection first, and says discussion is welcome. Anyone who cares about durians is invited to argue with it.