Machine learning · extension lesson
Convolutional neural networks: from pixels to features
A small pattern can appear anywhere in an image. A convolutional neural network (CNN) applies the same small set of weights across nearby pixels to build a map of responses.
No coding required. Multiplication and addition are shown one patch at a time. This standalone exploration and its checks do not save to your core lesson notebook.
One filter, many positions
A grayscale image is a grid of brightness values. A kernel is a small grid of weights. Here, nine weights cover a 3 × 3 patch. Multiply matching entries, add the products and a bias, then move the kernel. The results form a feature map.
Stride is how many pixels the kernel moves per step. Padding adds a border, here made of zeros. Try stride 2, then add padding: the highlighted patch and output dimensions change together.
As in common CNN libraries, the kernel is not flipped. The exact operation is cross-correlation, conventionally called convolution in this context. Source: PyTorch Conv2d.
Synthetic pixels · fixed hand-set filters · bias 0
Input image · 6 × 6
Click a pixel, or Tab to it and press Space. Brightness cycles 0 → 0.5 → 1. Blue borders mark the current patch; “pad” cells are added zeros.
| -1 | 2 | -1 |
| -1 | 2 | -1 |
| -1 | 2 | -1 |
Patch 1 / 16 · output row 1, column 1 live
Multiply each pixel by its matching kernel weight, then add all nine products. This is one output cell.
Nine products live
- 0 × -1
= 0 - 0 × 2
= 0 - 1 × -1
= -1 - 0 × -1
= 0 - 0 × 2
= 0 - 1 × -1
= -1 - 0 × -1
= 0 - 0 × 2
= 0 - 1 × -1
= -1
Sum + bias 0 = -3 live
| -3 | 6 | -3 | 0 |
| -3 | 6 | -3 | 0 |
| -3 | 6 | -3 | 0 |
| -3 | 6 | -3 | 0 |
Output side = floor((input + 2 × padding − kernel) ÷ stride) + 1
floor((6 + 2 × 0 − 3) ÷ 1) + 1 = 4 live
Keep positive responses, then reduce the grid
ReLU replaces negative values with zero. It adds a nonlinear step: a negative line-contrast response does not cancel a positive one later. Max pooling then keeps the largest value in each 2 × 2 window, moving two cells at a time.
| 0 | 6 | 0 | 0 |
| 0 | 6 | 0 | 0 |
| 0 | 6 | 0 | 0 |
| 0 | 6 | 0 | 0 |
| 6 | 0 |
| 6 | 0 |
Pooling uses no padding. An incomplete final row or column is dropped. These smaller maps lose exact location information; pooling does not guarantee the same answer for every image shift.
From two feature maps to one label
This second experiment uses your same edited image and both filters at once. Its convolution always uses stride 1 and no padding, independently of the controls above. Each branch applies ReLU, 2 × 2 max pooling, then averages its four pooled values. The larger average supplies the label.
liveHand-set feature/classification pipeline · no training · scores are not probabilities
- 6 × 6 image
- → two 4 × 4 maps
- → ReLU
- → two 2 × 2 pools
- → means → label
| 6 | 0 |
| 6 | 0 |
Mean = (6 + 0 + 6 + 0) ÷ 4 = 3 live
| 0 | 0 |
| 0 | 0 |
Mean = (0 + 0 + 0 + 0) ÷ 4 = 0 live
Output: Vertical live
A tie means equal scores, including a blank image. Any unequal scores force a vertical or horizontal label, even for a scribble that fits neither class. This rule has no “other” category.
Try the horizontal preset, then erase or add pixels. Which branch changes? Try a blank image: no response is evidence for neither orientation, rather than a confident classification.
What a trained CNN adds
The lab’s two filters and score rule were written by hand. In a trained CNN, convolution weights and classification weights are parameters. A training image passes through the network, its scores are compared with the labelled class using a loss, and backpropagation supplies gradients. An optimiser updates parameters to reduce that loss over training examples.
A filter shares its weights across spatial positions. With one grayscale input channel, a 3 × 3 filter has nine weights and one bias, however many positions it visits. Multiple filters produce multiple output channels; colour inputs add red, green and blue channels. Later convolution layers combine local patches across earlier feature channels.
More layers can combine local responses over larger regions. They do not guarantee useful features: the data, labels, architecture and training procedure matter. Test performance on held-out images, including shifted or noisy examples, instead of judging from a few training images.
Source and next reading: PyTorch: training and testing an image classifier.