Module 3 · Foundations

Neural Networks

Neurons, layers, forward pass, backpropagation, Adam, early stopping — this is the machinery both papers run on. By the end you should be able to describe exactly what happens between "data goes in" and "trained model comes out", and to defend every training choice Paper 1 made.

▶

Audio recap

A ~2-minute spoken summary of this module — great for revision on the go.

3.1 From one neuron to a network

A single neuron takes an input vector, computes a dot product with its weight vector, adds a bias, and passes the result through an activation:

output = ReLU(w · x + b)

Stack neurons that share the same input and you get a layer (matrix form: y = activation(Wx + b), Module 1). Stack layers so each feeds the next and you get a network. Two terms must be second nature, because Paper 1's entire hyperparameter search is over them:

Hidden layers (HLs): the layers between input and output — the network's depth.
Hidden units (HUs): the number of neurons in a hidden layer — its width.

The plainest architecture — every neuron in a layer connected to every output of the previous layer, data flowing strictly forward — is the feedforward neural network (FFNN), also called a fully-connected network or multilayer perceptron. It is Paper 1's first model and the building block of all three sub-networks in Paper 2's self-model (coordinate encoder, kinematic encoder, predictive model).

3.2 The forward pass

Inference is just function composition: the input flows through layer after layer until the output layer emits the prediction. For Paper 1's FFNN, the input is a flattened sliding window of recent feature history (e.g. the last hour of power, GHI, and temperature strung out into one long vector), and the output layer has 21 units — the multi-target forecast of the next 1–6 hours.

window of features→ hidden layer 1 (ReLU)→ hidden layer 2 (ReLU)→ hidden layer 3 (ReLU)→ 21 power values

One detail examiners like: Paper 1 applies ReLU on the output layer too, not just the hidden layers — so any wrongly-negative power prediction is clamped to zero, which is physically sensible for a solar plant.

3.3 Backpropagation

Training needs the gradient of the loss with respect to every weight — millions of slopes. Backpropagation computes them all in one backward sweep: it is the chain rule (multiply slopes through a composition, Module 1) organized efficiently from the loss backwards through the layers, reusing intermediate results so nothing is computed twice.

You will never differentiate a network by hand: frameworks do it automatically from the forward-pass code — TensorFlow/Keras for Paper 1, PyTorch for Paper 2. What you must be able to say: backprop computes gradients; gradient descent then uses them to update weights. Two separate ideas, one loop.

Why differentiability keeps paying Because Paper 2's whole self-model is built from differentiable pieces, gradients can flow not only to the weights during training, but also to the joint angles afterwards — which is how the robot steers its own arm through the model (Module 1's "same hammer, two nails").

3.4 Optimizers: from SGD to Adam

Plain SGD takes each noisy mini-batch gradient at face value. Two upgrades fix its worst habits. Momentum keeps a running average of recent gradients — like a heavy ball rolling downhill, it smooths out the mini-batch noise and powers through small bumps. Adaptive learning rates give each parameter its own effective step size, scaled by how large that parameter's gradients have recently been.

Adam combines both: it tracks an exponentially-decaying average of gradients (the "first moment", decay rate 0.9) and of squared gradients (the "second moment", decay rate 0.999), and uses them to take per-parameter, noise-robust steps.

An exam-worthy justification in Paper 1 Paper 1 trains with Adam at lr 10⁻⁴, decay rates 0.9/0.999 — and then deliberately excludes the learning rate from its manual hyperparameter search: since Adam adapts effective step sizes dynamically, tuning the base rate by hand would buy little at great cost in search time. Being able to reproduce that justification is worth real marks. Paper 2 uses Adam twice: for training the self-model, and later — at lr 0.04 — for gradient-descent control of the arm.

3.5 Regularization and early stopping

Dropout is the classic network regularizer: during training, each hidden unit is randomly switched off with some probability, so the network cannot rely on any single fragile co-adaptation — it is forced to learn redundant, robust features.

Paper 1 tried dropout — and dropped it Paper 1 reports that dropout "delivered no improvement" on this problem, and relies instead on early stopping with patience 20: training halts once the validation loss has failed to improve for 20 consecutive epochs, and the best-so-far weights are kept. Saying "they used dropout" in the exam would be exactly wrong — the honest negative result is the point.

A subtler regularizer: mini-batch size. Smaller batches give noisier gradient estimates, and that noise acts like a regularizer, jiggling the weights out of sharp, overfit minima. Paper 1 found mini-batches of 32 and 64 best, makes mb-size an explicit dimension of its Phase-3 hyperparameter optimisation, and sticks to powers of 2 for GPU memory efficiency.

3.6 Hyperparameters vs parameters

Parameters are the numbers training learns: every entry of every W and b. Hyperparameters are the choices a human makes before training: number of HLs, HUs per layer, mini-batch size, sliding-window length, which input features to use. Parameters are found by gradient descent; hyperparameters by search — training many candidate models and comparing them on the validation set.

Here lies a fairness trap: when comparing architectures (say FFNN vs LSTM vs GRU), spending more hyperparameter-tuning effort on one model biases the comparison in its favour. Paper 1's critique of the literature — and its own scrupulously even-handed 3-phase search framework — exists precisely to avoid this (Modules 7–8).

Final macro model	Hidden layers	Mini-batch	Window	Features
FFNN	3 HLs [64, 64, 64]	64	1 h	power + GHI + temperature
LSTM	3 HLs [16, 16, 16]	32	3 h	power + GHI + temperature
GRU	2 HLs [64, 64]	64	6 h	power + GHI + temperature

Note what the table quietly teaches: the recurrent models earn their keep with longer input windows (3 h, 6 h) and, for the LSTM, far fewer units — architecture and hyperparameters trade off against each other.

3.7 Reading training curves

A training curve plots loss against epochs, usually one line for training loss and one for validation loss. Three shapes to recognize on sight:

Healthy convergence: both curves fall and flatten, validation slightly above training. Stop when flat.
Overfitting: training keeps falling; validation bottoms out and climbs. The best model lived at the validation minimum — early stopping is "quit while you're ahead", and patience (Paper 1: 20 epochs) is how long you wait to be sure the minimum really was the minimum.
Learning rate too high: loss oscillates wildly or diverges from the first epochs — no amount of patience fixes a step size that overshoots.

Examiners love this move A favourite oral-exam gambit: sketch a loss curve on the whiteboard and ask "what is happening here, and what would you do?" Practise narrating each of the three shapes above in two sentences — diagnosis, then remedy.

Exam warm-up — say it out loud

Define hidden layers and hidden units, and explain why Paper 1's hyperparameter search is over exactly these.
Walk through one full training iteration: forward pass, loss, backpropagation, Adam update. Where does the mini-batch enter?
Why did Paper 1 leave the learning rate out of its hyperparameter search, and what did it use instead of dropout?
Training loss falls smoothly; validation loss rose 15 epochs ago. What happened, and what do the next 5 epochs decide under patience-20 early stopping?