Understanding Neural Networks: Structure, Layers, and Activation

The article introduces a program that can accurately recognize handwritten digits 0‑9, emphasizing the need for image preprocessing to center and size the digits before feeding them to a neural network.

Although modern networks perform better, the presented simple network is easy to understand and can be trained on a personal computer without extensive background knowledge.

Human vision effortlessly identifies digits despite varied handwriting; neural networks aim to mimic this ability by learning from many labeled examples rather than following explicit programming rules.

Neurons are described as units that store a value between 0.0 and 1.0, called the activation value. An entire network consists of many interconnected neurons.

The input layer contains 784 neurons, one for each pixel of a 28×28 grayscale image, with each pixel’s brightness represented as a value between 0.0 (black) and 1.0 (white).

The output layer has 10 neurons, each representing a possible digit (0‑9); their activation values indicate the network’s confidence that the input image corresponds to that digit.

Hidden layers (e.g., two layers with 16 neurons each) sit between input and output, allowing the network to decompose the recognition task into smaller sub‑problems such as detecting edges, rings, or line segments.

Each connection between neurons carries a weight, a numeric parameter that determines how strongly the activation of one neuron influences another. Positive weights increase activation, while negative weights decrease it.

Biases are additional scalar values added to the weighted sum before applying a non‑linear activation function, allowing a neuron to remain inactive unless the summed input exceeds a threshold.

The weighted sum of inputs is passed through a sigmoid function (σ), which compresses any real number into the range (0, 1), providing a smooth transition from inactive to active states.

All weights from one layer to the next can be organized into a weight matrix, and biases into a bias vector. The activation of the next layer is then computed efficiently as σ(W·a + b) , where a is the activation vector of the current layer.

This compact matrix‑vector formulation reduces code complexity and leverages optimized linear‑algebra libraries.

Overall, the network is a large, parameterized function that maps 784 input numbers to 10 output numbers, involving over 13,000 weights and biases, and will be trained in the next lesson to adjust these parameters for accurate digit recognition.