Why CNNs Outperform Fully Connected Networks: A Deep Dive into Architecture and Applications

Convolutional Neural Networks (CNNs) are feed‑forward neural networks that use convolution operations and deep architectures to achieve shift‑invariant classification, often described as simulating the human visual system with local perception + parameter sharing.

Comparison with Fully Connected Networks

Parameter count for a 1000×1000 image: fully connected ≈ 10⁶ weights, CNN ≈ 10⁴ weights (≈99 % reduction).

Spatial information: fully connected layers destroy local structure, while CNNs preserve local feature relationships.

Translation invariance: absent in fully connected networks, present in CNNs (objects remain recognizable after translation).

Typical domains: fully connected networks excel at structured data prediction; CNNs dominate image, video, and medical‑image analysis.

Three‑Layer Functionality

Input layer : receives raw grid‑like data, e.g., a 28×28 pixel handwritten digit.

Hidden layers (feature extraction):

Convolution – a 3×3 kernel slides over the input, producing feature maps that capture local patterns.

Pooling – reduces spatial dimensions, lowering parameter count and mitigating over‑fitting.

Activation – introduces non‑linearity to improve generalisation.

Fully connected – aggregates extracted features into a classifier.

Output layer : yields the final prediction, e.g., the digit (0‑9) with highest probability.

Interactive visualisation of convolution execution is available at https://poloclub.github.io/cnn-explainer/.

Practical Applications

Image classification – recognizing objects such as cats, dogs, airplanes, cars.

Object detection – locating faces, vehicles, animals within images.

Semantic segmentation – assigning a class label to each pixel.

Face recognition – verification and retrieval of human faces.

Image generation – style transfer, inpainting, and other generative tasks.

Cold Knowledge

In deep CNNs such as ResNet‑152, only ~15 % of convolution kernels are strongly activated; pruning the inactive kernels can shrink the model by ~90 % with <1 % accuracy loss, enabling mobile deployment.

The policy network of AlphaGo (which defeated Lee Sedol in 2016) was a 13‑layer CNN.

References

卷积神经网络 – CNN? – https://easyai.tech/ai-definition/cnn/