Convolutional Neural Networks

Site Admin · 11 Sep 2026 · 8 views

Convolutional Neural Networks (CNNs)

Fully connected layers look at images as long flat lists of pixels and treat each pixel independently, which is wasteful and weak. Convolutions exploit locality: patterns in images are local, so a small window scanned over the image can learn edges, textures, and shapes.

The Convolution Operation

A kernel is a small matrix of learnable weights, usually 3 by 3 or 5 by 5, slid over the image. At each position it multiplies local pixels by the kernel and produces one output value. Striding controls how far the window jumps; padding controls what happens at the borders.

Channels and Feature Maps

Color images have three channels; convolutional layers typically emit many more, with each channel acting as a detector for one pattern. Early layers find edges and gradients. Later layers combine them into corners, then shapes, then object parts. This hierarchy is learned, not programmed.

A Tiny CNN in PyTorch

import torch.nn as nn
import torch

class SmallCNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 16, kernel_size=3)
        self.pool = nn.MaxPool2d(2)
        self.fc = nn.Linear(16 * 13 * 13, 10)

    def forward(self, x):
        x = self.pool(torch.relu(self.conv1(x)))
        return self.fc(x.view(x.size(0), -1))

Here conv1 turns the single input channel into 16 feature maps, pooling halves each map, and the flattening view feeds the classifier. The final size, 16 times 13 times 13, falls out of the pooling math.

Pooling and Downsampling

Max pooling keeps the largest value in a window, which adds a little invariance to small shifts and aggressively cuts the compute. Pooling is how a detector built for the top-left corner still recognizes a shape centered elsewhere.

Modern Building Blocks

Modern networks are stacks of convolution, activation, normalization, and pooling, repeated at growing channel counts, often with residual shortcuts that let gradients flow through depth. Every architecture is a variant of this recipe.