Pool or stride?.

Chapter 3 introduced downsampling: making a map smaller so that each value summarizes a bigger patch and later layers have less to compute. There are now two ways to do it:

1. Convolve, then pool: a convolution at stride 1, usually with same padding, followed by 2×2 max pooling at stride 2.
2. Strided convolution: one convolution at stride 2.

Both halve the width and the height. A 64×64 map comes out 32×32 either way.

Output size

Pooling uses chapter 3’s formula with no dilation. Pooling layers usually have no padding either, so $p = 0$ and $d = 1$, and the formula simplifies to

Check it against figure 5.4. Set the image to 32, the window to 3 and the stride to 2: ⌊(32 − 3) / 2⌋ + 1 = ⌊14.5⌋ + 1 = 15.

Which to use

The two approaches trade off what is learned against what it costs:

• Pooling has no weights. Its rule is fixed, so it can’t be trained to keep the wrong thing. But it also can’t be trained to keep the right thing.
• A strided convolution learns its summary. Its weights are trained like any others, so it decides what to keep from each patch.
• A strided convolution is cheaper. Convolve-then-pool computes a dot product at every position and then throws three quarters of them away. A stride-2 convolution only computes the positions it keeps, a quarter as many.

Older networks like LeNet, AlexNet and VGG downsample by pooling. ResNet uses one max pooling layer near the input and strided convolutions after that. Many newer designs drop pooling entirely. Chapter 10 follows that history.

In PyTorch:

import torch.nn as nn

# 64×64 → 64×64 → 32×32: convolve, then pool
nn.Sequential(
    nn.Conv2d(1, 1, kernel_size=3, padding=1),
    nn.MaxPool2d(2),            # stride defaults to the window size, here 2
)

# 64×64 → 32×32 in one layer
nn.Conv2d(1, 1, kernel_size=3, stride=2, padding=1)

# Overlapping pooling, as in AlexNet: 13×13 → 6×6
nn.MaxPool2d(kernel_size=3, stride=2)

nn.AvgPool2d(2)                 # average pooling, same arguments