Inception and Multi-Scale Convolution Blocks

A standard convolutional layer picks one filter size, say 3x3, and applies it everywhere. But real images have features at different scales in the same picture: a small texture detail and a large object outline might both matter, and no single filter size is best for both. Choosing one filter size per layer forces a compromise the network has to live with for that whole layer.

An Inception block (introduced in GoogLeNet) sidesteps the choice by running several filter sizes, typically 1x1, 3x3, and 5x5 convolutions, plus a pooling operation, in parallel on the same input, then concatenating all their outputs together before passing them to the next layer. The network sees the same patch of the image through several different-sized "lenses" simultaneously and learns, through later layers' weights, how much to rely on each scale's output.

The obvious problem is cost: running a 5x5 convolution on every channel of a deep feature map is expensive, and doing it several times per block compounds fast. The practical fix is a 1x1 convolution inserted just before each larger filter, purely to shrink the number of channels first, cutting the computation of the subsequent 3x3 or 5x5 pass substantially before it happens, without changing the spatial resolution being examined.

Stacking many Inception blocks lets a network build up multi-scale features layer after layer rather than committing to one receptive field size architecture-wide, which was one of the ideas that let GoogLeNet reach high accuracy on ImageNet with a comparatively modest parameter count relative to contemporaries of similar depth.