Fully Convolutional Networks

A standard image classifier ends with fully-connected layers that flatten the image's features into a fixed-length vector and output a single label — which forces the input image to be a fixed size, since a fully-connected layer's weights are tied to a specific input dimension, and it throws away all spatial information about where in the image something was found. A fully convolutional network (FCN) replaces those fully-connected layers with more convolutional layers instead, so every layer in the network preserves the spatial grid structure of the image all the way through. The result is a network that accepts an input image of any size and outputs not a single label but a spatial map — for instance, a per-pixel classification, useful for tasks like semantic segmentation where every pixel needs its own label ("is this pixel part of a road, a car, or the sky?").

Because convolutional layers apply the same small filter across every position, an FCN can also be run efficiently over larger images than it was trained on, simply producing a larger output map, without needing to be retrained or resized. This architectural shift — all-convolutional, no fixed-size flattening — became foundational for later segmentation and detection architectures in computer vision.

A useful way to picture the difference: a standard classifier is like asking "what's the single most prominent object somewhere in this photo?" — it flattens everything into one answer and needs the photo cropped to a fixed size to even ask the question. An FCN is like asking "for every square inch of this photo, what's there?" — it can answer that question on a photo of any size, and its answer is itself a picture (a label map the same rough shape as the input) rather than a single word. A common technique in FCNs is to first shrink the spatial resolution through several convolutional and pooling layers (as a normal classifier would, to build up high-level features), then use "transposed" or "upsampling" convolutions to expand the coarse feature map back up to something close to the original image resolution, producing a dense, pixel-by-pixel prediction rather than one label for the whole image.