DCGAN Architecture Guidelines

Early attempts to build GANs out of convolutional layers — the standard building block for image models — were notoriously unstable, often collapsing to producing one or a handful of images regardless of the input noise. The DCGAN paper's contribution wasn't a new loss function or a new type of layer; it was a short list of architectural choices, found by trial and error, that made convolutional GANs train reliably, and the list became the default recipe for years of follow-on work.

The core guidelines: replace any pooling layers with strided convolutions in the discriminator and fractionally-strided (transposed) convolutions in the generator, so every spatial up- or down-sampling step is learned rather than fixed; use batch normalization in both networks (except the generator's output layer and the discriminator's input layer, where it was found to hurt); remove fully-connected hidden layers entirely in favour of all-convolutional architectures; use ReLU activations in the generator for all layers except a $\tanh$ output layer; and use LeakyReLU activations in the discriminator throughout.

None of these individually looks like a breakthrough, but together they addressed the specific failure modes — dead gradients, mode collapse, unstable batch statistics — that had made convolutional GANs unreliable before. Many of these choices (strided convolutions over pooling, LeakyReLU in discriminators, careful batchnorm placement) remained close to standard practice in later GAN architectures, even ones that changed the loss function or added entirely new stabilization tricks like spectral normalization.