RealNVP and Coupling Layers

A normalizing flow generates data by learning an invertible transformation between simple noise (like a standard Gaussian) and real data, which has an advantage GANs and diffusion models lack: because the transformation is exactly invertible, you can compute the exact likelihood of any real data point under the model, not just generate plausible-looking samples. The catch is that computing a likelihood this way requires the Jacobian determinant of the transformation, and for an arbitrary invertible neural network that determinant is generally far too expensive to compute.

RealNVP's fix is the coupling layer: split the input vector into two halves, leave the first half completely unchanged, and transform the second half using a scale-and-shift function computed from the first half. Because the first half passes through untouched, the transformation is trivially invertible — given the output, the first half is already known, and it can be plugged back into the same scale-and-shift function to undo the second half exactly. More usefully, this structure makes the Jacobian triangular, so its determinant is just the product of the diagonal terms, which is cheap to compute even when the scale-and-shift function inside is an arbitrarily complicated neural network. Stacking many coupling layers, alternating which half is held fixed each time, lets the model build up a highly flexible overall transformation while every individual layer stays cheaply invertible.

This exact-likelihood property makes flow-based models attractive wherever you need a genuine probability density rather than just samples — for instance, using the model's likelihood itself as an anomaly score for unusual market data.