Conditional GANs

An ordinary GAN's generator takes random noise $z$ and produces some sample from the data distribution — you don't get to choose which category it lands on. Train one on a mix of cats and dogs and each call to $G(z)$ might give either, with no control. A conditional GAN (cGAN) fixes this the same way a conditional VAE does: hand a label $y$ to both networks, so the generator learns to produce samples of that label and the discriminator learns to check both realism and whether the sample actually matches the label it was told to be.

The analogy: a custom order versus a grab bag

An ordinary GAN is a grab bag: reach in, get a random item from whatever mix was in there. A conditional GAN is a custom order: you specify "dog" and the vendor (generator) produces a dog, while a quality inspector (discriminator) checks not only "is this a real-looking animal" but also "did they actually deliver a dog, not a cat wearing a dog costume." Both checks matter — a perfectly realistic cat delivered against a dog order should still fail inspection.

The objective, with $y$ threaded through

The cGAN objective is the ordinary minimax objective with $y$ conditioning every term:

$$\min_G \max_D \; \mathbb{E}_{x,y \sim p_{\text{data}}}\big[\log D(x \mid y)\big] + \mathbb{E}_{z, y}\big[\log(1 - D(G(z \mid y) \mid y))\big]$$

In words: the discriminator now scores "is $x$ both real and consistent with label $y$," and the generator, given $z$ and $y$, tries to produce something that clears that bar. Practically, $y$ is concatenated into the generator's input noise and into whatever layer of the discriminator sees the sample, so both networks have access to the same label at every step.

Worked example 1: a discriminator that checks both things

Suppose the discriminator's score decomposes (illustratively) as $D(x\mid y) = D_{\text{real}}(x) \times D_{\text{match}}(x,y)$, both in $[0,1]$. A real, correctly-labeled dog photo: $D_{\text{real}}=0.95$, $D_{\text{match}}=0.9$, giving $D = 0.855$ — high, as it should be. A realistic dog photo mislabeled as "cat": $D_{\text{real}}=0.95$ (still looks real) but $D_{\text{match}}=0.05$ (clearly not a cat), giving $D = 0.0475$ — correctly rejected, even though the image itself is perfectly photorealistic. An unconditional GAN's discriminator has no $D_{\text{match}}$ term at all and would have accepted this sample.

Worked example 2: sampling multiple classes from one model

Train a cGAN on ten digit classes with $y$ one-hot. To generate five 6's and five 9's from a single trained generator: draw ten noise vectors $z_1,\dots,z_{10}$, pair the first five with $y=$one-hot(6) and the rest with $y=$one-hot(9), and decode each pair through the same $G$. Contrast with training ten separate unconditional GANs, one per digit — a cGAN gets all ten classes' worth of shared structure (stroke style, thickness) baked into one model, so a "6" and a "9" sampled with the same underlying $z$ tend to share the same handwriting style, something ten independent models could never guarantee.

Change the strategy above and notice each preset is really the same payoff machinery conditioned on a different choice — the same relationship a cGAN has to its label: one shared network, steered by which condition you hand it.

What this means in practice

Conditional GANs underlie most controllable image generation and are the natural template for finance use cases like generating synthetic order flow conditioned on a venue, or synthetic return paths conditioned on a volatility regime — the same generator can be asked for "calm regime" or "stressed regime" samples rather than needing a separate model per regime.