Validating Synthetic Market Data Realism

A generative model that produces synthetic price paths can be trained to convergence and still be useless — a GAN's discriminator loss looking good only means the discriminator, at the point training stopped, couldn't tell real from fake on the specific comparisons it happened to check. It says nothing about whether the generated paths actually reproduce the properties a risk manager cares about: fat tails, volatility clustering, realistic cross-asset correlation. Validating realism means checking those properties directly and explicitly, rather than trusting the training loss as a proxy for quality.

An analogy: a forger's test isn't a collector's test

A forger who fools one specific expert on one specific painting hasn't proven their work is a genuinely good match for the original artist's style — a real evaluation would check brushwork, pigment chemistry, and canvas aging against known genuine examples, one property at a time. A GAN's discriminator is like that one expert: it's a useful training signal, but "fooled the discriminator" and "statistically matches real markets" are different claims, and only the second one is what a risk manager actually needs.

The checklist: stylized facts to check

Real financial returns have well-documented "stylized facts" that synthetic data should reproduce. Fat tails: check that the kurtosis of synthetic returns is elevated (well above 3, the normal distribution's value) and roughly matches the real data's kurtosis, not a thin-tailed approximation. Volatility clustering: check that squared or absolute returns show significant autocorrelation over the following several days (large moves followed by more large moves) — testable with an ARCH-LM style test on the synthetic series, the same test used to check for this property in real return series. Correlation structure: for multi-asset synthetic data, check that pairwise correlations across the generated paths land close to the real historical correlations, not collapsed toward zero or toward one. Each of these is a specific, checkable number, not a vague impression.

Worked example: two candidate generators, one clear winner

Two GANs are trained to generate synthetic daily returns for a single equity index, using different architectures. Generator A's outputs have a kurtosis of 3.1 (essentially normal, no fat tails) and no significant autocorrelation in squared returns (no volatility clustering) — it produces paths that are smooth and plausible-looking at a glance but statistically nothing like real markets, having apparently learned to minimize the discriminator's confusion by producing safely "average" output. Generator B's outputs have a kurtosis of 6.8 (compared to the real data's 7.2 — a close match) and significant, decaying autocorrelation in squared returns over roughly ten days (close to the real data's pattern) — a much closer statistical match to actual market behavior. A stress test run on Generator A's paths would systematically understate tail risk; the same test run on Generator B's paths would give a far more trustworthy picture, which is the entire point of doing this validation before using either generator's output for anything decision-relevant.

What this means in practice

No synthetic dataset should be used for a stress test, a risk model, or an augmented training set without first running this validation checklist against known real-data properties. It's a cheap step relative to the cost of the generative model itself, and skipping it risks the worst kind of failure — one where the synthetic data looks fine on inspection but silently understates exactly the tail risk it was meant to help capture.