Deep Kernel Learning

A standard Gaussian process makes predictions using a kernel — a function that measures how "similar" two inputs are, which then determines how much one input's observed outcome should influence the prediction at a nearby input. The problem is that ordinary kernels (like the common squared-exponential kernel) measure similarity using raw, untransformed input features, which works poorly on complex data like images or high-dimensional financial time series where "closeness" in the raw feature space has little to do with closeness in outcome.

Deep kernel learning fixes this by first passing inputs through a neural network to learn a transformed representation, then applying a standard kernel to that representation instead of the raw inputs. In effect, the network learns which features actually matter and how to combine them, while the Gaussian process on top still produces the same calibrated, distribution-valued predictions — a mean estimate plus honest uncertainty bands — that make Gaussian processes attractive for risk-sensitive applications in the first place.

The network's parameters and the kernel's parameters are trained jointly by maximizing the same marginal likelihood a plain Gaussian process would use, so the feature learning is explicitly guided toward representations that make the kernel's similarity assumption true, rather than trained for some unrelated objective and bolted on afterward. This gives deep kernel learning a genuine advantage over both plain neural networks (which produce point predictions with no native uncertainty) and plain Gaussian processes (which scale poorly and struggle with raw high-dimensional or unstructured inputs) for tasks like estimating a fair price with a confidence interval on an illiquid instrument.