Transforming the Target Variable

Feature scaling and transforms get most of the attention, but the variable you're actually trying to predict — the target — can have exactly the same problems, and fixing it can matter even more. If you're predicting something like trade notional, company market cap, or time-to-event, the raw target is often heavily right-skewed: many small values and a few enormous ones. A model trained to minimize squared error on a skewed target will end up dominated by getting the handful of huge values roughly right, at the cost of being systematically imprecise on the much larger population of typical, smaller values.

Why the target's shape matters, not just the features'

Squared-error loss penalizes large absolute errors far more than small ones, so if the target ranges from 10 to 10,000,000, a model minimizing squared error will focus almost all of its effort on the 10,000,000-scale examples, since a modest percentage error there produces a huge absolute error, while being off by 50% on a typical value of 100 barely registers. This is often not what you want — usually a 50% relative error matters similarly regardless of scale. Applying a transform like $\log(y)$ to the target reshapes the problem so the model effectively minimizes something closer to relative error, since a fixed absolute change in log-space corresponds to a fixed percentage change in the original scale.

Worked example: predicting trade notional

Suppose trade notional ranges from $1,000 to $50,000,000 across your dataset, heavily right-skewed. A model trained directly on notional will chase the handful of $10M+ trades and may end up predicting $1,200 for a trade that's actually $1,000 (a 20% relative error, tiny in absolute squared-error terms). Training instead on $\log(\text{notional})$ means the loss treats a prediction off by $\log(1.2) \approx 0.18$ the same whether the true value is $1,000 or $50,000,000 — both a 20% relative miss — which matches the actual business cost far better. After training, predictions must be exponentiated back: $\hat{y} = e^{\hat{z}}$ where $\hat{z}$ is the model's prediction in log-space.

The subtle bug: back-transformation bias

Here is the part that trips people up. If a model is trained to predict $\mathbb{E}[\log y]$ (the expected value of the log-transformed target) and you naively exponentiate that prediction, you do not get an unbiased estimate of $\mathbb{E}[y]$ — because $e^{\mathbb{E}[\log y]} \neq \mathbb{E}[y]$ in general (Jensen's inequality: the exponential function is convex, so this naive back-transform systematically underestimates the true expected value). A standard correction, assuming the log-space residuals are approximately normal with variance $\hat{\sigma}^2$, multiplies the naive back-transform by a smearing factor: $\hat{y} = e^{\hat{z} + \hat{\sigma}^2/2}$. Skipping this correction is a common, quiet source of systematic underprediction whenever a log-transformed target is exponentiated back without adjustment.

The log curve in the explorer above compresses large values far more than small ones — exactly the property that reshapes a skewed target into something closer to symmetric, and exactly why the back-transform (its inverse, exponentiation) is nonlinear and requires a bias correction rather than a plain reversal.

What this means in practice

Whenever a target spans several orders of magnitude or is heavily skewed, consider transforming it before training rather than only transforming the input features — but evaluate the model's error metrics on the original scale after back-transforming, since improvement in log-space error doesn't automatically mean improvement in the units that actually matter for a business decision, and always check whether a back-transform bias correction is needed for your specific loss and error distribution.