Modelling Market Response to Agent Orders

The simplest possible trading simulator replays historical price data and assumes the agent's own orders never affect it — a price-taker buying and selling against a market that behaves exactly as it did historically. That assumption is fine for a tiny order relative to daily volume, but breaks down fast for anything larger: a real order consumes liquidity, moves the price against the trader who submitted it, and can change how other participants behave in response.

Modelling market response to agent orders means the simulator's price and liquidity dynamics are made to depend, at least partly, on the agent's own actions — not just replaying a fixed historical path as if the agent were invisible.

Levels of realism, roughly in increasing order of fidelity

• No impact (price-taker). Historical replay, agent trades have zero effect on subsequent prices. Simplest to build, badly wrong for any order of meaningful size.
• Parametric impact models. A simple functional form — often a linear or square-root relationship between order size (relative to average volume) and price impact — is bolted onto the historical replay, giving the agent some feedback that trading moves the price, without modeling the mechanism in detail.
• Order-book-level simulation. The simulator actually models the limit order book's depth and how the agent's order consumes it, giving impact that emerges naturally from liquidity rather than a fitted formula.
• Multi-agent simulation. Other simulated participants (market makers, momentum traders) react to the agent's orders and to each other, producing impact — and second-order effects like other algorithms detecting and trading against a visible pattern — that a single-agent model can't produce at all (see Multi-Agent Reinforcement Learning).

Higher fidelity is more realistic but also slower to simulate and harder to validate — a desk has to weigh how much impact realism actually changes the learned policy against the added complexity.

Worked example

An agent trained in a zero-impact, historical-replay simulator learns to execute its full 30,000-share order in a single aggressive burst at the day's historically tightest spread, since doing so costs nothing extra regardless of size. Retrained in a simulator using a simple square-root impact model — impact $\propto \sqrt{\text{order size} / \text{average daily volume}}$ — the same burst strategy now shows a simulated cost of 19 basis points, because the model correctly penalizes concentrating a large fraction of the day's volume into one moment. The retrained agent instead learns to split the order into roughly ten smaller child orders across the session, cutting simulated impact cost to 8 basis points, since the square-root relationship means impact grows sublinearly with size — splitting a large order into pieces reduces total impact cost even before any recovery between orders.

What this means in practice

Any RL agent trained without some model of its own footprint has effectively been trained in a fantasy world, and the gap between simulated and live performance grows directly with order size relative to available liquidity. A pragmatic starting point is a well-calibrated parametric impact model (square-root is a common default) layered onto historical replay — full order-book or multi-agent simulation is usually reserved for cases where the parametric approximation has been shown, through live monitoring, to be materially wrong.