Episode Definition and Horizon Choice
Before training a reinforcement-learning trading agent, someone has to decide where one episode starts and ends and how far ahead it looks, and that choice shapes every behaviour the agent can learn.
A reinforcement-learning agent learns by interacting across episodes — self-contained sequences of states, actions, and rewards that end and then restart. In trading applications this raises a design question that doesn't arise in games with a natural end state (chess ends at checkmate): where does one episode of market interaction start and stop, and how far into the future does the agent's reward horizon extend?
A common choice is to define an episode as one trading day, or one order's full execution from arrival to completion. The horizon — how many steps ahead the agent optimizes for — then has to match that: an agent rewarded only on the final price at the end of the day will behave very differently from one rewarded on the average price achieved throughout, because the first can afford to wait for a good moment near the close while the second cannot.
Episode boundaries and horizon length are design choices, not natural facts about the market, and they directly determine what behaviour is rewarded — a short horizon teaches urgency, a long one teaches patience, and getting this mismatched to the actual trading goal produces an agent optimizing for the wrong thing.
Worked example
An agent tasked with executing a large buy order is trained with episodes lasting exactly one hour, rewarded on the average execution price over that hour. It learns to spread its buying evenly across the full hour to minimize market impact. If the real trading desk actually needs the order filled within 15 minutes, this agent's learned patience is a mismatch — the episode definition trained it for a problem the desk doesn't actually have, and it needs to be retrained with a 15-minute horizon before deployment.