Compliance and Record-Keeping for LLM Use

Regulated financial firms have long-standing rules requiring that communications relevant to the business — emails, chat messages, research notes — be retained and retrievable for years, precisely because regulators need to be able to reconstruct what happened after the fact if something goes wrong. Large language models complicate this in an obvious way once they start drafting research summaries, answering client questions, or assisting compliance reviews: an LLM's output is a form of business communication, and if it isn't logged with the same rigor as an email, the firm has a gap in its record-keeping obligations that didn't exist before the model was introduced.

Doing this properly means logging more than just the final text a model produced. A defensible record needs the exact prompt sent to the model, the exact response received, the model version and any system instructions in effect at the time (since a model's behavior can change between versions), and often the identity of whichever employee triggered the request. Without the model version specifically, a firm can't answer a regulator's question like "why did the tool suggest this" months later, because the underlying model may have already been updated or retired, and its behavior at the time is no longer reproducible from the current model alone.

A practical example: a firm using an LLM to draft first-pass responses to client questions about account statements needs to retain not just the final response sent to the client, but the prompt and model version that generated it, exactly as it would retain an employee's draft email and any templates used to produce it — the same retention rule, applied to a new kind of author.

What this means in practice

Any firm deploying LLMs in a regulated workflow needs logging infrastructure built for this from day one, not bolted on after an incident — retrofitting record-keeping onto a system that wasn't designed to capture prompts and versions is far harder than building it in from the start. This is also one of the reasons firms are often cautious about using third-party hosted models for regulated tasks, since guaranteeing the exact model version and prompt history is retrievable is harder when the model itself is hosted and updated by an outside vendor.