Hallucination Controls for Numeric Answers

Ask an LLM "what was the company's operating margin last quarter?" and it may answer "18.4%" with exactly the same fluent confidence whether that number came from actually reading the filing or from a plausible-sounding guess extrapolated from general knowledge about similar companies. A wrong sentence in a summary is often obviously wrong on inspection; a wrong number embedded in an otherwise fluent, well-formatted answer usually isn't, because a hallucinated 18.4% looks structurally identical to a correctly retrieved 18.4% — there's no stylistic tell that distinguishes fabricated precision from real precision. This is why numeric answers need controls that text summaries generally don't: the failure is invisible until someone checks the source.

Techniques that reduce numeric hallucination

Extraction, not generation: instead of asking the model to state a number from memory or from a vague read of the surrounding text, have it locate and quote the exact sentence or table cell containing the figure, and only then parse the number out of that quoted span. A number extracted from a specific quoted source is verifiable in a way that a number generated in free text is not.

Forced citation: every numeric claim must come with a pointer back to its source passage (a page number, a table reference, a quoted sentence), so a human — or an automated checker — can confirm the number actually appears there rather than trusting the model's word for it.

Cross-checking against a second source or a calculation: where possible, verify a number two independent ways — for instance, checking that a stated operating margin is consistent with separately extracted revenue and operating income figures ($\text{margin} = \text{operating income} / \text{revenue}$) rather than trusting a single extracted "operating margin" field in isolation.

Explicit abstention: the model should be instructed, and ideally tested, to say "not stated in the provided document" when a number genuinely isn't present, rather than filling the gap with a plausible estimate — since a system that never says "I don't know" is a system that will eventually hallucinate exactly when the true answer is absent.

Worked example: catching a hallucinated margin figure

A pipeline is asked for a company's gross margin from its latest 10-K. Without controls, the model might respond "gross margin was approximately 42%" — plausible for the sector, fluent, and entirely fabricated if the filing never states a gross margin figure directly (many filings report revenue and cost of goods sold but not a precomputed margin percentage). With extraction-plus-cross-check controls, the pipeline instead extracts revenue ($820 million) and cost of goods sold ($495 million) as two separately quoted figures, computes $\frac{820 - 495}{820} \approx 0.396$, and reports "gross margin: approximately 39.6%, computed from revenue of $820 million and cost of goods sold of $495 million (both quoted from page 34)" — a number a reader can verify in thirty seconds against the cited page, rather than a number that sounds just as confident but has no such trail.

What this means in practice

Any pipeline reporting numbers from financial documents should default to extraction-and-citation over free generation for every number, treating "the model stated a figure" as meaningless without a traceable source, and should be evaluated specifically on how often it correctly abstains when a figure genuinely isn't present — a metric most eval sets skip because it requires deliberately including documents where the answer is "not disclosed."