LLM Evaluation and Benchmarking

"This model seems pretty good" is not an evaluation — it's an anecdote from whatever handful of prompts someone happened to try. A model that answers three cherry-picked test questions well can still fail badly and consistently on a broad category of realistic questions it wasn't tried on. LLM evaluation replaces that anecdote with a structured, repeatable measurement: a fixed test set, a defined scoring method, and a number that can be compared across models or across versions of the same model over time.

The analogy: grading an exam versus grading a class discussion

A teacher forming an opinion from a lively class discussion might walk away impressed by the two most articulate students and miss that half the class didn't understand the material. A standardized exam, graded the same way for every student against a fixed answer key, gives a genuinely comparable measurement — even if it's a narrower slice of what "understanding" really means. LLM evaluation is the deliberate move from impressionistic class-discussion judgments toward exam-style, repeatable measurement, with all the same tradeoffs a standardized test has: consistent and comparable, but only as good as the questions on it.

Three ways scoring actually happens

• Reference-based automatic metrics. For tasks with a single correct short answer (exact multiple-choice questions, extracted numeric facts), scoring is exact-match or close to it — fast and cheap, but only works when there's genuinely one right answer to check against.
• Model-graded evaluation. For open-ended tasks (summarizing a transcript, answering a nuanced question), a second, usually stronger, language model is prompted to judge the quality of the first model's output against a rubric — cheaper and faster than human grading, but inherits the judge model's own blind spots and biases.
• Human evaluation. Human raters score or compare outputs directly — the most reliable ground truth for judgment calls like tone, helpfulness, or subtle factual errors, but slow and expensive to run at scale, which is why it's typically reserved for final validation rather than routine iteration.

In plain English: the cheaper the evaluation method, the more it risks measuring "sounds right" rather than "is right," so the choice of evaluation method is itself a judgment call about what's actually being tested.

Worked example: computing an exact-match benchmark score

A benchmark of 500 financial-arithmetic word problems, each with one exact correct numeric answer, is run against a model. It answers 431 correctly. The benchmark score is

$$\text{accuracy} = \frac{431}{500} = 86.2\% .$$

This number is comparable across model versions run on the exact same 500 questions — but says nothing directly about the model's performance on financial questions that aren't purely arithmetic, like summarizing tone or judging materiality, which the benchmark never tested.

Worked example: model-graded evaluation with a rubric

An open-ended task — "summarize this earnings call in three sentences" — is scored by a stronger judge model given a rubric: 1 point each for accuracy, conciseness, and coverage of the call's key numbers, for a maximum of 3. Across 100 summaries, the model under test averages 2.3/3 according to the judge model. This score is useful for comparing versions of the summarizing model against each other under a fixed judge and rubric, but a low inter-rater agreement between the judge model and separately sampled human raters on the same 100 summaries would be a warning sign that the automated score isn't tracking what a human actually cares about.

What this means in practice

Any claim that one LLM is "better" than another for a financial task should specify what benchmark, what scoring method, and what test set that claim rests on — the same rigor expected of any other empirical result in quant work. Benchmarks are also public and finite, and once a benchmark is well known, there's a real risk that models are trained (deliberately or through data contamination) on text that overlaps with the benchmark itself, inflating the score without a genuine improvement in the underlying capability — the same leaderboard-overfitting problem that shows up in any widely reused test set.