Sizing Research Compute and Its Cost

A research team either buys too little compute and spends its days waiting for jobs to finish, or buys too much and pays for idle machines every month whether anyone uses them or not. Both mistakes are common because the honest answer — "how much compute do we need?" — depends on details that are easy to skip past: how big is a typical job, how many run at once, and how much slower a researcher is willing to tolerate their work being before they'll complain.

The three numbers that actually determine the answer

The first number is job size: how much memory and CPU time does one typical piece of research work — a backtest, a feature build — actually consume. The second is concurrency: how many researchers, or how many parallel jobs from one researcher, run at the same moment rather than one after another. The third is tolerance for waiting: a job that takes 30 seconds versus 5 minutes changes how someone works, since a 30-second job gets rerun interactively a dozen times while a 5-minute job gets queued and the researcher moves to something else. Multiplying job size by peak concurrency gives the compute you need at the busiest moment; sizing for anything less means the busiest moment turns into a queue.

Cost follows directly from that sizing decision but with an extra wrinkle: idle capacity still costs money if it's owned outright, while capacity rented on demand costs nothing when idle but more per hour when it's used. A team whose usage is bursty — quiet most of the day, then everyone runs backtests right before a weekly review — is usually cheaper on rented, elastic capacity than on owned machines sized for the peak and idle the rest of the time.

Worked example: comparing two sizing choices

A team of five researchers each runs, on average, one job needing 8GB of memory and 10 minutes of CPU time, with all five occasionally running simultaneously right before a Monday meeting. Sizing for the average case — one job at a time — needs only 8GB total, but on Monday mornings the other four researchers queue behind whoever got there first, each waiting up to 40 extra minutes. Sizing for the peak case — five simultaneous jobs — needs 40GB, which sits mostly idle the rest of the week but eliminates the Monday queue entirely. Whether that's worth it comes down to what 40 minutes of a researcher's time each Monday is worth against the cost of the extra 32GB sitting idle the other four days.

What this means in practice

Compute decisions are budget decisions, not just engineering ones, and the sizing conversation only produces a sensible answer once someone attaches a real cost to researcher waiting time and compares it honestly against the cost of the extra hardware. Most teams find that a small buffer above average usage — enough to absorb the routine bursts, not enough to cover every conceivable simultaneous peak — is the practical middle ground, revisited every few months as the team and its typical job sizes grow.