ETL vs ELT Pipelines

Every data pipeline does three things: extract data from a source, transform it into a usable shape, and load it somewhere it can be queried. The only real question is what order those three steps happen in, and that order has meaningful consequences. ETL — extract, transform, load — cleans and reshapes the data before it ever touches permanent storage, so what lands in the warehouse is already the finished, analysis-ready version. ELT — extract, load, transform — dumps the raw data into storage first, exactly as it arrived from the source, and does the cleaning and reshaping afterward, as a separate step that reads from storage and writes back a transformed copy.

ETL's appeal is that storage only ever holds clean, consistent data — every table has a known shape, and anyone querying it doesn't need to worry about malformed or duplicate records, because those were filtered out before loading. Its cost is that the raw, original data is gone once transformed; if a transformation step turns out to have been wrong — a bad currency conversion, a bug in a rounding rule — there's no way to recompute it correctly without going back to the original source, which may no longer be available in the same form. For a quant pipeline, this is a real risk: today's transformation logic might be improved in a year, and ETL means you can't reprocess history with the better logic unless you kept the raw data somewhere else too.

ELT flips this trade-off. Because raw data is loaded untouched first, it's always possible to redo a transformation later with corrected logic, replay history under a new methodology, or debug a pipeline bug by comparing the transformed output against the untouched original. The cost is that storage now holds messier, inconsistent raw data that anyone querying directly has to handle carefully, and transformation logic has to be re-run (and re-validated) rather than trusted as already-applied. ELT became far more practical once storage got cheap enough that keeping a full untouched copy of everything, in addition to the transformed version, stopped being a meaningful cost.

Worked example: fixing a bad adjustment factor

A pipeline transforms raw price data by applying corporate action adjustments (splits, dividends) before storing it. Two years later, someone discovers the adjustment logic mishandled a rare edge case — a special dividend — for about a dozen historical events, quietly distorting some backtests. Under ETL, fixing this requires re-extracting from the original vendor source, which may no longer offer that exact historical snapshot, or may have since revised its own data. Under ELT, the raw unadjusted prices were preserved in storage the whole time; fixing the bug means correcting the transformation logic and simply re-running it against the raw data already sitting there, producing a corrected adjusted dataset without touching the original source at all.

What this means in practice

Most modern quant data pipelines lean ELT for exactly this reason: research and backtesting logic evolves, and being able to replay it against unchanged raw history is worth far more than the convenience of storage always being pre-cleaned. ETL still makes sense for high-volume, low-value operational data where reprocessing history is never going to matter and storing two copies is wasteful. The decision comes down to one question: will you ever need to redo the transformation with different logic? If yes, keep the raw data.