Reliability Functions and System Redundancy
How the probability that a whole system keeps working depends on whether its components are wired in series (any one failure kills the system) or in parallel (all must fail together) — and why adding redundant backups is one of the few ways to buy reliability cheaply.
Prerequisites: The Law of Large Numbers
A trading system depends on a chain of infrastructure — a network link, a database, a matching-engine connection, a risk-check service — and each piece can fail independently with some probability. Does the whole system's reliability just add up these individual failure chances, or is there something more structural going on? It turns out the answer depends entirely on how the pieces are wired together: whether the system needs every component to work (series), or only needs at least one of several redundant copies to work (parallel). The two wiring patterns produce wildly different reliability, even from identical individual components, and understanding both is the entire basis for deciding where redundancy is worth paying for.
An analogy: a chain versus a rope of many strands
A chain is only as strong as its weakest link — if any single link breaks, the whole chain fails, no matter how strong the other links are. This is a series system: reliability degrades with every additional link, because there are more chances for any one of them to be the failure. A climbing rope made of many independent strands twisted together is the opposite — it only fails if every single strand snaps, since the rope holds as long as at least one strand is intact. This is a parallel system: reliability improves with every additional strand, because now there are more chances for at least one to survive. Real systems are built (deliberately or not) as mixtures of both patterns, and identifying which is which is the first step to improving reliability.
The idea, one symbol at a time
Let each component have reliability — the probability it functions correctly over the period in question — and assume components fail independently. For a series system of components (the whole system works only if every component works):
In plain English: multiply the individual reliabilities together, and since each , this product only ever gets smaller as you add more components — every additional link in a chain can only hurt, never help. For a parallel system of components (the whole system works if at least one component works), it's easier to compute the complement — the probability all fail:
In plain English: the system fails only if every single redundant copy fails, so system failure probability is the product of individual failure probabilities, and system reliability is one minus that — this only ever gets larger as you add more redundant components, since more independent copies means a shrinking chance they all fail together.
Worked example 1: a series pipeline of four services
A trading pipeline needs its network link (99.9% reliable), database (99.5%), matching-engine connection (99.8%), and risk-check service (99.7%) to all work — a pure series system:
Even though every individual component looks excellent (all above 99.5%), the combined system reliability drops to about 98.9% — a failure rate of roughly 1.1%, more than double any single component's own failure rate, purely from chaining four "almost certain" pieces together.
Worked example 2: adding redundancy to the weakest link
Suppose the database (99.5% reliable, the weakest link above) is duplicated with an independent standby that's also 99.5% reliable, and the system now needs only one of the two databases to work (parallel within that sub-system):
The database sub-system's reliability jumps from 99.5% to about 99.9975%. Recomputing the whole series system with this upgraded database:
Overall system reliability rises from 98.9% to about 99.4% — cutting the system's overall failure rate roughly in half, just from duplicating the single weakest component, without touching anything else.
What this means in practice
Reliability engineering for trading infrastructure follows directly from these two formulas: identify which parts of the system are wired in series (where reliability is dragged down by every added dependency, so minimizing the number of series dependencies matters) versus which parts could be made parallel (where adding redundant, independent copies of the weakest components buys reliability cheaply). This is the quantitative reasoning behind decisions like dual network paths, standby databases, and redundant exchange connections — and it clarifies that redundancy is worth the most exactly at the weakest link in a series chain, not spread evenly across every component.
A series system's reliability is the product of its components' reliabilities and can only fall as components are added, making it dominated by its weakest link; a parallel system's reliability is one minus the product of the components' failure probabilities and rises as redundant components are added. Combining the two — finding the weakest link in a series chain and duplicating it in parallel — is usually the cheapest way to raise overall system reliability.
Both formulas assume component failures are independent. In practice, redundant components often share a common failure cause — the same power supply, the same data center, the same software bug — which breaks the independence assumption the parallel formula relies on. A "redundant" pair of databases that both go down together in a data-center outage delivers none of the reliability gain the formula promises; always check that backup components fail for genuinely different reasons before crediting them with the full parallel-system reliability boost.
Related concepts
Practice in interviews
Further reading
- Barlow & Proschan, Statistical Theory of Reliability and Life Testing