Theory 062/Information & Computation /Distributed systems
Distributed Failure Is a Latency Problem
Outages are blamed on traffic spikes. The spike may only matter when the tail crosses a deadline.
Distributed systems fail at a threshold in tail latency relative to timeout rather than at a threshold in load, so a system at low utilisation can fail if its tail latency crosses the timeout, and a busy one can survive if it is consistent.
01The evidence
Large-scale outages are frequently traced to retry storms and timeout cascades rather than to raw capacity, and tail latency amplification is a known mechanism.
02The tension
Capacity planning uses average utilisation and headroom. That predicts failure from load and misses the latency-triggered cascades that dominate real outages.
03The mechanism
Once a fraction of requests exceed a timeout, clients retry, which adds load, which raises latency further. The feedback is unstable above a threshold in the ratio of tail latency to timeout.
That makes the ratio the instability condition, and predicts failure at low utilisation when the tail is long, which is what retry cascades do.
04What this is not
Those models predict failure from load. This predicts it from a latency ratio, which explains low-load outages.
05Predictions
- P1Failure should occur when tail latency approaches timeout.
- P2Low-utilisation systems with long tails should fail.
- P3Reducing retries should shift the threshold.
06Refutation
Failure tracking utilisation at controlled tail latency would restore capacity models.
07What it changes
It would change how distributed systems are provisioned.