Incoprea hypothesis tests · volume II

Floating-Point Error Has a Phase Transition

Mathematics & Computation · notebook volume 2 · Simulation n = 10,000,000

The claim

Error in large reductions switches from benign to catastrophic at a critical reduction depth rather than growing smoothly, so reproducibility is a threshold property and the threshold shifts predictably with precision and ordering.

What was tested

Error should show a sharp increase beyond a critical reduction depth, and reordering below that depth should be harmless.

Falsified by: Error that grows smoothly across depths, with no threshold, would return the problem to the random-walk account.

Method

Sequential summation in float16, float32 and float64 of N uniform(-1,1) draws, N from 100 to 10^7, twelve seeds, median. Doubles as reference. Error is measured in units of unit-roundoff times the sum of absolute values, so the amplification factor is directly comparable across precisions. Local log-log slope over an eleven-point window tests for a knee. Reordering tested at N = 10^6.

Result

2026-10-30T13:29:58.605795 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ 1 0 2 1 0 3 1 0 4 1 0 5 1 0 6 1 0 7 reduction depth N 1 0 − 1 6 × 1 0 − 2 2 × 1 0 − 1 3 × 1 0 − 1 4 × 1 0 − 1 rounding amplification Error grows smoothly; no depth threshold float32 sequential float16 sequential float64 (reference floor)
Figure generated by code/tests/floating-point-phase-transition.py from the recorded run.
Amplification exponent0.03
Amplification at N = 1000.09
Amplification at N = 10^70.27
Largest local slope jump0.19
Reordering change (rounding units)0.05
not supported

Rounding amplification in float32 is flat at about 0.06 to 0.30 across five decades (fitted slope 0.03): absolute error grows only in proportion to the sum of magnitudes, not superlinearly with depth. The local log-log slope never jumps by more than 0.19 around a median of 0.10. Relative to the answer, error still grows as N^0.5, because a cancelling sum is a random walk of size sqrt(N) while the error is fixed to the magnitudes. Reordering 10^6 terms changes the result by 0.05 rounding units — small, and equally smooth in N. There is no critical depth.

Limits

← Uncertainty Has a FloorDesalination Has a Floor →