Does averaging have a floor?
Supplied working report; original verdicts not validated.
Review status: Needs review. This is an archival source note, not a passed proof.
README.md
# Does averaging have a floor? **Test of hypothesis 158, Volume XII — "Averaging has a floor"** · run 30 October 2026 Published proposal: <https://incoprea.com/research-library/original-theories-vol12-20261030/> ## The result in one paragraph No. Over 30 years of BIPM Circular T the Allan deviation of `[UTC-UTC(k)]` keeps falling for most laboratories: **63 of the 90** that qualify (70.0%) reach their smallest value at the longest averaging time the record allows and never turn up (Figure 1). Another 27 (30.0%) do turn up, but usually as a shallow bend — only **5** rise by as much as a factor of two — and where a turn occurs it sits between one and seven years of averaging, not at one characteristic time (Figure 2). The refutation criterion fixed in advance, "sigma(tau) falls monotonically to the longest available tau for most laboratories", is met. What survives is narrower: among the ten laboratories where a post-minimum slope can be fitted at all, all ten are positive (median +0.85), and the *ordering* of the minimum across laboratories tracks the drift time predicted from each clock's own drift rate (Spearman rho = 0.74, p = 0.0002, n = 24) — but that predicted time is on average 6,800 times longer than the observed one and, for all 24, longer than the record (Figure 3). Drift orders the minima; it does not set them. ## Design Claim verbatim, predictions and every criterion are in `hypothesis.md`, fixed before the first number was computed: **P1** the deviation falls, reaches a minimum, then rises; **P2** the minimum sits at a drift-related averaging time; **P3** the post-minimum slope is positive. Data: every BIPM Circular T issue from 100 (1996 March) to 465 (2026 September), 366 issues fetched sequentially, parsed, reduced to one series table; raw text was hashed and discarded. That is 166,767 distinct `(laboratory, epoch)` values across 117 laboratory codes, of which 90 have the required 500 five-day points. Estimator: the overlapping Allan deviation of phase data, `sigma_y^2(tau) = (1/2 n tau^2) * sum_k [x(t_k+2tau) - 2x(t_k+tau) + x(t_k)]^2`, over every triple present in the record, on a log-spaced grid from 5 to 3,000 days. `sigma_y` is fractional frequency. Second differences cancel a constant frequency offset, so the estimator is blind to a linear trend in phase by construction; S1 demonstrates that rather than assuming it. **Parse check.** 359 of 365 consecutive issue pairs share an epoch, giving 23,111 values quoted twice. 22,318 agree exactly — **96.57%** — and 22,322 within 0.1 ns. All **793** disagreements sit at the boundary epoch that BIPM revises when later data arrive (median revision 1 ns, 99th percentile 524 ns, largest 1,345 ns); the later value is kept. ## P1 — refuted for the majority | Outcome | Laboratories | |---|---| | interior minimum (turns up before the end) | 27 (30.0%) | | falls to the longest available tau | 63 (70.0%) | | interior minimum rising at least 2× | 5 (5.6%) | Median `tau_min` 1,915 days (IQR 1,305–2,320; range 340–2,320), median `sigma_min` 3.5 × 10⁻¹⁵. For the 63 falling curves the median log-log slope over the whole curve is −0.65 (IQR −0.78 to −0.54): averaging keeps paying, though more slowly than the τ⁻¹ of a perfectly bounded phase. NIST, USNO, PTB, OP and SU all fall to about 2,800 days; four of the five have no interior minimum, and NIST's ends only 4.7% above its minimum. Their minima are lower than everyone else's (Mann–Whitney p = 0.032 on `sigma_min`) but occur at the same late times (p = 0.15 on `tau_min`). ## P2 — the ordering survives, the level does not For white frequency noise plus linear drift `D`, `sigma_y^2 = a/tau + D^2 tau^2/2` gives a minimum at `tau_opt = (a/D^2)^(1/3)`. With `D` fitted per series and `a` from the shortest tau, the 24 laboratories with well-determined drift show a strong rank association (rho = 0.74, permutation p = 0.00015) but a median observed:predicted ratio of 1 : 6,800; none is within a factor of three and all 24 predictions exceed the record. A minimum "set by" the drift time would sit on the identity line in Figure 3. None does. ## P3 — positive, unresolved All ten laboratories with enough points past the minimum to fit a slope have a positive one (median +0.85), as drift would produce. None has a 95% CI excluding zero: with a handful of correlated points past the minimum and `df_effective = span/tau_min`, every interval spans zero, the closest being MTC at p = 0.059. The pre-specified support criterion — at least half of all included laboratories showing an interior minimum, a positive slope and a CI excluding zero — is met by **0 of 90** (exact binomial p = 1.6 × 10⁻²⁷), and no test in the 10-member Benjamini–Hochberg family (the per-laboratory post-minimum slopes, the only declared family) survives. ## Sensitivity - **S1** linear phase trend removed: identical, 27 of 90. The estimator never saw it. - **S2** quadratic trend removed (drift removed): minima 27 → 20, falling 63 → 70. Some minima are drift; most are not. - **S3** first-difference estimator: 8 of 90. **S4** rise ≥ 2×: 5 of 90. - **S5** sub-nanosecond era only (issues 184–465; the 84 earlier issues print whole nanoseconds): 25 of 84 interior (29.8%), 59 falling — the verdict does not depend on the 1 ns quantisation of 1996–2003. - **S6** classification on a three-point median of log σ, which suppresses a single noisy end point: identical, 27 of 90. ## Limits The test measures the *steered* residual `[UTC-UTC(k)]`, not a free-running clock. A laboratory steering towards UTC keeps its phase bounded, and a bounded phase makes the Allan deviation fall roughly as τ⁻¹ however bad the underlying oscillator is. That is why 70% of curves fall, with slopes near −0.65: the record shows the control loop more than the physics of averaging, and cannot settle the claim for a free-running clock. Nothing beyond about 2,800 days is observable, so a floor further out is invisible. Step changes are endemic (73 of 90 series contain a five-day change above 100 ns) and inflate long-τ deviation in a way that mimics a rising floor without any smooth drift. ## What would change the answer Free-running data — `TAI-TA(k)` before steering, or a laboratory's own frequency measurements — showing minima *at* the drift-predicted time would convert P2's rank association into the level agreement it lacks. Removing the controller is the decisive experiment. `run.py` rebuilds every table and figure from the cached parse, and `--refetch` re-downloads all 366 issues; `SOURCE.md` records every URL and hash. *Designed and executed with AI assistance (DeepSeek, via its coding agent); not peer reviewed. Observational data: associations only, no causal claim.*
hypothesis.md
# The hypothesis, as published
Source: <https://incoprea.com/research-library/original-theories-vol12-20261030/>
Volume XII, hypothesis 158 — "Averaging has a floor".
## Claim (verbatim)
> The uncertainty of a measurement averaged over time reaches a minimum set by
> the ratio of averaging time to drift time instead of falling indefinitely.
## Predictions (verbatim, from the brief)
- **P1** — the Allan deviation of a clock comparison falls, reaches a minimum, then rises.
- **P2** — the minimum sits at a drift-related averaging time.
- **P3** — the post-minimum slope is positive.
## Refutation criterion (verbatim, from the brief)
> refute if sigma(tau) falls monotonically to the longest available tau for most
> laboratories.
Support criterion (verbatim, from the brief):
> support if at least half the laboratories show an interior minimum with a
> positive post-minimum slope whose CI excludes zero.
---
# Pre-specified criteria for this test — fixed 30 October 2026, before the first number was computed
## Data
`https://webtai.bipm.org/ftp/pub/tai/Circular-T/cirt/cirt.NNN` for every NNN from
100 to the largest issue in the BIPM directory index at fetch time, capped at 400
issues, fetched sequentially with a 0.35 s delay, one 20 s wall-clock deadline per
request. A laboratory enters the analysis if its series has **at least 500 distinct
5-day epochs** (brief, step 2). Values are `[UTC-UTC(k)]` in ns, taken from section
1 only, read at the MJD columns of each issue's own header (the column count
changes with era: 4, 5, 6 or 7).
Parse self-check (brief, step 1): consecutive issues are compared at their shared
MJD epochs; a value quoted twice must agree. The agreement rate is reported, both
exact and within 0.1 ns. Note in advance: issues of the 1996–2001 era carry a
continuation page, so overlap between consecutive issues arises through the
continuation block, not through a repeat of the first page.
## Estimator
Overlapping Allan deviation of phase data, second differences:
sigma_y^2(tau) = 1/(2 n tau^2) * sum_k [ x(t_k + 2 tau) - 2 x(t_k + tau) + x(t_k) ]^2
with `x` in seconds, `tau` in seconds, and the sum running over every `k` for which
all three epochs are present — which is the overlapping estimator when the grid is
complete and degrades gracefully where it is not. `tau = m * 5 d`, grid log-spaced
at 12 points per decade from 5 d to 3000 d, snapped to multiples of 5 d, and
truncated at half each laboratory's span. A `tau` enters a laboratory's curve only
if at least 3 second-difference triples exist at it; a curve needs at least 8 `tau`
values.
This estimator is *insensitive to a constant frequency offset by construction*: a
linear trend in phase cancels in the second difference. Removing a linear trend
from the series therefore cannot change the answer, and is run anyway (S1) to
demonstrate it rather than assert it. The drift term that can produce a minimum is
a *quadratic* phase trend, i.e. a linear frequency drift.
## Per-laboratory quantities
- `tau_min` / `sigma_min`: the grid point of the minimum of `sigma_y(tau)`.
- **Interior minimum**: the minimum is not at either end of the curve. Primary
definition, exactly as written. Pre-declared stricter variant: interior *and*
`sigma_y(tau_max) / sigma_min >= 2` (S4).
- **Post-minimum slope**: OLS of `log10 sigma_y` on `log10 tau` over all grid points
from `tau_min` to `tau_max`. Pre-minimum slope: same over `tau_first .. tau_min`.
- **CI for the slope**: the points of an Allan curve are not independent — within a
span `T` at lag `tau` there are only about `T/tau` independent second differences.
The reported CI and p-value therefore use `df_eff = floor(span / tau_min) - 2`,
which is conservative, not nominal. The nominal-df numbers are also reported, and
the verdict uses the conservative ones. CI level 95%, two-sided.
- **Drift**: from a quadratic fit `x(t) = c0 + c1 t + c2 t^2` to the phase series,
`D = 2 c2` ns/day², with its OLS standard error.
## Decisive comparisons, fixed in advance
**Verdict 1 (P1).** Supported if more than half of the included laboratories show an
interior minimum. Refuted if more than half fall monotonically to their longest
available `tau`.
**Verdict 2 (P3).** Supported if more than half of the *included* laboratories show
an interior minimum whose post-minimum slope is positive with a 95% CI excluding
zero (the brief's support criterion, verbatim). The aggregate is tested with an
exact two-sided binomial against p = 0.5.
**Verdict 3 (P2, mechanistic).** The claim says the minimum is *set by* the drift
time, so P2 is tested quantitatively, not by inspection. For white frequency noise
plus a linear frequency drift `D`, `sigma_y^2 = a/tau + D^2 tau^2 / 2` with
`a = sigma_y^2(tau_first) * tau_first`, and the minimum sits at
tau_opt = (a / D^2)^(1/3).
P2 is supported in its mechanistic reading if, across laboratories with a
well-determined drift (`|D| / se(D) > 2`), (i) Spearman rho between
`log tau_opt` and `log tau_min` is positive with a seeded permutation p < 0.05
(20,000 permutations, seed 7), and (ii) the median ratio `tau_min / tau_opt` lies
within a factor of 3 of 1.
**Group contrast.** Primary standards `NIST, USNO, PTB, OP, SU` against the rest:
Mann–Whitney on `tau_min` and on `sigma_min`, two-sided normal approximation.
**Correction family.** The per-laboratory post-minimum slope p-values, one per
included laboratory, form one Benjamini–Hochberg family. The binomial test of the
support criterion and the Spearman test of P2 are pre-specified aggregates and are
*not* members of that family; no second correction is applied to them.
## Sensitivity analyses, fixed in advance
- **S1** linear phase trend removed per laboratory (expected: no change; proves the
estimator's immunity rather than assuming it);
- **S2** quadratic phase trend removed, i.e. linear frequency drift removed. If the
minima disappear here, the minimum *is* the drift signal and will be reported as
such; if minima survive, drift does not explain them;
- **S3** first-difference (pairwise) estimator instead of second differences — a
different, non-standard estimator that is sensitive to a constant frequency offset;
- **S4** stricter minimum: `sigma(tau_max)/sigma_min >= 2` (stricter definition, pre-declared);
- **S5** resolution-clean subsample: only issues whose published values carry
sub-nanosecond resolution. The 1996–2001 era prints whole nanoseconds, which
quantises the series and can manufacture a short-`tau` floor out of rounding
noise. **Rule fixed now:** if S5 and the all-issue analysis disagree about a
verdict, S5 is the deciding analysis, because quantisation is a measurement
artefact and the longer record is not worth it.
If the acquisition stops early (server slow, 404, time budget) the test runs on the
issues obtained and the README says so.
## What would change the answer
A distribution of minima with no relation to the fitted drift time would break P2
while leaving P1 and P3 standing; that is the most likely split and it is declared
here in advance. A majority of monotonically falling curves would refute the claim
outright.
---
# Implementation notes (added after the run; documentation only, no criterion changed)
- The effective degrees of freedom actually used for the slope intervals are
`df_eff = min(round(span / tau_min), n_points - 2)` with `span = tau_max - tau_first`,
i.e. at most the number of independent second differences in the fitted segment and
never sharper than the nominal fit. The line above said `floor(span/tau_min) - 2`;
the two differ by two degrees of freedom, and both leave the P3 verdict unchanged:
0 of 90 either way, and the smallest p-value in the family is 0.059 (MTC), which is
also its nominal-df value.
- **S6** was added during the analysis and is labelled post hoc: the interior-minimum
classification repeated on a running three-point median of `log10 sigma_y`. It
returns the same 27 of 90, so the headline is not an artefact of a single noisy
end point.
- The per-laboratory drift model (`D` from a quadratic fit to phase) is only computed
for laboratories that have an interior minimum, because P2 needs a minimum to
compare the prediction against.