Incoprea research · leads F · data, provenance and replication · 3 November 2026

Two hundred leads on data, provenance and replication

This page exists because of one failure and one anomaly in the earlier parts of the programme. The failure: a catalogue column (impact energy) turned out to be a deterministic recalibration of another column (radiated energy) — r = 0.99992 over four and a half decades — so every analysis that treated the two as two measurements was using one measurement twice. The anomaly: a tail index that did not reproduce, for reasons that were never separated into data, estimator and threshold. Both are provenance problems before they are physics problems, and neither is exotic.

Part 1 · the leads

The four sections, and what the tags mean

test means the sentence could be settled by computation or by an examination of public catalogues, without a new experiment. note means it could be settled only by a study of practice — a survey, a replication trial, or an audit of published tables — which this programme has not run and, in several cases, could not run from a desk.

A. Derived columns, correlated variables and false independence

Fifty leads on the failure that started this page: a catalogue column that is a function of another column, used as though it were a second measurement.

B. Catalogue completeness, snapshot drift and the fitted index

Fifty leads on what truncation and a moving database do to any index fitted on top of them.

C. Replication practice: what makes a claim checkable from the record alone

Fifty leads on the difference between a claim that can be checked and one that can only be believed.

D. Reporting standards that would prevent the failures seen here

Fifty leads on standards, each stated so that it could be shown to fail.

Part 2 · tests run

Tests run

Three computations, all synthetic. Each has a known generating truth, so the failure modes can be measured rather than argued about. Code: incoprea_nov3/code/data_leads_compute.py. Output: incoprea_nov3/results/data_leads.json. Seed 20261103, numpy 2.5.4. Re-run with python3 data_leads_compute.py.

T1. A derived column that the obvious test cannot see

A catalogue of n = 400 rows with one measured column x spanning 6.9 decades and a second column generated as y = 0.032·x^0.92 with 5% multiplicative error — that is, y is derived, and perfectly so in the sense that matters. The question is which summary statistic reveals it.

The raw-scale fit leaves R² = 0.9837, which a detector requiring 0.99 does not flag on this draw; across 400 replicates of the same generating process that detector fires on 53% of pairs at a 0.99 cut, 22% at 0.995 and 5.5% at 0.999. The log–log fit fires on 100% at 0.99 and recovers the exponent as 0.9180 against a true 0.92. The reason is not subtle: a straight line is the wrong functional form on the raw scale, so the fit spends its residual on curvature. The raw-scale residual correlates with x² at -0.32, which is the signature.

The failure is exponent-dependent, which is the part I did not expect. Sweeping the true exponent at fixed span and noise:

Detection peaks near exponent 1, where the relation is linear in raw units and the raw-scale detector is not testing derivation at all but near-linearity. At exponent 0.33 — a cube-root scaling, entirely ordinary in catalogue work — the raw detector fires on 0% of pairs while the log–log detector fires on all of them. Any screen calibrated on a linear relation transfers badly.

The case the log–log test also misses

A derived column need not be monotone. With y = x² + noise and x symmetric about zero, both the Pearson and the Spearman statistic sit near zero. Power at the 5% level over 300 replicates:

Pearson retains some power (42%) only because the quadratic leaks into the covariance; Spearman, which is supposed to be the robust choice, is worse (16%). Distance correlation separates the cases cleanly (100% power, example statistic 0.56 against a null 95th percentile of 0.12). That is the specific test the title of this section refers to — and it is also, as T3 shows, too permissive to use as a derivation test on its own.

A third shape: the capped column

The same power law, but values above the 40th percentile are replaced by that percentile — an instrument ceiling recorded as a value. 60% of rows now sit exactly at the cap.

The raw correlation collapses to 0.12 and the log–log R² to 0.67: an analyst screening on either would conclude the two columns are weakly related, when in fact one is a censored function of the other. The diagnostic is not a correlation at all but the mass at a single value — 0.60 against 0.013 for the uncapped column.

T2. What a missing 10% of the smallest events does to a tail index

The prior expectation, mine included, was that dropping the smallest decile of a catalogue biases the tail index because the small events carry the shape information. That expectation is wrong for the estimators that are actually used, and the computation says so exactly.

Exact Pareto, α = 1.6, n = 20,000

Hill at a fixed threshold above the deletion cut returns 1.6408 either way — bias 0.0000%, and that is not a numerical accident: the estimator never sees the deleted points, so their absence cannot move it. The MLE at the observed minimum is equally unmoved, because a Pareto is scale-invariant. The only estimator that moves at all is the one whose threshold is re-anchored to a fraction of the retained sample, and it moves by -0.14%.

The bias everyone fears does not live in the deletion. It lives in the fitted range. Take a catalogue that is a mixture — a lognormal body carrying a Pareto tail — and fit a single power law across the whole retained range, which is what a paper does when it quotes one exponent for a catalogue:

The single fitted power law moves from 1.0436 to 1.2281, a shift of +17.7%, while the local Hill estimator at a fixed threshold does not move at all. The deletion removed the part of the curve that was holding the global fit down. Note also that the global fit has R² = 0.8701 — below 0.9 — and is quoted anyway, in this simulation as in practice.

The case where the bias is real: detection that ramps through the threshold

Real catalogues do not truncate cleanly. Detection probability rises smoothly through the threshold, so events just above the cut are preferentially missing. With a logistic detection ramp of width w centred on u₀ = 2.0 and true α = 1.6:

Losing 12.8% of the exceedances, concentrated near the threshold, biases Hill low by 10.2%; losing 20.2% biases it by 13.3%. The bias grows with the width of the ramp, not with the count of missing events — which is why a completeness correction that only adjusts the count does not fix it. A hard 10% truncation biases any rate normalised by the retained total by exactly -10.0%, and that one is unavoidable arithmetic.

The result, stated plainly. A missing 10% of the smallest events has zero effect on a threshold-conditional tail index, and a +17.7% effect on a single power law fitted across the whole range. Whether truncation matters is a property of the estimator, not of the catalogue. Reporting the estimator therefore does more work than reporting the completeness.

T3. An automatic derived-column detector, and its two ceilings

Rule: fit log–log OLS to each numeric pair; flag the pair as derived when R² ≥ 0.99 and the residual sd ≤ 0.1 dex. Seven synthetic classes with known labels, n = 200 per replicate, 200 replicates each; the distance-correlation companion uses the 95th percentile of the independent class as its threshold (0.179).

Read the last three columns as the finding. The log–log rule is a clean separator where the relation is functional and monotone: 100% sensitivity at 0.03 dex of noise, 0% false positives on independent pairs, 0% on the confounded class. But it collapses to 0% once the derivation carries 0.15 dex of noise, and to 0% on a capped derivation and 0% on a non-monotone one.

Distance correlation is the opposite failure. It catches every derived class (100% including the non-monotone and the capped) and also flags 100% of the confounded class — pairs that share a latent driver and are not derived. Combining the two with a logical OR, which was my first instinct, therefore has a 100% false-positive rate on confounded pairs. The OR is not a better detector; it is a different detector measuring dependence rather than derivation.

Two ceilings, then. The threshold sweep shows the first:

At a 0.10 dex residual cut, sensitivity is 100% for a 0.03 dex derivation and exactly 0% for anything noisier, at every R² cut from 0.95 to 0.999. This is a wall, not a curve. It follows from the definitions: R² and residual sd are near-redundant when the span of x is fixed, so two thresholds give one degree of freedom. Raising the residual cut to 0.20 dex buys sensitivity at 0.15 dex of noise — and would admit any two columns whose scatter is under a factor of 1.6, which in a real catalogue is most of them.

The second ceiling is that more data does not help:

At 0.12 dex of noise the detector fires on 0 to 6% of derived pairs at every sample size from 30 to 1,600. The residual sd is an unbiased estimate of the noise at any n, so a threshold below the noise level is never crossed by adding rows. Only the noise matters, never the count.

What does move sensitivity is the dynamic range of the column, which no threshold statement mentions:

At 0.3 decades of span, even a perfect 0.08 dex derivation reaches only R² = 0.934 and is missed by every threshold in the sweep. A detector validated on a three-decade synthetic column will silently fail on a real column that spans a factor of two. Sensitivity is a property of the pair’s range as much as of the model.

What the detector cannot do at all

The confounded class is not a nuisance case. Two genuinely separate measurements driven by one latent variable reach a median R² of 0.63 and a distance correlation of 0.75 — high enough that every bivariate statistic examined here calls them related, and none can call them derived. Derivation and confounding are observationally equivalent from two columns. The only thing that separates them is provenance: which column was computed from which. That is a metadata fact, not a statistical one, and it is the reason the first lead in section D is a boolean field rather than a better test.

Part 3 · limits

Limits

• Everything in the tests is a simulation. The generating truth is known because I wrote it. That makes the failure modes measurable and says nothing about how often they occur in real catalogues. The two real-catalogue facts quoted in the introduction come from the earlier pages of this programme, not from this one.
• The classes are mine, and chosen. Seven classes is not a sample of catalogue columns; it is a set of cases I could construct. A prevalence estimate for derived columns in public data cannot be read off these numbers and is not offered.
• The detector thresholds were tuned on the same simulations that evaluate them. The 0.99 / 0.10 operating point was chosen after seeing the sweep. Its sensitivity and false-positive rate on these classes are therefore optimistic, and the honest statement is the shape of the trade-off, not the two numbers.
• Distance correlation is reported without a permutation test for most classes; the threshold is the 95th percentile of a 200-draw null from the same generator. A proper test would permute and would be slower by two orders of magnitude.
• The tail-index results are exact for the models simulated and say nothing about the mixture structure of any real catalogue. The +17.7% shift depends on the body-to-tail ratio I chose; a different mixture gives a different number with the same sign.
• One seed. The single-run numbers (as opposed to the replicate means) carry sampling error of a few per cent. Seed 20261103 is recorded, and re-running reproduces them exactly.
• The 200 leads are untested by construction. Several are probably already known in the metrology or reproducibility literature under other names; no literature search was run. Their value is as a falsifiable checklist, not as findings.
• No claim that any named catalogue or paper is wrong. The failure discussed in the introduction is one this programme found in its own use of a catalogue, not an accusation against the catalogue.

If one sentence survives: a derived column is invisible to the tests people actually run — 53% at best on the raw scale, 42% by Pearson against a non-monotone derivation, and 0% by any of them once the column is capped — and the only reliable detector is a metadata flag that costs one boolean per column.

Incoprea Research · leads F · 3 November 2026. Two hundred leads, three simulations, no measured data. Code and JSON in incoprea_nov3/code/ and incoprea_nov3/results/. No novelty claimed; no systematic literature search.