Theory 07/Effect sizes/Method
The Replication Decay Law
The expected effect size recovered by a replication is a predictable fraction of the originally published effect, set by the ratio of original sample size to replication sample size and by the original study's selection threshold — so shrinkage is forecastable rather than surprising.
01What is already known
Replication projects in psychology, cancer biology and economics consistently find smaller effects than the originals. This is usually explained after the fact by publication bias or by regression to the mean. Both are real, but neither gives a quantitative forecast.
02The mechanism
Published effects are selected for exceeding a significance threshold. Conditional on selection, the reported effect overestimates the true effect by an amount that depends on power: low-powered studies that pass the threshold are necessarily lucky draws. A replication with a different sample draws from the same truth without the selection filter, so its expected effect is the true effect, and the ratio of expected replication effect to published effect is a function of the original power.
03Predictions
- Shrinkage should be larger for studies with smaller original sample sizes, in a quantitatively specified way.
- Studies with very high power should replicate at close to their published effect.
- The distribution of replication effects should be predictable in advance from the original design alone, before replication.
04Refutation
If shrinkage were unrelated to original power across a large replication sample, the selection account would be wrong and some other mechanism would be needed.
Pre-register the predicted replication effect from each original study's design, then run the replications. The prediction is quantitative and can be scored, not merely described.