Incoprea Theory
Vol. XXX · 29

Theory 02/The rival is a confound/Meta-science

Selection or Heterogeneity?

The claim

Published effect sizes exceed replication effects because of selection on significance.

01The rival explanation

The difference is true heterogeneity: replications deliberately sample different populations, so a smaller average effect is expected even with no bias anywhere.

02What would tell them apart

Selection predicts shrinkage proportional to original statistical power, calculable in advance. Heterogeneity predicts shrinkage proportional to differences between replication settings, with no power dependence.

03Prediction

Shrinkage should be predictable from original power alone.

04Refutation

If shrinkage follows setting differences and not power, the rival wins.