Theory 11/More data means better decisions
Variables Outrun Signal
The claim
Adding variables increases overfitting faster than explanatory power once the ratio of variables to observations exceeds a threshold.
01Mechanism
Each variable adds variance to the estimate while adding little signal.
02Prediction
Out-of-sample error should rise past a variable ratio.
03Refutation
Monotonic improvement with variables would remove the threshold.