Theory 06/Speed vs Accuracy/Machine learning
Fast Inference Is Miscalibrated
The claim
Model inference speed trades against calibration, so a distilled model that matches accuracy still produces confidence scores that are too sharp.
01Mechanism
Compression removes the variability that smooths predictions; accuracy can survive while the confidence distribution narrows.
02Prediction
Distilled models should be less calibrated at equal accuracy.
03Refutation
Calibration preserved under compression would remove the trade-off.