How should we report uncertainty in machine-learning predictions?

Standard ML models produce point predictions or softmax probabilities that are often badly calibrated and ignore distributional shift, label noise and model misspecification.

Conformal prediction, Bayesian deep learning and ensemble methods offer partial answers. A unified framework for uncertainty that is computationally cheap, distributionally robust and interpretable is missing.