How should we evaluate forecasts probabilistically?

Point forecasts (best guess) are easy to evaluate against outcomes. Probabilistic forecasts (a distribution over outcomes) are harder. Brier scores, log scores, CRPS and calibration diagrams are all in use.

Which scoring rule is appropriate for which problem, especially for rare events or in adversarial settings (e.g., political forecasting), is debated.