Uncertainty quantification for data-driven weather prediction: From probabilistic forecasts to fair model comparisons

Artificial intelligence (AI)-based data-driven weather prediction (AIWP) models have experienced rapid progress over the last years. They achieve impressive results and demonstrate substantial improvements over state-of-the-art physics-based numerical weather prediction (NWP) models across a range of variables and evaluation metrics. However, most efforts in data-driven weather forecasting have been limited to deterministic, point-valued predictions, making it impossible to quantify forecast uncertainties, which is crucial in research and for optimal decision making in applications.I will present recent work on uncertainty quantification (UQ) methods in the context of data-driven weather prediction. The post-hoc use of UQ methods enables the generation of skillful probabilistic weather forecasts from a state-of-the-art deterministic AIWP model [1]. Further, by subjecting the deterministic backbone of physics-based and data-driven models post hoc to the same UQ technique, and computing the in-sample mean continuous ranked probability score of the resulting forecast, we propose a new measure that enables fair and meaningful comparisons of single-valued output from AIWP and NWP models, called potential continuous ranked probability score [2].References[1] Bülte, C., Horat, N., Quinting, J. and Lerch, S. (2025). Uncertainty quantification for data-driven weather models. Artificial Intelligence for the Earth System, in press. DOI:10.1175/AIES-D-24-0049.1[2] Gneiting, T., Biegert, T., Kraus, K., Walz, E.-M., Jordan, A. I., and Lerch, S. (2025). Probabilistic measures afford fair comparisons of AIWP and NWP model output. Preprint, arXiv:2506.03744. DOI:10.48550/arXiv.2506.03744