Articles | Volume 11, issue 1
https://doi.org/10.5194/ascmo-11-23-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/ascmo-11-23-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
13 Mar 2025
| 13 Mar 2025
Proper scoring rules for multivariate probabilistic forecasts based on aggregation and transformation
Université Marie et Louis Pasteur, CNRS, LmB (UMR 6623), 25000 Besançon, France
Clément Dombry
Université Marie et Louis Pasteur, CNRS, LmB (UMR 6623), 25000 Besançon, France
Philippe Naveau
Laboratoire des Sciences du Climat et de l'Environnement, UMR 8212, CEA-CNRS-UVSQ, EstimR, IPSL & U Paris-Saclay, Gif-sur-Yvette, France
Maxime Taillardat
CNRM, Université de Toulouse, Météo-France, CNRS, Toulouse, France
Related authors
No articles found.
Cedric Gacial Ngoungue Langue, Helene Brogniez, and Philippe Naveau
EGUsphere, https://doi.org/10.5194/egusphere-2024-3481, https://doi.org/10.5194/egusphere-2024-3481, 2025
This preprint is open for discussion and under review for Atmospheric Measurement Techniques (AMT).
Short summary
Short summary
This work evaluates the representation of total column water vapor and total cloud cover in General Circulation Models, ERA5 reanalysis and satellite data records from the European Space Agency Climate Change Initiative. A new technique, called "multiresolution analysis," is applied to this evaluation, which enables an analysis of model behavior across different temporal frequencies, from daily to decadal scales, including subseasonal and seasonal variations.
Pauline Rivoire, Olivia Martius, Philippe Naveau, and Alexandre Tuel
Nat. Hazards Earth Syst. Sci., 23, 2857–2871, https://doi.org/10.5194/nhess-23-2857-2023, https://doi.org/10.5194/nhess-23-2857-2023, 2023
Short summary
Short summary
Heavy precipitation can lead to floods and landslides, resulting in widespread damage and significant casualties. Some of its impacts can be mitigated if reliable forecasts and warnings are available. In this article, we assess the capacity of the precipitation forecast provided by ECMWF to predict heavy precipitation events on a subseasonal-to-seasonal (S2S) timescale over Europe. We find that the forecast skill of such events is generally higher in winter than in summer.
Jonathan Demaeyer, Jonas Bhend, Sebastian Lerch, Cristina Primo, Bert Van Schaeybroeck, Aitor Atencia, Zied Ben Bouallègue, Jieyu Chen, Markus Dabernig, Gavin Evans, Jana Faganeli Pucer, Ben Hooper, Nina Horat, David Jobst, Janko Merše, Peter Mlakar, Annette Möller, Olivier Mestre, Maxime Taillardat, and Stéphane Vannitsem
Earth Syst. Sci. Data, 15, 2635–2653, https://doi.org/10.5194/essd-15-2635-2023, https://doi.org/10.5194/essd-15-2635-2023, 2023
Short summary
Short summary
A benchmark dataset is proposed to compare different statistical postprocessing methods used in forecasting centers to properly calibrate ensemble weather forecasts. This dataset is based on ensemble forecasts covering a portion of central Europe and includes the corresponding observations. Examples on how to download and use the data are provided, a set of evaluation methods is proposed, and a first benchmark of several methods for the correction of 2 m temperature forecasts is performed.
Manuela Irene Brunner and Philippe Naveau
Hydrol. Earth Syst. Sci., 27, 673–687, https://doi.org/10.5194/hess-27-673-2023, https://doi.org/10.5194/hess-27-673-2023, 2023
Short summary
Short summary
Reservoir regulation affects various streamflow characteristics. Still, information on when water is stored in and released from reservoirs is hardly available. We develop a statistical model to reconstruct reservoir operation signals from observed streamflow time series. By applying this approach to 74 catchments in the Alps, we find that reservoir management varies by catchment elevation and that seasonal redistribution from summer to winter is strongest in high-elevation catchments.
Antoine Grisart, Mathieu Casado, Vasileios Gkinis, Bo Vinther, Philippe Naveau, Mathieu Vrac, Thomas Laepple, Bénédicte Minster, Frederic Prié, Barbara Stenni, Elise Fourré, Hans Christian Steen-Larsen, Jean Jouzel, Martin Werner, Katy Pol, Valérie Masson-Delmotte, Maria Hoerhold, Trevor Popp, and Amaelle Landais
Clim. Past, 18, 2289–2301, https://doi.org/10.5194/cp-18-2289-2022, https://doi.org/10.5194/cp-18-2289-2022, 2022
Short summary
Short summary
This paper presents a compilation of high-resolution (11 cm) water isotopic records, including published and new measurements, for the last 800 000 years from the EPICA Dome C ice core, Antarctica. Using this new combined water isotopes (δ18O and δD) dataset, we study the variability and possible influence of diffusion at the multi-decadal to multi-centennial scale. We observe a stronger variability at the onset of the interglacial interval corresponding to a warm period.
Julie Bessac and Philippe Naveau
Adv. Stat. Clim. Meteorol. Oceanogr., 7, 53–71, https://doi.org/10.5194/ascmo-7-53-2021, https://doi.org/10.5194/ascmo-7-53-2021, 2021
Short summary
Short summary
We propose a new forecast evaluation scheme in the context of models that incorporate errors of the verification data. We rely on existing scoring rules and incorporate uncertainty and error of the verification data through a hidden variable and the conditional expectation of scores. By considering scores to be random variables, one can access the entire range of their distribution and illustrate that the commonly used mean score can be a misleading representative of the distribution.
Guillaume Evin, Matthieu Lafaysse, Maxime Taillardat, and Michaël Zamo
Nonlin. Processes Geophys., 28, 467–480, https://doi.org/10.5194/npg-28-467-2021, https://doi.org/10.5194/npg-28-467-2021, 2021
Short summary
Short summary
Forecasting the height of new snow is essential for avalanche hazard surveys, road and ski resort management, tourism attractiveness, etc. Météo-France operates a probabilistic forecasting system using a numerical weather prediction system and a snowpack model. It provides better forecasts than direct diagnostics but exhibits significant biases. Post-processing methods can be applied to provide automatic forecasting products from this system.
Jakob Zscheischler, Philippe Naveau, Olivia Martius, Sebastian Engelke, and Christoph C. Raible
Earth Syst. Dynam., 12, 1–16, https://doi.org/10.5194/esd-12-1-2021, https://doi.org/10.5194/esd-12-1-2021, 2021
Short summary
Short summary
Compound extremes such as heavy precipitation and extreme winds can lead to large damage. To date it is unclear how well climate models represent such compound extremes. Here we present a new measure to assess differences in the dependence structure of bivariate extremes. This measure is applied to assess differences in the dependence of compound precipitation and wind extremes between three model simulations and one reanalysis dataset in a domain in central Europe.
Stephan Hemri, Sebastian Lerch, Maxime Taillardat, Stéphane Vannitsem, and Daniel S. Wilks
Nonlin. Processes Geophys., 27, 519–521, https://doi.org/10.5194/npg-27-519-2020, https://doi.org/10.5194/npg-27-519-2020, 2020
Maxime Taillardat and Olivier Mestre
Nonlin. Processes Geophys., 27, 329–347, https://doi.org/10.5194/npg-27-329-2020, https://doi.org/10.5194/npg-27-329-2020, 2020
Short summary
Short summary
Statistical post-processing of ensemble forecasts is now a well-known procedure in order to correct biased and misdispersed ensemble weather predictions. But practical application in European national weather services is in its infancy. Different applications of ensemble post-processing using machine learning at an industrial scale are presented. Forecast quality and value are improved compared to the raw ensemble, but several facilities have to be made to adjust to operational constraints.
Yoann Robin, Mathieu Vrac, Philippe Naveau, and Pascal Yiou
Hydrol. Earth Syst. Sci., 23, 773–786, https://doi.org/10.5194/hess-23-773-2019, https://doi.org/10.5194/hess-23-773-2019, 2019
Short summary
Short summary
Bias correction methods are used to calibrate climate model outputs with respect to observations. In this article, a non-stationary, multivariate and stochastic bias correction method is developed based on optimal transport, accounting for inter-site and inter-variable correlations. Optimal transport allows us to construct a joint distribution that minimizes energy spent in bias correction. Our methodology is tested on precipitation and temperatures over 12 locations in southern France.
Yoann Robin, Pascal Yiou, and Philippe Naveau
Nonlin. Processes Geophys., 24, 393–405, https://doi.org/10.5194/npg-24-393-2017, https://doi.org/10.5194/npg-24-393-2017, 2017
Short summary
Short summary
If climate is viewed as a chaotic dynamical system, its trajectories yield on an object called an attractor. Being perturbed by an external forcing, this attractor could be modified. With Wasserstein distance, we estimate on a derived Lorenz model the impact of a forcing similar to climate change. Our approach appears to work with small data sizes. We have obtained a methodology quantifying the deformation of well-known attractors, coherent with the size of data available.
Pascal Yiou, Aglaé Jézéquel, Philippe Naveau, Frederike E. L. Otto, Robert Vautard, and Mathieu Vrac
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 17–31, https://doi.org/10.5194/ascmo-3-17-2017, https://doi.org/10.5194/ascmo-3-17-2017, 2017
Short summary
Short summary
The attribution of classes of extreme events, such as heavy precipitation or heatwaves, relies on the estimate of small probabilities (with and without climate change). Such events are connected to the large-scale atmospheric circulation. This paper links such probabilities with properties of the atmospheric circulation by using a Bayesian decomposition. We test this decomposition on a case of extreme precipitation in the UK, in January 2014.
Allison H. Baker, Dorit M. Hammerling, Sheri A. Mickelson, Haiying Xu, Martin B. Stolpe, Phillipe Naveau, Ben Sanderson, Imme Ebert-Uphoff, Savini Samarasinghe, Francesco De Simone, Francesco Carbone, Christian N. Gencarelli, John M. Dennis, Jennifer E. Kay, and Peter Lindstrom
Geosci. Model Dev., 9, 4381–4403, https://doi.org/10.5194/gmd-9-4381-2016, https://doi.org/10.5194/gmd-9-4381-2016, 2016
Short summary
Short summary
We apply lossy data compression to output from the Community Earth System Model Large Ensemble Community Project. We challenge climate scientists to examine features of the data relevant to their interests and identify which of the ensemble members have been compressed, and we perform direct comparisons on features critical to climate science. We find that applying lossy data compression to climate model data effectively reduces data volumes with minimal effect on scientific results.
Related subject area
Statistics
Reducing reliability bias in assessments of extreme weather risk using calibrating priors
A non-stationary climate-informed weather generator for assessing future flood risks
A robust approach to Gaussian process implementation
Spatiotemporal functional permutation tests for comparing observed climate behavior to climate model projections
Parametric model for post-processing visibility ensemble forecasts
Spatiotemporal methods for estimating subsurface ocean thermal response to tropical cyclones
Applying different methods to model dry and wet spells at daily scale in a large range of rainfall regimes across Europe
Comparison of climate time series – Part 5: Multivariate annual cycles
Regridding uncertainty for statistical downscaling of solar radiation
Quantifying the statistical dependence of mid-latitude heatwave intensity and likelihood on prevalent physical drivers and climate change
Statistical modeling of the space–time relation between wind and significant wave height
Modeling general circulation model bias via a combination of localized regression and quantile mapping methods
Evaluation of simulated responses to climate forcings: a flexible statistical framework using confirmatory factor analysis and structural equation modelling – Part 1: Theory
Evaluation of simulated responses to climate forcings: a flexible statistical framework using confirmatory factor analysis and structural equation modelling – Part 2: Numerical experiment
A conditional approach for joint estimation of wind speed and direction under future climates
Comparing climate time series – Part 2: A multivariate test
Forecast score distributions with imperfect observations
Novel measures for summarizing high-resolution forecast performance
Copula approach for simulated damages caused by landfalling US hurricanes
Nonstationary extreme value analysis for event attribution combining climate models and observations
Comparing climate time series – Part 1: Univariate test
A statistical approach to fast nowcasting of lightning potential fields
Spatial trend analysis of gridded temperature data at varying spatial scales
An improved projection of climate observations for detection and attribution
Bivariate Gaussian models for wind vectors in a distributional regression framework
Fitting a stochastic fire spread model to data
Influence of initial ocean conditions on temperature and precipitation in a coupled climate model's solution
NWP-based lightning prediction using flexible count data regression
An integration and assessment of multiple covariates of nonstationary storm surge statistical behavior by Bayesian model averaging
Probabilistic evaluation of competing climate models
Assessing NARCCAP climate model effects using spatial confidence regions
Generalised block bootstrap and its use in meteorology
Estimating trends in the global mean temperature record
Reconstruction of spatio-temporal temperature from sparse historical records using robust probabilistic principal component regression
Analysis of variability of tropical Pacific sea surface temperatures
Evaluating NARCCAP model performance for frequencies of severe-storm environments
Estimating changes in temperature extremes from millennial-scale climate simulations using generalized extreme value (GEV) distributions
A comparison of two methods for detecting abrupt changes in the variance of climatic time series
Calibrating regionally downscaled precipitation over Norway through quantile-based approaches
Comparison of hidden and observed regime-switching autoregressive models for (u, v)-components of wind fields in the northeastern Atlantic
Autoregressive spatially varying coefficients model for predicting daily PM2.5 using VIIRS satellite AOT
Bivariate spatial analysis of temperature and precipitation from general circulation models and observation proxies
Joint inference of misaligned irregular time series with application to Greenland ice core data
Simulation of future climate under changing temporal covariance structures
Stephen Jewson, Trevor Sweeting, and Lynne Jewson
Adv. Stat. Clim. Meteorol. Oceanogr., 11, 1–22, https://doi.org/10.5194/ascmo-11-1-2025, https://doi.org/10.5194/ascmo-11-1-2025, 2025
Short summary
Short summary
We investigate how to make statistical predictions of extreme weather such that events predicted to occur with a probability of 1 % will occur 1 % of the time. We apply the methods we describe to a standard extreme weather attribution example from the recent climate literature. We find that the methods we describe imply that extremes are roughly twice as likely as when estimated using maximum likelihood. We have developed a software package to make it easy to apply these methods.
Viet Dung Nguyen, Sergiy Vorogushyn, Katrin Nissen, Lukas Brunner, and Bruno Merz
Adv. Stat. Clim. Meteorol. Oceanogr., 10, 195–216, https://doi.org/10.5194/ascmo-10-195-2024, https://doi.org/10.5194/ascmo-10-195-2024, 2024
Short summary
Short summary
We present a novel stochastic weather generator conditioned on circulation patterns and regional temperature, accounting for dynamic and thermodynamic atmospheric changes. We extensively evaluate the model for the central European region. It statistically downscales precipitation for future periods, generating long, spatially and temporally consistent series. Results suggest an increase in extreme precipitation over the region, offering key benefits for hydrological impact studies.
Juliette Mukangango, Amanda Muyskens, and Benjamin W. Priest
Adv. Stat. Clim. Meteorol. Oceanogr., 10, 143–158, https://doi.org/10.5194/ascmo-10-143-2024, https://doi.org/10.5194/ascmo-10-143-2024, 2024
Short summary
Short summary
In this study, we investigated the performance of Gaussian process regression (GP) models in handling outlier-affected spatial datasets. Our findings emphasized that models with the proposed methods provided accurate predictions and reliable uncertainty quantification, showcasing resilience against outliers. Overall, our study contributes to advancing the understanding of GP regression in spatial contexts and offers practical solutions to enhance its applicability in outlier-rich environments.
Joshua P. French, Piotr S. Kokoszka, and Seth McGinnis
Adv. Stat. Clim. Meteorol. Oceanogr., 10, 123–141, https://doi.org/10.5194/ascmo-10-123-2024, https://doi.org/10.5194/ascmo-10-123-2024, 2024
Short summary
Short summary
Future climate behavior is typically modeled using computer-based simulations, which are generated for both historical and future time periods. The trustworthiness of these models can be assessed by determining whether the simulated historical climate matches what was observed. We provide a tool that allows researchers to identify major differences between observed climate and climate model predictions, which will hopefully lead to further model refinements.
Ágnes Baran and Sándor Baran
Adv. Stat. Clim. Meteorol. Oceanogr., 10, 105–122, https://doi.org/10.5194/ascmo-10-105-2024, https://doi.org/10.5194/ascmo-10-105-2024, 2024
Short summary
Short summary
The paper proposes a novel parametric model for statistical post-processing of visibility ensemble forecasts; investigates various approaches to parameter estimation; and, using two case studies, provides a detailed comparison with the existing state-of-the-art forecasts. The introduced approach consistently outperforms both the raw ensemble forecasts and the reference parametric post-processing method.
Addison J. Hu, Mikael Kuusela, Ann B. Lee, Donata Giglio, and Kimberly M. Wood
Adv. Stat. Clim. Meteorol. Oceanogr., 10, 69–93, https://doi.org/10.5194/ascmo-10-69-2024, https://doi.org/10.5194/ascmo-10-69-2024, 2024
Short summary
Short summary
We introduce a new statistical framework to estimate the change in subsurface ocean temperature following the passage of a tropical cyclone (TC). Our approach combines tools handling seasonal variations and spatial dependence in the data, culminating in a three-dimensional characterization of the interaction between TCs and the ocean. Our work allows us to obtain new scientific insights, and we expect it to be generally applicable to studying the impact of TCs on other ocean phenomena.
Giorgio Baiamonte, Carmelo Agnese, Carmelo Cammalleri, Elvira Di Nardo, Stefano Ferraris, and Tommaso Martini
Adv. Stat. Clim. Meteorol. Oceanogr., 10, 51–67, https://doi.org/10.5194/ascmo-10-51-2024, https://doi.org/10.5194/ascmo-10-51-2024, 2024
Short summary
Short summary
In hydrology, the probability distributions are used to determine the probability of occurrence of rainfall events. In this study, two different methods for modeling rainfall time characteristics have been applied: a direct method and an indirect method that make it possible to relax the assumptions of the renewal process. The analysis was extended to two additional time variables that may be of great interest for practical hydrological applications: wet chains and dry chains.
Timothy DelSole and Michael K. Tippett
Adv. Stat. Clim. Meteorol. Oceanogr., 10, 1–27, https://doi.org/10.5194/ascmo-10-1-2024, https://doi.org/10.5194/ascmo-10-1-2024, 2024
Short summary
Short summary
This paper introduces a method to assess whether two data sets come from the same source. Current methods do not adequately consider spatial and temporal correlations and their annual cycles in a comprehensive test. This method addresses that gap, thereby providing a new and rigorous tool for evaluating the realism of climate simulations and measuring changes in variability over time.
Maggie D. Bailey, Douglas Nychka, Manajit Sengupta, Aron Habte, Yu Xie, and Soutir Bandyopadhyay
Adv. Stat. Clim. Meteorol. Oceanogr., 9, 103–120, https://doi.org/10.5194/ascmo-9-103-2023, https://doi.org/10.5194/ascmo-9-103-2023, 2023
Short summary
Short summary
To ensure photovoltaic (PV) plants last, we need to understand how climate change affects solar radiation. Climate models help predict future radiation for PV plants, but there is often uncertainty. We explore this uncertainty and its impact on building PV plants. We highlight the importance of considering uncertainties for accurate planning and management. A California case study shows a practical application for the solar industry.
Joel Zeder and Erich M. Fischer
Adv. Stat. Clim. Meteorol. Oceanogr., 9, 83–102, https://doi.org/10.5194/ascmo-9-83-2023, https://doi.org/10.5194/ascmo-9-83-2023, 2023
Short summary
Short summary
The intensities of recent heatwave events, such as the record-breaking heatwave in early June 2021 in the Pacific Northwest area, are substantially altered by climate change. We further quantify the contribution of the local weather situation and the land surface conditions with a statistical model suited for extreme data. Based on this method, we can answer
what ifquestions, such as estimating the change in the 2021 heatwave temperature if it happened in a world without climate change.
Said Obakrim, Pierre Ailliot, Valérie Monbet, and Nicolas Raillard
Adv. Stat. Clim. Meteorol. Oceanogr., 9, 67–81, https://doi.org/10.5194/ascmo-9-67-2023, https://doi.org/10.5194/ascmo-9-67-2023, 2023
Short summary
Short summary
Ocean wave climate has a significant impact on human activities, and its understanding is of socioeconomic and environmental importance. In this study, we propose a statistical model that predicts wave heights in a location in the Bay of Biscay. The proposed method allows us to understand the spatiotemporal relationship between wind and waves and predicts well both wind seas and swells.
Benjamin James Washington, Lynne Seymour, and Thomas L. Mote
Adv. Stat. Clim. Meteorol. Oceanogr., 9, 1–28, https://doi.org/10.5194/ascmo-9-1-2023, https://doi.org/10.5194/ascmo-9-1-2023, 2023
Short summary
Short summary
We develop new methodology to statistically model known bias in general atmospheric circulation models. We focus on Puerto Rico specifically because of other important ongoing and long-term ecological and environmental research taking place there. Our methods work even in the presence of Puerto Rico's broken climate record. With our methods, we find that climate change will not only favor a warmer and wetter climate in Puerto Rico, but also increase the frequency of extreme rainfall events.
Katarina Lashgari, Gudrun Brattström, Anders Moberg, and Rolf Sundberg
Adv. Stat. Clim. Meteorol. Oceanogr., 8, 225–248, https://doi.org/10.5194/ascmo-8-225-2022, https://doi.org/10.5194/ascmo-8-225-2022, 2022
Short summary
Short summary
This work theoretically motivates an extension of the statistical model used in so-called detection and attribution studies to structural equation modelling. The application of one of the models suggested is exemplified in a small numerical study, whose aim was to check the assumptions typically placed on ensembles of climate model simulations when constructing mean sequences. he result of this study indicated that some ensembles for some regions may not satisfy the assumptions in question.
Katarina Lashgari, Anders Moberg, and Gudrun Brattström
Adv. Stat. Clim. Meteorol. Oceanogr., 8, 249–271, https://doi.org/10.5194/ascmo-8-249-2022, https://doi.org/10.5194/ascmo-8-249-2022, 2022
Short summary
Short summary
The performance of a new statistical framework containing various structural equation modelling (SEM) models is evaluated in a pseudo-proxy experiment in comparison with the performance of statistical models used in many detection and attribution studies. Each statistical model was fitted to seven continental-scale regional temperature data sets. The results indicated the SEM specification is the most appropriate for describing the underlying latent structure of the simulated data analysed.
Qiuyi Wu, Julie Bessac, Whitney Huang, Jiali Wang, and Rao Kotamarthi
Adv. Stat. Clim. Meteorol. Oceanogr., 8, 205–224, https://doi.org/10.5194/ascmo-8-205-2022, https://doi.org/10.5194/ascmo-8-205-2022, 2022
Short summary
Short summary
We study wind conditions and their potential future changes across the U.S. via a statistical conditional framework. We conclude that changes between historical and future wind directions are small, but wind speeds are generally weakened in the projected period, with some locations being intensified. Moreover, winter wind speeds are projected to decrease in the northwest, Colorado, and the northern Great Plains (GP), while summer wind speeds over the southern GP slightly increase in the future.
Timothy DelSole and Michael K. Tippett
Adv. Stat. Clim. Meteorol. Oceanogr., 7, 73–85, https://doi.org/10.5194/ascmo-7-73-2021, https://doi.org/10.5194/ascmo-7-73-2021, 2021
Short summary
Short summary
After a new climate model is constructed, a natural question is whether it generates realistic simulations. Here,
realisticdoes not mean that the detailed patterns on a particular day are correct, but rather that the statistics over many years are realistic. Past approaches to answering this question often neglect correlations in space and time. This paper proposes a method for answering this question that accounts for correlations in space and time.
Julie Bessac and Philippe Naveau
Adv. Stat. Clim. Meteorol. Oceanogr., 7, 53–71, https://doi.org/10.5194/ascmo-7-53-2021, https://doi.org/10.5194/ascmo-7-53-2021, 2021
Short summary
Short summary
We propose a new forecast evaluation scheme in the context of models that incorporate errors of the verification data. We rely on existing scoring rules and incorporate uncertainty and error of the verification data through a hidden variable and the conditional expectation of scores. By considering scores to be random variables, one can access the entire range of their distribution and illustrate that the commonly used mean score can be a misleading representative of the distribution.
Eric Gilleland
Adv. Stat. Clim. Meteorol. Oceanogr., 7, 13–34, https://doi.org/10.5194/ascmo-7-13-2021, https://doi.org/10.5194/ascmo-7-13-2021, 2021
Short summary
Short summary
Verifying high-resolution weather forecasts has become increasingly complicated,
and simple, easy-to-understand summary measures are a good alternative. Recent work has demonstrated some common pitfalls with many such summaries. Here, new summary measures are introduced that do not suffer from these drawbacks, while still providing meaningful information.
Thomas Patrick Leahy
Adv. Stat. Clim. Meteorol. Oceanogr., 7, 1–11, https://doi.org/10.5194/ascmo-7-1-2021, https://doi.org/10.5194/ascmo-7-1-2021, 2021
Short summary
Short summary
This study looked at estimating damages caused by hurricanes in the United States. It assessed the relationship between the maximum wind speed at landfall and the resulting damage caused. The study found that the complex processes that determine the size of the damages inflicted could be estimated using this simple relationship. This work could be used to examine how often extreme damage events are likely to occur and the impact of stronger hurricane winds on the US Atlantic and Gulf coasts.
Yoann Robin and Aurélien Ribes
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 205–221, https://doi.org/10.5194/ascmo-6-205-2020, https://doi.org/10.5194/ascmo-6-205-2020, 2020
Short summary
Short summary
We have developed a new statistical method to describe how a severe weather event, such as a heat wave, may have been influenced by climate change. Our method incorporates both observations and data from various climate models to reflect climate model uncertainty. Our results show that both the probability and the intensity of the French July 2019 heatwave have increased significantly in response to human influence. We find that this heat wave might not have been possible without climate change.
Timothy DelSole and Michael K. Tippett
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 159–175, https://doi.org/10.5194/ascmo-6-159-2020, https://doi.org/10.5194/ascmo-6-159-2020, 2020
Short summary
Short summary
Scientists often are confronted with the question of whether two time series are statistically distinguishable. This paper proposes a test for answering this question. The basic idea is to fit each time series to a time series model and then test whether the parameters in that model are equal. If a difference is detected, then new ways of visualizing those differences are proposed, including a clustering technique and a method based on optimal initial conditions.
Joshua North, Zofia Stanley, William Kleiber, Wiebke Deierling, Eric Gilleland, and Matthias Steiner
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 79–90, https://doi.org/10.5194/ascmo-6-79-2020, https://doi.org/10.5194/ascmo-6-79-2020, 2020
Short summary
Short summary
Very short-term forecasting, called nowcasting, is used to monitor storms that pose a significant threat to people and infrastructure. These threats could include lightning strikes, hail, heavy precipitation, strong winds, and possible tornados. This paper proposes a fast approach to nowcasting lightning threats using simple statistical methods. The proposed model results in fast nowcasts that are more accurate than a competitive, computationally expensive, approach.
Ola Haug, Thordis L. Thorarinsdottir, Sigrunn H. Sørbye, and Christian L. E. Franzke
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 1–12, https://doi.org/10.5194/ascmo-6-1-2020, https://doi.org/10.5194/ascmo-6-1-2020, 2020
Short summary
Short summary
Trends in gridded temperature data are commonly assessed independently for each grid cell, ignoring spatial coherencies. This may severely affect the interpretation of the results. This article proposes a space–time model for temperatures that allows for joint assessments of the trend across locations. In a case study of summer season trends in Europe, it is found that the region with a significant trend under spatial coherency is vastly different from that under independent assessments.
Alexis Hannart
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 161–171, https://doi.org/10.5194/ascmo-5-161-2019, https://doi.org/10.5194/ascmo-5-161-2019, 2019
Short summary
Short summary
In climate change attribution studies, one often seeks to maximize a signal-to-noise ratio, where the
signalis the anthropogenic response and the
noiseis climate variability. A solution commonly used in D&A studies thus far consists of projecting the signal on the subspace spanned by the leading eigenvectors of climate variability. Here I show that this approach is vastly suboptimal – in fact, it leads instead to maximizing the noise-to-signal ratio. I then describe an improved solution.
Moritz N. Lang, Georg J. Mayr, Reto Stauffer, and Achim Zeileis
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 115–132, https://doi.org/10.5194/ascmo-5-115-2019, https://doi.org/10.5194/ascmo-5-115-2019, 2019
Short summary
Short summary
Accurate wind forecasts are of great importance for decision-making processes in today's society. This work presents a novel probabilistic post-processing method for wind vector forecasts employing a bivariate Gaussian response distribution. To capture a possible mismatch between the predicted and observed wind direction caused by location-specific properties, the approach incorporates a smooth rotation of the wind direction conditional on the season and the forecasted ensemble wind direction.
X. Joey Wang, John R. J. Thompson, W. John Braun, and Douglas G. Woolford
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 57–66, https://doi.org/10.5194/ascmo-5-57-2019, https://doi.org/10.5194/ascmo-5-57-2019, 2019
Short summary
Short summary
This paper presents the analysis of data from small-scale laboratory experimental smouldering fires that were digitally video-recorded. The video images of these fires bear a resemblance to remotely sensed images of wildfires and provide an opportunity to fit and assess a spatial model for fire spread that attempts to account for uncertainty in fire growth. We found that the fitting method is feasible, and the spatial model provides a suitable mathematical for the fire spread process.
Robin Tokmakian and Peter Challenor
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 17–35, https://doi.org/10.5194/ascmo-5-17-2019, https://doi.org/10.5194/ascmo-5-17-2019, 2019
Short summary
Short summary
As an example of how to robustly determine climate model uncertainty, the paper describes an experiment that perturbs the initial conditions for the ocean's temperature of a climate model. A total of 30 perturbed simulations are used (via an emulator) to estimate spatial uncertainties for temperature and precipitation fields. We also examined (using maximum covariance analysis) how ocean temperatures affect air temperatures and precipitation over land and the importance of feedback processes.
Thorsten Simon, Georg J. Mayr, Nikolaus Umlauf, and Achim Zeileis
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 1–16, https://doi.org/10.5194/ascmo-5-1-2019, https://doi.org/10.5194/ascmo-5-1-2019, 2019
Short summary
Short summary
Lightning in Alpine regions is associated with events such as thunderstorms,
extreme precipitation, high wind gusts, flash floods, and debris flows.
We present a statistical approach to predict lightning counts based on
numerical weather predictions. Lightning counts are considered on a grid
with 18 km mesh size. Skilful prediction is obtained for a forecast horizon
of 5 days over complex terrain.
Tony E. Wong
Adv. Stat. Clim. Meteorol. Oceanogr., 4, 53–63, https://doi.org/10.5194/ascmo-4-53-2018, https://doi.org/10.5194/ascmo-4-53-2018, 2018
Short summary
Short summary
Millions of people worldwide are at a risk of coastal flooding, and this number will increase as the climate continues to change. This study analyzes how climate change affects future flood hazards. A new model that uses multiple climate variables for flood hazard is developed. For the case study of Norfolk, Virginia, the model predicts 23 cm higher flood levels relative to previous work. This work shows the importance of accounting for climate change in effectively managing coastal risks.
Amy Braverman, Snigdhansu Chatterjee, Megan Heyman, and Noel Cressie
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 93–105, https://doi.org/10.5194/ascmo-3-93-2017, https://doi.org/10.5194/ascmo-3-93-2017, 2017
Short summary
Short summary
In this paper, we introduce a method for expressing the agreement between climate model output time series and time series of observational data as a probability value. Our metric is an estimate of the probability that one would obtain two time series as similar as the ones under consideration, if the climate model and the observed series actually shared the same underlying climate signal.
Joshua P. French, Seth McGinnis, and Armin Schwartzman
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 67–92, https://doi.org/10.5194/ascmo-3-67-2017, https://doi.org/10.5194/ascmo-3-67-2017, 2017
Short summary
Short summary
We assess the mean temperature effect of global and regional climate model combinations for the North American Regional Climate Change Assessment Program using varying classes of linear regression models, including possible interaction effects. We use both pointwise and simultaneous inference procedures to identify regions where global and regional climate model effects differ. We conclusively show that accounting for multiple comparisons is important for making proper inference.
László Varga and András Zempléni
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 55–66, https://doi.org/10.5194/ascmo-3-55-2017, https://doi.org/10.5194/ascmo-3-55-2017, 2017
Short summary
Short summary
This paper proposes a new generalisation of the block bootstrap methodology, which allows for any positive real number as expected block size. We use this bootstrap for determining the p values of a homogeneity test for copulas. The methods are applied to a temperature data set - we have found some significant changes in the dependence structure between the standardised temperature values of pairs of observation points within the Carpathian Basin.
Andrew Poppick, Elisabeth J. Moyer, and Michael L. Stein
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 33–53, https://doi.org/10.5194/ascmo-3-33-2017, https://doi.org/10.5194/ascmo-3-33-2017, 2017
Short summary
Short summary
We show that ostensibly empirical methods of analyzing trends in the global mean temperature record, which appear to de-emphasize assumptions, can nevertheless produce misleading inferences about trends and associated uncertainty. We illustrate how a simple but physically motivated trend model can provide better-fitting and more broadly applicable results, and show the importance of adequately characterizing internal variability for estimating trend uncertainty.
John Tipton, Mevin Hooten, and Simon Goring
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 1–16, https://doi.org/10.5194/ascmo-3-1-2017, https://doi.org/10.5194/ascmo-3-1-2017, 2017
Short summary
Short summary
We present a statistical framework for the reconstruction of historic temperature patterns from sparse, irregular data collected from observer stations. A common statistical technique for climate reconstruction uses modern era data as a set of temperature patterns that can be used to estimate the spatial temperature patterns. We present a framework for exploration of different assumptions about the sets of patterns used in the reconstruction while providing statistically rigorous estimates.
Georgina Davies and Noel Cressie
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 155–169, https://doi.org/10.5194/ascmo-2-155-2016, https://doi.org/10.5194/ascmo-2-155-2016, 2016
Short summary
Short summary
Sea surface temperature (SST) is a key component of global climate models, particularly in the tropical Pacific Ocean where El Niño and La Nina events have worldwide implications. In our paper, we analyse monthly SSTs in the Niño 3.4 region and find a transformation that removes a spatial mean-variance dependence for each month. For 10 out of 12 months in the year, the transformed monthly time series gave more accurate or as accurate forecasts than those from the untransformed time series.
Eric Gilleland, Melissa Bukovsky, Christopher L. Williams, Seth McGinnis, Caspar M. Ammann, Barbara G. Brown, and Linda O. Mearns
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 137–153, https://doi.org/10.5194/ascmo-2-137-2016, https://doi.org/10.5194/ascmo-2-137-2016, 2016
Short summary
Short summary
Several climate models are evaluated under current climate conditions to determine how well they are able to capture frequencies of severe-storm environments (conditions conducive for the formation of hail storms, tornadoes, etc.). They are found to underpredict the spatial extent of high-frequency areas (such as tornado alley), as well as underpredict the frequencies in the areas.
Whitney K. Huang, Michael L. Stein, David J. McInerney, Shanshan Sun, and Elisabeth J. Moyer
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 79–103, https://doi.org/10.5194/ascmo-2-79-2016, https://doi.org/10.5194/ascmo-2-79-2016, 2016
Sergei N. Rodionov
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 63–78, https://doi.org/10.5194/ascmo-2-63-2016, https://doi.org/10.5194/ascmo-2-63-2016, 2016
David Bolin, Arnoldo Frigessi, Peter Guttorp, Ola Haug, Elisabeth Orskaug, Ida Scheel, and Jonas Wallin
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 39–47, https://doi.org/10.5194/ascmo-2-39-2016, https://doi.org/10.5194/ascmo-2-39-2016, 2016
Julie Bessac, Pierre Ailliot, Julien Cattiaux, and Valerie Monbet
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 1–16, https://doi.org/10.5194/ascmo-2-1-2016, https://doi.org/10.5194/ascmo-2-1-2016, 2016
Short summary
Short summary
Several multi-site stochastic generators of zonal and meridional components of wind are proposed in this paper. Various questions are explored, such as the modeling of the regime in a multi-site context, the extraction of relevant clusterings from extra variables or from the local wind data, and the link between weather types extracted from wind data and large-scale weather regimes. We also discuss the relative advantages of hidden and observed regime-switching models.
E. M. Schliep, A. E. Gelfand, and D. M. Holland
Adv. Stat. Clim. Meteorol. Oceanogr., 1, 59–74, https://doi.org/10.5194/ascmo-1-59-2015, https://doi.org/10.5194/ascmo-1-59-2015, 2015
Short summary
Short summary
There is considerable demand for accurate air quality information in human health analyses. The sparsity of ground monitoring stations across the US motivates the need for advanced statistical models to predict air quality metrics. We propose a statistical model that jointly models ground-monitoring station data and satellite-obtained data allowing for temporal and spatial misalignment, missingness, and spatially and temporally varying correlation to enhance prediction of particulate matter.
R. Philbin and M. Jun
Adv. Stat. Clim. Meteorol. Oceanogr., 1, 29–44, https://doi.org/10.5194/ascmo-1-29-2015, https://doi.org/10.5194/ascmo-1-29-2015, 2015
T. K. Doan, J. Haslett, and A. C. Parnell
Adv. Stat. Clim. Meteorol. Oceanogr., 1, 15–27, https://doi.org/10.5194/ascmo-1-15-2015, https://doi.org/10.5194/ascmo-1-15-2015, 2015
W. B. Leeds, E. J. Moyer, and M. L. Stein
Adv. Stat. Clim. Meteorol. Oceanogr., 1, 1–14, https://doi.org/10.5194/ascmo-1-1-2015, https://doi.org/10.5194/ascmo-1-1-2015, 2015
Cited articles
Agnolucci, P., Rapti, C., Alexander, P., De Lipsis, V., Holland, R. A., Eigenbrod, F., and Ekins, P.: Impacts of rising temperatures and farm management practices on global yields of 18 crops, Nature Food, 1, 562–571, https://doi.org/10.1038/s43016-020-00148-x, 2020. a
Al Masry, Z., Pic, R., Dombry, C., and Devalland, C.: A new methodology to predict the oncotype scores based on clinico-pathological data with similar tumor profiles, Breast Cancer Res. Tr., https://doi.org/10.1007/s10549-023-07141-5, 2023. a
Alexander, C., Coulon, M., Han, Y., and Meng, X.: Evaluating the discrimination ability of proper multi-variate scoring rules, Ann. Oper. Res., 334, 857–883, https://doi.org/10.1007/s10479-022-04611-9, 2022. a
Allen, S.: sallen12/MultivCalibration: MultivCalibration v.1.0 (v.1.0), Zenodo [code, data set], https://doi.org/10.5281/zenodo.10201289, 2023. a
Allen, S., Bhend, J., Martius, O., and Ziegel, J.: Weighted Verification Tools to Evaluate Univariate and Multivariate Probabilistic Forecasts for High-Impact Weather Events, Weather Forecast., 38, 499–516, https://doi.org/10.1175/waf-d-22-0161.1, 2023a. a, b, c
Allen, S., Ginsbourger, D., and Ziegel, J.: Evaluating Forecasts for High-Impact Events Using Transformed Kernel Scores, SIAM/ASA Journal on Uncertainty Quantification, 11, 906–940, https://doi.org/10.1137/22m1532184, 2023b. a, b, c
Anderson, J. L.: A Method for Producing and Evaluating Probabilistic Forecasts from Ensemble Model Integrations, J. Climate, 9, 1518–1530, https://doi.org/10.1175/1520-0442(1996)009<1518:amfpae>2.0.co;2, 1996. a
Basse-O'Connor, A., Pilipauskaitė, V., and Podolskij, M.: Power variations for fractional type infinitely divisible random fields, Electron. J. Probab., 26, 1–35, https://doi.org/10.1214/21-EJP617, 2021. a
Ben Bouallègue, Z., Clare, M. C. A., Magnusson, L., Gascón, E., Maier-Gerber, M., Janoušek, M., Rodwell, M., Pinault, F., Dramsch, J. S., Lang, S. T. K., Raoult, B., Rabier, F., Chevallier, M., Sandu, I., Dueben, P., Chantry, M., and Pappenberger, F.: The Rise of Data-Driven Weather Forecasting: A First Statistical Assessment of Machine Learning–Based Weather Forecasts in an Operational-Like Context, B. Am. Meteorol. Soc., 105, E864–E883, https://doi.org/10.1175/bams-d-23-0162.1, 2024a. a
Ben Bouallègue, Z., Weyn, J. A., Clare, M. C. A., Dramsch, J., Dueben, P., and Chantry, M.: Improving Medium-Range Ensemble Weather Forecasts with Hierarchical Ensemble Transformers, Artificial Intelligence for the Earth Systems, 3, e230027, https://doi.org/10.1175/aies-d-23-0027.1, 2024b. a, b
Benassi, A., Cohen, S., and Istas, J.: On roughness indices for fractional fields, Bernoulli, 10, 357–373, https://doi.org/10.3150/bj/1082380223, 2004. a
Berlinet, A. and Thomas-Agnan, C.: Reproducing kernel Hilbert spaces in probability and statistics, with a preface by Persi Diaconis, Kluwer Academic Publishers, Boston, MA, ISBN 1-4020-7679-7, https://doi.org/10.1007/978-1-4419-9096-9, 2004. a
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., and Tian, Q.: Accurate medium-range global weather forecasting with 3D neural networks, Nature, 619, 533–538, https://doi.org/10.1038/s41586-023-06185-3, 2023. a
Bjerregård, M. B., Møller, J. K., and Madsen, H.: An introduction to multivariate probabilistic forecast evaluation, Energy and AI, 4, 100058, https://doi.org/10.1016/j.egyai.2021.100058, 2021. a, b
Bolin, D. and Wallin, J.: Local scale invariance and robustness of proper scoring rules, Stat. Science, 38, 140–159, https://doi.org/10.1214/22-sts864, 2023. a
Bosse, N. I., Abbott, S., Cori, A., van Leeuwen, E., Bracher, J., and Funk, S.: Scoring epidemiological forecasts on transformed scales, PLOS Comput. Biol., 19, e1011393, https://doi.org/10.1371/journal.pcbi.1011393, 2023. a
Brehmer, J.: Elicitability and its Application in Risk Management, arXiv [thesis], https://doi.org/10.48550/ARXIV.1707.09604, 2017. a
Brehmer, J. R. and Strokorb, K.: Why scoring functions cannot assess tail properties, Electronic Journal of Statistics, 13, https://doi.org/10.1214/19-ejs1622, 2019. a
Bremnes, J. B.: Ensemble Postprocessing Using Quantile Function Regression Based on Neural Networks and Bernstein Polynomials, Mon. Weather Rev., 148, 403–414, https://doi.org/10.1175/mwr-d-19-0227.1, 2019. a
Brier, G. W.: Verification of Forecasts Expressed in Terms of Probability, Mon. Weather Rev., 78, 1–3, https://doi.org/10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2, 1950. a
Bröcker, J.: Reliability, sufficiency, and the decomposition of proper scores, Q. J. Roy. Meteor. Soc., 135, 1512–1519, https://doi.org/10.1002/qj.456, 2009. a
Bröcker, J. and Ben Bouallègue, Z.: Stratified rank histograms for ensemble forecast verification under serial dependence, Q. J. Roy. Meteor. Soc., 146, 1976–1990, https://doi.org/10.1002/qj.3778, 2020. a
Bröcker, J. and Smith, L. A.: Scoring Probabilistic Forecasts: The Importance of Being Proper, Weather Forecast., 22, 382–388, https://doi.org/10.1175/waf966.1, 2007. a
Buschow, S.: Measuring Displacement Errors with Complex Wavelets, Weather Forecast., 37, 953–970, https://doi.org/10.1175/waf-d-21-0180.1, 2022. a
Buschow, S. and Friederichs, P.: Using wavelets to verify the scale structure of precipitation forecasts, Adv. Stat. Clim. Meteorol. Oceanogr., 6, 13–30, https://doi.org/10.5194/ascmo-6-13-2020, 2020. a
Buschow, S. and Friederichs, P.: SAD: Verifying the scale, anisotropy and direction of precipitation forecasts, Q. J. Roy. Meteor. Soc., 147, 1150–1169, https://doi.org/10.1002/qj.3964, 2021. a
Casati, B., Dorninger, M., Coelho, C. A. S., Ebert, E. E., Marsigli, C., Mittermaier, M. P., and Gilleland, E.: The 2020 International Verification Methods Workshop Online: Major Outcomes and Way Forward, B. Am. Meteorol. Soc., 103, E899–E910, https://doi.org/10.1175/bams-d-21-0126.1, 2022. a
Chapman, W. E., Delle Monache, L., Alessandrini, S., Subramanian, A. C., Ralph, F. M., Xie, S.-P., Lerch, S., and Hayatbini, N.: Probabilistic Predictions from Deterministic Atmospheric River Forecasts with Deep Learning, Mon. Weather Rev., 150, 215–234, https://doi.org/10.1175/mwr-d-21-0106.1, 2022. a
Chen, K., Han, T., Gong, J., Bai, L., Ling, F., Luo, J.-J., Chen, X., Ma, L., Zhang, T., Su, R., Ci, Y., Li, B., Yang, X., and Ouyang, W.: FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead, arXiv [preprint], https://doi.org/10.48550/ARXIV.2304.02948, 2023. a
Christensen, H. M., Moroz, I. M., and Palmer, T. N.: Evaluation of ensemble forecast uncertainty using a new proper score: Application to medium‐range and seasonal forecasts, Q. J. Roy. Meteor. Soc., 141, 538–549, https://doi.org/10.1002/qj.2375, 2014. a, b, c
Clark, M., Gangopadhyay, S., Hay, L., Rajagopalan, B., and Wilby, R.: The Schaake Shuffle: A Method for Reconstructing Space–Time Variability in Forecasted Precipitation and Temperature Fields, J. Hydrometeorol., 5, 243–262, https://doi.org/10.1175/1525-7541(2004)005<0243:tssamf>2.0.co;2, 2004. a
Dawid, A. P.: Present Position and Potential Developments: Some Personal Views: Statistical Theory: The Prequential Approach, J. R. Stat. Soc. Ser. A-G., 147, 278, https://doi.org/10.2307/2981683, 1984. a
Dawid, A. P. and Musio, M.: Theory and applications of proper scoring rules, METRON, 72, 169–183, https://doi.org/10.1007/s40300-014-0039-y, 2014. a
Dawid, A. P., Musio, M., and Ventura, L.: Minimum Scoring Rule Inference, Scand. J. Stat., 43, 123–138, https://doi.org/10.1111/sjos.12168, 2015. a
Delle Monache, L., Eckel, F. A., Rife, D. L., Nagarajan, B., and Searight, K.: Probabilistic Weather Prediction with an Analog Ensemble, Mon. Weather Rev., 141, 3498–3516, https://doi.org/10.1175/mwr-d-12-00281.1, 2013. a
Demaeyer, J.: EUPPBench postprocessing benchmark dataset – gridded data – Part I (v1.0), Zenodo [data set], https://doi.org/10.5281/zenodo.7429236, 2022. a
Demaeyer, J., Bhend, J., Lerch, S., Primo, C., Van Schaeybroeck, B., Atencia, A., Ben Bouallègue, Z., Chen, J., Dabernig, M., Evans, G., Faganeli Pucer, J., Hooper, B., Horat, N., Jobst, D., Merše, J., Mlakar, P., Möller, A., Mestre, O., Taillardat, M., and Vannitsem, S.: The EUPPBench postprocessing benchmark dataset v1.0, Earth Syst. Sci. Data, 15, 2635–2653, https://doi.org/10.5194/essd-15-2635-2023, 2023. a, b, c
Diebold, F. X. and Mariano, R. S.: Comparing Predictive Accuracy, J. Bus. Econ. Stat., 13, 253–263, https://doi.org/10.1080/07350015.1995.10524599, 1995. a, b
Ebert, E. E.: Fuzzy verification of high‐resolution gridded forecasts: a review and proposed framework, Meteorol. Appl., 15, 51–64, https://doi.org/10.1002/met.25, 2008. a
Ehm, W. and Gneiting, T.: Local proper scoring rules of order two, Ann. Stat., 40, 609–637, https://doi.org/10.1214/12-aos973, 2012. a
EUMETNET: MeteoAlarm, https://www.meteoalarm.org/en/live/, last access: 16 October 2024. a
Ferro, C. A. T., Richardson, D. S., and Weigel, A. P.: On the effect of ensemble size on the discrete and continuous ranked probability scores, Meteorol. Appl., 15, 19–24, https://doi.org/10.1002/met.45, 2008. a
Friederichs, P. and Hense, A.: A Probabilistic Forecast Approach for Daily Precipitation Totals, Weather Forecast., 23, 659–673, https://doi.org/10.1175/2007waf2007051.1, 2008. a, b
Gilleland, E.: Spatial Forecast Verification: Baddeley's Delta Metric Applied to the ICP Test Cases, Weather Forecast., 26, 409–415, https://doi.org/10.1175/waf-d-10-05061.1, 2011. a
Gilleland, E., Ahijevych, D., Brown, B. G., Casati, B., and Ebert, E. E.: Intercomparison of Spatial Forecast Verification Methods, Weather Forecast., 24, 1416–1430, https://doi.org/10.1175/2009waf2222269.1, 2009. a, b, c
Gneiting, T. and Katzfuss, M.: Probabilistic Forecasting, Annu. Rev. Stat. Appl., 1, 125–151, https://doi.org/10.1146/annurev-statistics-062713-085831, 2014. a, b
Gneiting, T., Balabdaoui, F., and Raftery, A. E.: Probabilistic Forecasts, Calibration and Sharpness, J. R. Stat. Soc. B, 69, 243–268, https://doi.org/10.1111/j.1467-9868.2007.00587.x, 2007. a, b
Gneiting, T., Stanberry, L. I., Grimit, E. P., Held, L., and Johnson, N. A.: Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds, TEST, 17, 211–235, https://doi.org/10.1007/s11749-008-0114-x, 2008. a, b
Gneiting, T., Lerch, S., and Schulz, B.: Probabilistic solar forecasting: Benchmarks, post-processing, verification, Sol. Energy, 252, 72–80, https://doi.org/10.1016/j.solener.2022.12.054, 2023. a
Good, I. J.: Rational Decisions, J. Roy. Stat. Soc. B Met., 14, 107–114, https://doi.org/10.1111/j.2517-6161.1952.tb00104.x, 1952. a, b
Han, F. and Szunyogh, I.: A Technique for the Verification of Precipitation Forecasts and Its Application to a Problem of Predictability, Mon. Weather Rev., 146, 1303–1318, https://doi.org/10.1175/mwr-d-17-0040.1, 2018. a
Heinrich‐Mertsching, C., Thorarinsdottir, T. L., Guttorp, P., and Schneider, M.: Validation of point process predictions with proper scoring rules, Scand. J. Stat., 51, 1533–1566, https://doi.org/10.1111/sjos.12736, 2024. a, b, c, d
Hersbach, H.: Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems, Weather Forecast., 15, 559–570, https://doi.org/10.1175/1520-0434(2000)015<0559:dotcrp>2.0.co;2, 2000. a
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati, G., Bidlot, J., Bonavita, M., De Chiara, G., Dahlgren, P., Dee, D., Diamantakis, M., Dragani, R., Flemming, J., Forbes, R., Fuentes, M., Geer, A., Haimberger, L., Healy, S., Hogan, R. J., Hólm, E., Janisková, M., Keeley, S., Laloyaux, P., Lopez, P., Lupu, C., Radnoti, G., de Rosnay, P., Rozum, I., Vamborg, F., Villaume, S., and Thépaut, J.: The ERA5 global reanalysis, Q. J. Roy. Meteor. Soc., 146, 1999–2049, https://doi.org/10.1002/qj.3803, 2020. a
Holzmann, H. and Eulert, M.: The role of the information set for forecasting – with applications to risk management, Ann. Appl. Stat., 8, 595–621, https://doi.org/10.1214/13-aoas709, 2014. a
Hu, W., Ghazvinian, M., Chapman, W. E., Sengupta, A., Ralph, F. M., and Delle Monache, L.: Deep Learning Forecast Uncertainty for Precipitation over the Western United States, Mon. Weather Rev., 151, 1367–1385, https://doi.org/10.1175/mwr-d-22-0268.1, 2023. a
Jolliffe, I. T. and Primo, C.: Evaluating Rank Histograms Using Decompositions of the Chi-Square Test Statistic, Mon. Weather Rev., 136, 2133–2139, https://doi.org/10.1175/2007mwr2219.1, 2008. a
Jordan, A., Krüger, F., and Lerch, S.: Evaluating Probabilistic Forecasts with scoringRules, J. Stat. Softw., 90, 1–37, https://doi.org/10.18637/jss.v090.i12, 2019. a, b
Jordan, T. H., Chen, Y.-T., Gasparini, P., Madariaga, R., Main, I., Marzocchi, W., Papadopoulos, G., Sobolev, G., Yamaoka, K., and Zschau, J.: OPERATIONAL EARTHQUAKE FORECASTING. State of Knowledge and Guidelines for Utilization, Ann. Geophys.-Italy, 54, 316–391, https://doi.org/10.4401/ag-5350, 2011. a
Jose, V. R.: A Characterization for the Spherical Scoring Rule, Theor. Decis., 66, 263–281, https://doi.org/10.1007/s11238-007-9067-x, 2007. a
Keisler, R.: Forecasting Global Weather with Graph Neural Networks, arXiv [preprint], https://doi.org/10.48550/ARXIV.2202.07575, 2022. a
Kullback, S. and Leibler, R. A.: On Information and Sufficiency, Ann. Math. Stat., 22, 79–86, https://doi.org/10.1214/aoms/1177729694, 1951. a
Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., Merose, A., Hoyer, S., Holland, G., Vinyals, O., Stott, J., Pritzel, A., Mohamed, S., and Battaglia, P.: GraphCast: Learning skillful medium-range global weather forecasting, arXiv [preprint], https://doi.org/10.48550/ARXIV.2212.12794, 2022. a
Lerch, S. and Thorarinsdottir, T. L.: Comparison of non-homogeneous regression models for probabilistic wind speed forecasting, Tellus A, 65, 21206, https://doi.org/10.3402/tellusa.v65i0.21206, 2013. a
Lerch, S., Thorarinsdottir, T. L., Ravazzolo, F., and Gneiting, T.: Forecaster's Dilemma: Extreme Events and Forecast Evaluation, Stat. Sci., 32, 106–127, https://doi.org/10.1214/16-sts588, 2017. a
Matheron, G.: Principles of geostatistics, Econ. Geol., 58, 1246–1266, https://doi.org/10.2113/gsecongeo.58.8.1246, 1963. a, b
Matheson, J. E. and Winkler, R. L.: Scoring Rules for Continuous Probability Distributions, Manage. Sci., 22, 1087–1096, 1976. a
Meng, X., Taylor, J. W., Ben Taieb, S., and Li, S.: Scores for Multivariate Distributions and Level Sets, Oper. Res., 344–362, https://doi.org/10.1287/opre.2020.0365, 2023. a
Murphy, A. H. and Winkler, R. L.: A General Framework for Forecast Verification, Mon. Weather Rev., 115, 1330–1338, https://doi.org/10.1175/1520-0493(1987)115<1330:agfffv>2.0.co;2, 1987. a
Nowotarski, J. and Weron, R.: Recent advances in electricity price forecasting: A review of probabilistic forecasting, Renew. Sust. Energ. Rev., 81, 1548–1568, https://doi.org/10.1016/j.rser.2017.05.234, 2018. a
Palmer, T. N.: Towards the probabilistic Earth‐system simulator: a vision for the future of climate and weather prediction, Q. J. Roy. Meteor. Soc., 138, 841–861, https://doi.org/10.1002/qj.1923, 2012. a
Parry, M., Dawid, A. P., and Lauritzen, S.: Proper local scoring rules, Ann. Stat., 40, 561–592, https://doi.org/10.1214/12-aos971, 2012. a, b, c
Pathak, J., Subramanian, S., Harrington, P., Raja, S., Chattopadhyay, A., Mardani, M., Kurth, T., Hall, D., Li, Z., Azizzadenesheli, K., Hassanzadeh, P., Kashinath, K., and Anandkumar, A.: FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators, arXiv [preprint], https://doi.org/10.48550/arXiv.2202.11214, 2022. a, b
Pic, R.: aggregation-transformation, Zenodo [code], https://doi.org/10.5281/zenodo.14982271, 2024. a
Pinson, P.: Wind Energy: Forecasting Challenges for Its Operational Management, Stat. Sci., 28, 564–585, https://doi.org/10.1214/13-sts445, 2013. a
Pinson, P. and Girard, R.: Evaluating the quality of scenarios of short-term wind power generation, Appl. Energ., 96, 12–20, https://doi.org/10.1016/j.apenergy.2011.11.004, 2012. a, b
Radanovics, S., Vidal, J.-P., and Sauquet, E.: Spatial Verification of Ensemble Precipitation: An Ensemble Version of SAL, Weather Forecast., 33, 1001–1020, https://doi.org/10.1175/waf-d-17-0162.1, 2018. a, b
Rasp, S. and Lerch, S.: Neural Networks for Postprocessing Ensemble Weather Forecasts, Mon. Weather Rev., 146, 3885–3900, https://doi.org/10.1175/mwr-d-18-0187.1, 2018. a
Rasp, S., Hoyer, S., Merose, A., Langmore, I., Battaglia, P., Russel, T., Sanchez-Gonzalez, A., Yang, V., Carver, R., Agrawal, S., Chantry, M., Ben Bouallègue, Z., Dueben, P., Bromberg, C., Sisk, J., Barrington, L., Bell, A., and Sha, F.: WeatherBench 2: A Benchmark for the Next Generation of Data-Driven Global Weather Models, J. Adv. Model. Earth Sy., 16, e2023MS004019, https://doi.org/10.1029/2023MS004019, 2024. a
Rivoire, P., Martius, O., Naveau, P., and Tuel, A.: Assessment of subseasonal-to-seasonal (S2S) ensemble extreme precipitation forecast skill over Europe, Nat. Hazards Earth Syst. Sci., 23, 2857–2871, https://doi.org/10.5194/nhess-23-2857-2023, 2023. a
Roberts, N. M. and Lean, H. W.: Scale-Selective Verification of Rainfall Accumulations from High-Resolution Forecasts of Convective Events, Mon. Weather Rev., 136, 78–97, https://doi.org/10.1175/2007mwr2123.1, 2008. a
Roulston, M. S. and Smith, L. A.: Evaluating Probabilistic Forecasts Using Information Theory, Mon. Weather Rev., 130, 1653–1660, https://doi.org/10.1175/1520-0493(2002)130<1653:epfuit>2.0.co;2, 2002. a
Schefzik, R., Thorarinsdottir, T. L., and Gneiting, T.: Uncertainty Quantification in Complex Simulation Models Using Ensemble Copula Coupling, Stat. Sci., 28, 616–640, https://doi.org/10.1214/13-sts443, 2013. a
Scheuerer, M. and Möller, D.: Probabilistic wind speed forecasting on a grid based on ensemble model output statistics, Ann. Appl. Stat., 9, 1328–1349, https://doi.org/10.1214/15-aoas843, 2015. a
Schlather, M., Malinowski, A., Menck, P. J., Oesting, M., and Strokorb, K.: Analysis, Simulation and Prediction of Multivariate Random Fields with PackageRandomFields, J. Stat. Softw., 63, 1–25, https://doi.org/10.18637/jss.v063.i08, 2015. a
Schorlemmer, D., Werner, M. J., Marzocchi, W., Jordan, T. H., Ogata, Y., Jackson, D. D., Mak, S., Rhoades, D. A., Gerstenberger, M. C., Hirata, N., Liukis, M., Maechling, P. J., Strader, A., Taroni, M., Wiemer, S., Zechar, J. D., and Zhuang, J.: The Collaboratory for the Study of Earthquake Predictability: Achievements and Priorities, Seismol. Res. Lett., 89, 1305–1313, https://doi.org/10.1785/0220180053, 2018. a
Schulz, B. and Lerch, S.: Machine Learning Methods for Postprocessing Ensemble Forecasts of Wind Gusts: A Systematic Comparison, Mon. Weather Rev., 150, 235–257, https://doi.org/10.1175/mwr-d-21-0150.1, 2022. a
Shannon, C. E.: A Mathematical Theory of Communication, Bell Syst. Tech. J., 27, 623–656, https://doi.org/10.1002/j.1538-7305.1948.tb00917.x, 1948. a
Smola, A., Gretton, A., Song, L., and Schölkopf, B.: A Hilbert Space Embedding for Distributions, in: Algorithmic Learning Theory, edited by: Hutter, M., Servedio, R. A., and Takimoto, E., 13–31, Springer Berlin Heidelberg, Berlin, Heidelberg, ISBN 978-3-540-75225-7, 2007. a
Stein, J. and Stoop, F.: Neighborhood-Based Ensemble Evaluation Using the CRPS, Mon. Weather Rev., 150, 1901–1914, https://doi.org/10.1175/mwr-d-21-0224.1, 2022. a
Steinwart, I. and Christmann, A.: Support Vector Machines, Information Science and Statistics, Springer, New York, ISBN 978-0-387-77241-7, 2008. a
Steinwart, I. and Ziegel, J. F.: Strictly proper kernel scores and characteristic kernels on compact spaces, Appl. Comput. Harmon. A., 51, 510–542, https://doi.org/10.1016/j.acha.2019.11.005, 2021. a
Székely, G.: E-statistics: The Energy of Statistical Samples, techreport, Bowling Green State University, https://doi.org/10.13140/RG.2.1.5063.9761, 2003. a
Taillardat, M.: Skewed and Mixture of Gaussian Distributions for Ensemble Postprocessing, Atmosphere, 12, 966, https://doi.org/10.3390/atmos12080966, 2021. a
Taillardat, M. and Mestre, O.: From research to applications – examples of operational ensemble post-processing in France using machine learning, Nonlin. Processes Geophys., 27, 329–347, https://doi.org/10.5194/npg-27-329-2020, 2020. a
Taillardat, M., Mestre, O., Zamo, M., and Naveau, P.: Calibrated Ensemble Forecasts Using Quantile Regression Forests and Ensemble Model Output Statistics, Mon. Weather Rev., 144, 2375–2393, https://doi.org/10.1175/mwr-d-15-0260.1, 2016. a
Talagrand, O., Vautard, R., and Strauss, B.: Evaluation of probabilistic prediction systems, in: Workshop on Predictability, 20–22 October 1997, 1–26, ECMWF, Shinfield Park, Reading, 1997. a
Thorarinsdottir, T. L. and Schuhen, N.: Verification: Assessment of Calibration and Accuracy, 155–186, Elsevier, https://doi.org/10.1016/b978-0-12-812372-0.00006-6, 2018. a, b, c, d
Thorarinsdottir, T. L., Gneiting, T., and Gissibl, N.: Using Proper Divergence Functions to Evaluate Climate Models, SIAM/ASA Journal on Uncertainty Quantification, 1, 522–534, https://doi.org/10.1137/130907550, 2013. a
Tsyplakov, A.: Evaluating Density Forecasts: A Comment, SSRN Electronic Journal, 1907799, https://doi.org/10.2139/ssrn.1907799, 2011. a
Tsyplakov, A.: Evaluation of Probabilistic Forecasts: Proper Scoring Rules and Moments, SSRN Electronic Journal, 2236605, https://doi.org/10.2139/ssrn.2236605, 2013. a
Tsyplakov, A.: Evaluation of probabilistic forecasts: Conditional auto-calibration, https://www.sas.upenn.edu/~fdiebold/papers2/Tsyplakov_Auto_calibration_sent_eswc2020.pdf (last access: 6 March 2025), 2020. a
Vannitsem, S., Bremnes, J. B., Demaeyer, J., Evans, G. R., Flowerdew, J., Hemri, S., Lerch, S., Roberts, N., Theis, S., Atencia, A., Ben Bouallègue, Z., Bhend, J., Dabernig, M., De Cruz, L., Hieta, L., Mestre, O., Moret, L., Plenković, I. O., Schmeits, M., Taillardat, M., Van den Bergh, J., Van Schaeybroeck, B., Whan, K., and Ylhaisi, J.: Statistical Postprocessing for Weather Forecasts: Review, Challenges, and Avenues in a Big Data World, B. Am. Meteorol. Soc., 102, E681–E699, https://doi.org/10.1175/bams-d-19-0308.1, 2021. a
Wernli, H., Paulat, M., Hagen, M., and Frei, C.: SAL – A Novel Quality Measure for the Verification of Quantitative Precipitation Forecasts, Mon. Weather Rev., 136, 4470–4487, https://doi.org/10.1175/2008mwr2415.1, 2008. a, b
Winkelbauer, A.: Moments and Absolute Moments of the Normal Distribution, arXiv [preprint], https://doi.org/10.48550/ARXIV.1209.4340, 2014. a
Winkler, R. L.: Rewarding Expertise in Probability Assessment, 127–140, Springer Netherlands, ISBN 9789401012768, https://doi.org/10.1007/978-94-010-1276-8_10, 1977. a, b
Winkler, R. L., Muñoz, J., Cervera, J. L., Bernardo, J. M., Blattenberger, G., Kadane, J. B., Lindley, D. V., Murphy, A. H., Oliver, R. M., and Ríos-Insua, D.: Scoring rules and the evaluation of probabilities, Test, 5, 1–60, https://doi.org/10.1007/bf02562681, 1996. a, b
Zamo, M. and Naveau, P.: Estimation of the Continuous Ranked Probability Score with Limited Information and Applications to Ensemble Weather Forecasts, Math. Geosci., 50, 209–234, https://doi.org/10.1007/s11004-017-9709-7, 2017. a
Ziel, F. and Berk, K.: Multivariate Forecasting Evaluation: On Sensitive and Strictly Proper Scoring Rules, arXiv [preprint], https://doi.org/10.48550/arXiv.1910.07325, 2019. a, b, c
Short summary
Correctly forecasting weather is crucial for decision-making in various fields. Standard multivariate verification tools have limitations, and a single tool cannot fully characterize predictive performance. We formalize a framework based on aggregation and transformation to build interpretable verification tools. These tools target specific features of forecasts, improving predictive performance characterization and bridging the gap between theoretical and physics-based tools.
Correctly forecasting weather is crucial for decision-making in various fields. Standard...
