Articles | Volume 7, issue 1
Adv. Stat. Clim. Meteorol. Oceanogr., 7, 13–34, 2021
https://doi.org/10.5194/ascmo-7-13-2021
© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Adv. Stat. Clim. Meteorol. Oceanogr., 7, 13–34, 2021
https://doi.org/10.5194/ascmo-7-13-2021
© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
16 Feb 2021
16 Feb 2021
Novel measures for summarizing high-resolution forecast performance
Eric Gilleland
Related authors
Manuela I. Brunner, Eric Gilleland, and Andrew W. Wood
Earth Syst. Dynam., 12, 621–634, https://doi.org/10.5194/esd-12-621-2021, https://doi.org/10.5194/esd-12-621-2021, 2021
Short summary
Short summary
Compound hot and dry events can lead to severe impacts whose severity may depend on their timescale and spatial extent. Here, we show that the spatial extent and timescale of compound hot–dry events are strongly related, spatial compound event extents are largest at
sub-seasonal timescales, and short events are driven more by high temperatures, while longer events are more driven by low precipitation. Future climate impact studies should therefore be performed at different timescales.
Chiara Marsigli, Elizabeth Ebert, Raghavendra Ashrit, Barbara Casati, Jing Chen, Caio A. S. Coelho, Manfred Dorninger, Eric Gilleland, Thomas Haiden, Stephanie Landman, and Marion Mittermaier
Nat. Hazards Earth Syst. Sci., 21, 1297–1312, https://doi.org/10.5194/nhess-21-1297-2021, https://doi.org/10.5194/nhess-21-1297-2021, 2021
Short summary
Short summary
This paper reviews new observations for the verification of high-impact weather and provides advice for their usage in objective verification. New observations include remote sensing datasets, products developed for nowcasting, datasets derived from telecommunication systems, data collected from citizens, reports of impacts and reports from insurance companies. This work has been performed in the framework of the Joint Working Group on Forecast Verification Research (JWGFVR) of the WMO.
Manuela I. Brunner and Eric Gilleland
Hydrol. Earth Syst. Sci., 24, 3967–3982, https://doi.org/10.5194/hess-24-3967-2020, https://doi.org/10.5194/hess-24-3967-2020, 2020
Short summary
Short summary
Stochastically generated streamflow time series are used for various water management and hazard estimation applications. They provide realizations of plausible but yet unobserved streamflow time series with the same characteristics as the observed data. We propose a stochastic simulation approach in the frequency domain instead of the time domain. Our evaluation results suggest that the flexible, continuous simulation approach is valuable for a diverse range of water management applications.
Joshua North, Zofia Stanley, William Kleiber, Wiebke Deierling, Eric Gilleland, and Matthias Steiner
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 79–90, https://doi.org/10.5194/ascmo-6-79-2020, https://doi.org/10.5194/ascmo-6-79-2020, 2020
Short summary
Short summary
Very short-term forecasting, called nowcasting, is used to monitor storms that pose a significant threat to people and infrastructure. These threats could include lightning strikes, hail, heavy precipitation, strong winds, and possible tornados. This paper proposes a fast approach to nowcasting lightning threats using simple statistical methods. The proposed model results in fast nowcasts that are more accurate than a competitive, computationally expensive, approach.
Eric Gilleland, Melissa Bukovsky, Christopher L. Williams, Seth McGinnis, Caspar M. Ammann, Barbara G. Brown, and Linda O. Mearns
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 137–153, https://doi.org/10.5194/ascmo-2-137-2016, https://doi.org/10.5194/ascmo-2-137-2016, 2016
Short summary
Short summary
Several climate models are evaluated under current climate conditions to determine how well they are able to capture frequencies of severe-storm environments (conditions conducive for the formation of hail storms, tornadoes, etc.). They are found to underpredict the spatial extent of high-frequency areas (such as tornado alley), as well as underpredict the frequencies in the areas.
Manuela I. Brunner, Eric Gilleland, and Andrew W. Wood
Earth Syst. Dynam., 12, 621–634, https://doi.org/10.5194/esd-12-621-2021, https://doi.org/10.5194/esd-12-621-2021, 2021
Short summary
Short summary
Compound hot and dry events can lead to severe impacts whose severity may depend on their timescale and spatial extent. Here, we show that the spatial extent and timescale of compound hot–dry events are strongly related, spatial compound event extents are largest at
sub-seasonal timescales, and short events are driven more by high temperatures, while longer events are more driven by low precipitation. Future climate impact studies should therefore be performed at different timescales.
Chiara Marsigli, Elizabeth Ebert, Raghavendra Ashrit, Barbara Casati, Jing Chen, Caio A. S. Coelho, Manfred Dorninger, Eric Gilleland, Thomas Haiden, Stephanie Landman, and Marion Mittermaier
Nat. Hazards Earth Syst. Sci., 21, 1297–1312, https://doi.org/10.5194/nhess-21-1297-2021, https://doi.org/10.5194/nhess-21-1297-2021, 2021
Short summary
Short summary
This paper reviews new observations for the verification of high-impact weather and provides advice for their usage in objective verification. New observations include remote sensing datasets, products developed for nowcasting, datasets derived from telecommunication systems, data collected from citizens, reports of impacts and reports from insurance companies. This work has been performed in the framework of the Joint Working Group on Forecast Verification Research (JWGFVR) of the WMO.
Manuela I. Brunner and Eric Gilleland
Hydrol. Earth Syst. Sci., 24, 3967–3982, https://doi.org/10.5194/hess-24-3967-2020, https://doi.org/10.5194/hess-24-3967-2020, 2020
Short summary
Short summary
Stochastically generated streamflow time series are used for various water management and hazard estimation applications. They provide realizations of plausible but yet unobserved streamflow time series with the same characteristics as the observed data. We propose a stochastic simulation approach in the frequency domain instead of the time domain. Our evaluation results suggest that the flexible, continuous simulation approach is valuable for a diverse range of water management applications.
Joshua North, Zofia Stanley, William Kleiber, Wiebke Deierling, Eric Gilleland, and Matthias Steiner
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 79–90, https://doi.org/10.5194/ascmo-6-79-2020, https://doi.org/10.5194/ascmo-6-79-2020, 2020
Short summary
Short summary
Very short-term forecasting, called nowcasting, is used to monitor storms that pose a significant threat to people and infrastructure. These threats could include lightning strikes, hail, heavy precipitation, strong winds, and possible tornados. This paper proposes a fast approach to nowcasting lightning threats using simple statistical methods. The proposed model results in fast nowcasts that are more accurate than a competitive, computationally expensive, approach.
Eric Gilleland, Melissa Bukovsky, Christopher L. Williams, Seth McGinnis, Caspar M. Ammann, Barbara G. Brown, and Linda O. Mearns
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 137–153, https://doi.org/10.5194/ascmo-2-137-2016, https://doi.org/10.5194/ascmo-2-137-2016, 2016
Short summary
Short summary
Several climate models are evaluated under current climate conditions to determine how well they are able to capture frequencies of severe-storm environments (conditions conducive for the formation of hail storms, tornadoes, etc.). They are found to underpredict the spatial extent of high-frequency areas (such as tornado alley), as well as underpredict the frequencies in the areas.
Related subject area
Statistics
Copula approach for simulated damages caused by landfalling US hurricanes
Nonstationary extreme value analysis for event attribution combining climate models and observations
Comparing climate time series – Part 1: Univariate test
A statistical approach to fast nowcasting of lightning potential fields
Spatial trend analysis of gridded temperature data at varying spatial scales
An improved projection of climate observations for detection and attribution
Bivariate Gaussian models for wind vectors in a distributional regression framework
Fitting a stochastic fire spread model to data
Influence of initial ocean conditions on temperature and precipitation in a coupled climate model's solution
NWP-based lightning prediction using flexible count data regression
An integration and assessment of multiple covariates of nonstationary storm surge statistical behavior by Bayesian model averaging
Probabilistic evaluation of competing climate models
Assessing NARCCAP climate model effects using spatial confidence regions
Generalised block bootstrap and its use in meteorology
Estimating trends in the global mean temperature record
Reconstruction of spatio-temporal temperature from sparse historical records using robust probabilistic principal component regression
Analysis of variability of tropical Pacific sea surface temperatures
Evaluating NARCCAP model performance for frequencies of severe-storm environments
Estimating changes in temperature extremes from millennial-scale climate simulations using generalized extreme value (GEV) distributions
A comparison of two methods for detecting abrupt changes in the variance of climatic time series
Calibrating regionally downscaled precipitation over Norway through quantile-based approaches
Comparison of hidden and observed regime-switching autoregressive models for (u, v)-components of wind fields in the northeastern Atlantic
Autoregressive spatially varying coefficients model for predicting daily PM2.5 using VIIRS satellite AOT
Bivariate spatial analysis of temperature and precipitation from general circulation models and observation proxies
Joint inference of misaligned irregular time series with application to Greenland ice core data
Simulation of future climate under changing temporal covariance structures
Thomas Patrick Leahy
Adv. Stat. Clim. Meteorol. Oceanogr., 7, 1–11, https://doi.org/10.5194/ascmo-7-1-2021, https://doi.org/10.5194/ascmo-7-1-2021, 2021
Short summary
Short summary
This study looked at estimating damages caused by hurricanes in the United States. It assessed the relationship between the maximum wind speed at landfall and the resulting damage caused. The study found that the complex processes that determine the size of the damages inflicted could be estimated using this simple relationship. This work could be used to examine how often extreme damage events are likely to occur and the impact of stronger hurricane winds on the US Atlantic and Gulf coasts.
Yoann Robin and Aurélien Ribes
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 205–221, https://doi.org/10.5194/ascmo-6-205-2020, https://doi.org/10.5194/ascmo-6-205-2020, 2020
Short summary
Short summary
We have developed a new statistical method to describe how a severe weather event, such as a heat wave, may have been influenced by climate change. Our method incorporates both observations and data from various climate models to reflect climate model uncertainty. Our results show that both the probability and the intensity of the French July 2019 heatwave have increased significantly in response to human influence. We find that this heat wave might not have been possible without climate change.
Timothy DelSole and Michael K. Tippett
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 159–175, https://doi.org/10.5194/ascmo-6-159-2020, https://doi.org/10.5194/ascmo-6-159-2020, 2020
Short summary
Short summary
Scientists often are confronted with the question of whether two time series are statistically distinguishable. This paper proposes a test for answering this question. The basic idea is to fit each time series to a time series model and then test whether the parameters in that model are equal. If a difference is detected, then new ways of visualizing those differences are proposed, including a clustering technique and a method based on optimal initial conditions.
Joshua North, Zofia Stanley, William Kleiber, Wiebke Deierling, Eric Gilleland, and Matthias Steiner
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 79–90, https://doi.org/10.5194/ascmo-6-79-2020, https://doi.org/10.5194/ascmo-6-79-2020, 2020
Short summary
Short summary
Very short-term forecasting, called nowcasting, is used to monitor storms that pose a significant threat to people and infrastructure. These threats could include lightning strikes, hail, heavy precipitation, strong winds, and possible tornados. This paper proposes a fast approach to nowcasting lightning threats using simple statistical methods. The proposed model results in fast nowcasts that are more accurate than a competitive, computationally expensive, approach.
Ola Haug, Thordis L. Thorarinsdottir, Sigrunn H. Sørbye, and Christian L. E. Franzke
Adv. Stat. Clim. Meteorol. Oceanogr., 6, 1–12, https://doi.org/10.5194/ascmo-6-1-2020, https://doi.org/10.5194/ascmo-6-1-2020, 2020
Short summary
Short summary
Trends in gridded temperature data are commonly assessed independently for each grid cell, ignoring spatial coherencies. This may severely affect the interpretation of the results. This article proposes a space–time model for temperatures that allows for joint assessments of the trend across locations. In a case study of summer season trends in Europe, it is found that the region with a significant trend under spatial coherency is vastly different from that under independent assessments.
Alexis Hannart
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 161–171, https://doi.org/10.5194/ascmo-5-161-2019, https://doi.org/10.5194/ascmo-5-161-2019, 2019
Short summary
Short summary
In climate change attribution studies, one often seeks to maximize a signal-to-noise ratio, where the
signalis the anthropogenic response and the
noiseis climate variability. A solution commonly used in D&A studies thus far consists of projecting the signal on the subspace spanned by the leading eigenvectors of climate variability. Here I show that this approach is vastly suboptimal – in fact, it leads instead to maximizing the noise-to-signal ratio. I then describe an improved solution.
Moritz N. Lang, Georg J. Mayr, Reto Stauffer, and Achim Zeileis
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 115–132, https://doi.org/10.5194/ascmo-5-115-2019, https://doi.org/10.5194/ascmo-5-115-2019, 2019
Short summary
Short summary
Accurate wind forecasts are of great importance for decision-making processes in today's society. This work presents a novel probabilistic post-processing method for wind vector forecasts employing a bivariate Gaussian response distribution. To capture a possible mismatch between the predicted and observed wind direction caused by location-specific properties, the approach incorporates a smooth rotation of the wind direction conditional on the season and the forecasted ensemble wind direction.
X. Joey Wang, John R. J. Thompson, W. John Braun, and Douglas G. Woolford
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 57–66, https://doi.org/10.5194/ascmo-5-57-2019, https://doi.org/10.5194/ascmo-5-57-2019, 2019
Short summary
Short summary
This paper presents the analysis of data from small-scale laboratory experimental smouldering fires that were digitally video-recorded. The video images of these fires bear a resemblance to remotely sensed images of wildfires and provide an opportunity to fit and assess a spatial model for fire spread that attempts to account for uncertainty in fire growth. We found that the fitting method is feasible, and the spatial model provides a suitable mathematical for the fire spread process.
Robin Tokmakian and Peter Challenor
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 17–35, https://doi.org/10.5194/ascmo-5-17-2019, https://doi.org/10.5194/ascmo-5-17-2019, 2019
Short summary
Short summary
As an example of how to robustly determine climate model uncertainty, the paper describes an experiment that perturbs the initial conditions for the ocean's temperature of a climate model. A total of 30 perturbed simulations are used (via an emulator) to estimate spatial uncertainties for temperature and precipitation fields. We also examined (using maximum covariance analysis) how ocean temperatures affect air temperatures and precipitation over land and the importance of feedback processes.
Thorsten Simon, Georg J. Mayr, Nikolaus Umlauf, and Achim Zeileis
Adv. Stat. Clim. Meteorol. Oceanogr., 5, 1–16, https://doi.org/10.5194/ascmo-5-1-2019, https://doi.org/10.5194/ascmo-5-1-2019, 2019
Short summary
Short summary
Lightning in Alpine regions is associated with events such as thunderstorms,
extreme precipitation, high wind gusts, flash floods, and debris flows.
We present a statistical approach to predict lightning counts based on
numerical weather predictions. Lightning counts are considered on a grid
with 18 km mesh size. Skilful prediction is obtained for a forecast horizon
of 5 days over complex terrain.
Tony E. Wong
Adv. Stat. Clim. Meteorol. Oceanogr., 4, 53–63, https://doi.org/10.5194/ascmo-4-53-2018, https://doi.org/10.5194/ascmo-4-53-2018, 2018
Short summary
Short summary
Millions of people worldwide are at a risk of coastal flooding, and this number will increase as the climate continues to change. This study analyzes how climate change affects future flood hazards. A new model that uses multiple climate variables for flood hazard is developed. For the case study of Norfolk, Virginia, the model predicts 23 cm higher flood levels relative to previous work. This work shows the importance of accounting for climate change in effectively managing coastal risks.
Amy Braverman, Snigdhansu Chatterjee, Megan Heyman, and Noel Cressie
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 93–105, https://doi.org/10.5194/ascmo-3-93-2017, https://doi.org/10.5194/ascmo-3-93-2017, 2017
Short summary
Short summary
In this paper, we introduce a method for expressing the agreement between climate model output time series and time series of observational data as a probability value. Our metric is an estimate of the probability that one would obtain two time series as similar as the ones under consideration, if the climate model and the observed series actually shared the same underlying climate signal.
Joshua P. French, Seth McGinnis, and Armin Schwartzman
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 67–92, https://doi.org/10.5194/ascmo-3-67-2017, https://doi.org/10.5194/ascmo-3-67-2017, 2017
Short summary
Short summary
We assess the mean temperature effect of global and regional climate model combinations for the North American Regional Climate Change Assessment Program using varying classes of linear regression models, including possible interaction effects. We use both pointwise and simultaneous inference procedures to identify regions where global and regional climate model effects differ. We conclusively show that accounting for multiple comparisons is important for making proper inference.
László Varga and András Zempléni
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 55–66, https://doi.org/10.5194/ascmo-3-55-2017, https://doi.org/10.5194/ascmo-3-55-2017, 2017
Short summary
Short summary
This paper proposes a new generalisation of the block bootstrap methodology, which allows for any positive real number as expected block size. We use this bootstrap for determining the p values of a homogeneity test for copulas. The methods are applied to a temperature data set - we have found some significant changes in the dependence structure between the standardised temperature values of pairs of observation points within the Carpathian Basin.
Andrew Poppick, Elisabeth J. Moyer, and Michael L. Stein
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 33–53, https://doi.org/10.5194/ascmo-3-33-2017, https://doi.org/10.5194/ascmo-3-33-2017, 2017
Short summary
Short summary
We show that ostensibly empirical methods of analyzing trends in the global mean temperature record, which appear to de-emphasize assumptions, can nevertheless produce misleading inferences about trends and associated uncertainty. We illustrate how a simple but physically motivated trend model can provide better-fitting and more broadly applicable results, and show the importance of adequately characterizing internal variability for estimating trend uncertainty.
John Tipton, Mevin Hooten, and Simon Goring
Adv. Stat. Clim. Meteorol. Oceanogr., 3, 1–16, https://doi.org/10.5194/ascmo-3-1-2017, https://doi.org/10.5194/ascmo-3-1-2017, 2017
Short summary
Short summary
We present a statistical framework for the reconstruction of historic temperature patterns from sparse, irregular data collected from observer stations. A common statistical technique for climate reconstruction uses modern era data as a set of temperature patterns that can be used to estimate the spatial temperature patterns. We present a framework for exploration of different assumptions about the sets of patterns used in the reconstruction while providing statistically rigorous estimates.
Georgina Davies and Noel Cressie
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 155–169, https://doi.org/10.5194/ascmo-2-155-2016, https://doi.org/10.5194/ascmo-2-155-2016, 2016
Short summary
Short summary
Sea surface temperature (SST) is a key component of global climate models, particularly in the tropical Pacific Ocean where El Niño and La Nina events have worldwide implications. In our paper, we analyse monthly SSTs in the Niño 3.4 region and find a transformation that removes a spatial mean-variance dependence for each month. For 10 out of 12 months in the year, the transformed monthly time series gave more accurate or as accurate forecasts than those from the untransformed time series.
Eric Gilleland, Melissa Bukovsky, Christopher L. Williams, Seth McGinnis, Caspar M. Ammann, Barbara G. Brown, and Linda O. Mearns
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 137–153, https://doi.org/10.5194/ascmo-2-137-2016, https://doi.org/10.5194/ascmo-2-137-2016, 2016
Short summary
Short summary
Several climate models are evaluated under current climate conditions to determine how well they are able to capture frequencies of severe-storm environments (conditions conducive for the formation of hail storms, tornadoes, etc.). They are found to underpredict the spatial extent of high-frequency areas (such as tornado alley), as well as underpredict the frequencies in the areas.
Whitney K. Huang, Michael L. Stein, David J. McInerney, Shanshan Sun, and Elisabeth J. Moyer
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 79–103, https://doi.org/10.5194/ascmo-2-79-2016, https://doi.org/10.5194/ascmo-2-79-2016, 2016
Sergei N. Rodionov
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 63–78, https://doi.org/10.5194/ascmo-2-63-2016, https://doi.org/10.5194/ascmo-2-63-2016, 2016
David Bolin, Arnoldo Frigessi, Peter Guttorp, Ola Haug, Elisabeth Orskaug, Ida Scheel, and Jonas Wallin
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 39–47, https://doi.org/10.5194/ascmo-2-39-2016, https://doi.org/10.5194/ascmo-2-39-2016, 2016
Julie Bessac, Pierre Ailliot, Julien Cattiaux, and Valerie Monbet
Adv. Stat. Clim. Meteorol. Oceanogr., 2, 1–16, https://doi.org/10.5194/ascmo-2-1-2016, https://doi.org/10.5194/ascmo-2-1-2016, 2016
Short summary
Short summary
Several multi-site stochastic generators of zonal and meridional components of wind are proposed in this paper. Various questions are explored, such as the modeling of the regime in a multi-site context, the extraction of relevant clusterings from extra variables or from the local wind data, and the link between weather types extracted from wind data and large-scale weather regimes. We also discuss the relative advantages of hidden and observed regime-switching models.
E. M. Schliep, A. E. Gelfand, and D. M. Holland
Adv. Stat. Clim. Meteorol. Oceanogr., 1, 59–74, https://doi.org/10.5194/ascmo-1-59-2015, https://doi.org/10.5194/ascmo-1-59-2015, 2015
Short summary
Short summary
There is considerable demand for accurate air quality information in human health analyses. The sparsity of ground monitoring stations across the US motivates the need for advanced statistical models to predict air quality metrics. We propose a statistical model that jointly models ground-monitoring station data and satellite-obtained data allowing for temporal and spatial misalignment, missingness, and spatially and temporally varying correlation to enhance prediction of particulate matter.
R. Philbin and M. Jun
Adv. Stat. Clim. Meteorol. Oceanogr., 1, 29–44, https://doi.org/10.5194/ascmo-1-29-2015, https://doi.org/10.5194/ascmo-1-29-2015, 2015
T. K. Doan, J. Haslett, and A. C. Parnell
Adv. Stat. Clim. Meteorol. Oceanogr., 1, 15–27, https://doi.org/10.5194/ascmo-1-15-2015, https://doi.org/10.5194/ascmo-1-15-2015, 2015
W. B. Leeds, E. J. Moyer, and M. L. Stein
Adv. Stat. Clim. Meteorol. Oceanogr., 1, 1–14, https://doi.org/10.5194/ascmo-1-1-2015, https://doi.org/10.5194/ascmo-1-1-2015, 2015
Cited articles
Baddeley, A. J.: An error metric for
binary images, in: Robust Computer Vision Algorithms, edited by: Forstner, W. and Ruwiedel, S., Wichmann, 402 pp., 59–78, available at:
https://pdfs.semanticscholar.org/aa50/669b71429f2ca54d64f93839a9da95ceba6b.pdf (last access: 14 May 2020), 1992b. a, b, c, d, e, f
Baddeley, A. and Turner, R.: spatstat: An R Package for Analyzing Spatial
Point Patterns, J. Stat. Softw., 12, 1–42,
available at: http://www.jstatsoft.org/v12/i06/ (last access: 11 February 2021), 2005. a
Baldwin, M. E. and Elmore, K. L.: Objective verification of high-resolution WRF forecasts during 2005 NSSL/SPC Spring Program, in: 21st Conf. on
Weather Analysis and Forecasting/17th Conf. on Numerical Weather Prediction,
Amer. Meteor. Soc., Washington, D.C., 11B4, 2005. a
Becker, R. A., Wilks, A. R., Brownrigg, R., Minka, T. P., and Deckmyn, A.:
maps: Draw Geographical Maps, R package version
3.3.0, available at: https://CRAN.R-project.org/package=maps (last access: 11 February 2021), 2018. a
Brown, B. G., Gilleland, E., and Ebert, E. E.: Forecasts of Spatial Fields,
John Wiley and Sons, Ltd., 95–117, 2011. a
Brunet, D. and Sills, D.: A generalized distance transform: Theory and
applications to weather analysis and forecasting, IEEE T. Geosci. Remote, 55, 1752–1764,
https://doi.org/10.1109/TGRS.2016.2632042, 2017. a
Brunet, D., Sills, D., and Casati, B.: A spatio-temporal user-centric distance
for forecast verification, Meterol. Z., 27, 441–453,
https://doi.org/10.1127/metz/2018/0883, 2018. a
Casati, B., Ross, G., and Stephenson, D. B.: A new intensity-scale approach for
the verification of spatial precipitation forecasts, Meteorol. Appl., 11,
141–154, https://doi.org/10.1017/S1350482704001239, 2004. a, b
Cressie, N. and Suesse, T.: Great expectations and even greater exceedances
from spatially referenced data, Spat. Stat.-Neth., 37, 100420,
https://doi.org/10.1016/j.spasta.2020.100420, 2020. a
Dorninger, M., Mittermaier, M. P., Gilleland, E., Ebert, E. E., Brown, B. G.,
and Wilson, L. J.: MesoVICT: Mesoscale Verification Inter-Comparison over
Complex Terrain, Tech. rep., No. NCAR/TN-505+STR, https://doi.org/10.5065/D6416V21,
2013. a, b
Dorninger, M., Gilleland, E., Casati, B., Mittermaier, M. P., Ebert, E. E.,
Brown, B. G., and Wilson, L.: The set-up of the Mesoscale Verification
Inter-Comparison over Complex Terrain project, B. Am. Meteorol. Soc.,
99, 1887–1906, https://doi.org/10.1175/BAMS-D-17-0164.1, 2018. a, b
Ebert, E. E.: Fuzzy verification of high-resolution gridded forecasts: a review
and proposed framework, Meteorol. Appl., 15, 51–64, 2008. a
Ebert, E. E. and McBride, J. L.: Verification of precipitation in weather
systems: determination of systematic errors, J. Hydrol., 239, 179–202,
https://doi.org/10.1016/S0022-1694(00)00343-7, 2000. a
Gilleland, E.: Spatial Forecast Verification: Baddeley's Delta Metric Applied
to the ICP Test Cases, Weather Forecast., 26, 409–415, 2011. a
Gilleland, E.: SpatialVx: Spatial Forecast Verification, R package
version 0.6-6,
available at: http://www.ral.ucar.edu/projects/icp/SpatialVx (last access: 11 February 2021), 2019. a
Gilleland, E., Skok, G., Brown, B. G., Casati, B., Dorninger, M., Mittermaier, M. P., Roberts, N., and Wilson, L. J.: A novel set of verification test
fields with application to distance measures, Mon. Weather Rev., 148, 1653–1673,
https://doi.org/10.1175/MWR-D-19-0256.1, 2020. a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t
Kain, J. S., Weiss, S. J., Bright, D. R., Baldwin, M. E., Levit, J. J., Carbin, G. W., Schwartz, C. S., Weisman, M. L., Droegemeier, K. K., Weber, D. B., and
Thomas, K. W.: Some practical considerations regarding horizontal resolution
in the first generation of operational convection-allowing NWP, Weather
Forecast., 23, 931–952, https://doi.org/10.1175/WAF2007106.1, 2008. a
Keil, C. and Craig, G.: A displacement-based error measure applied in a
regional ensemble forecasting system, Mon. Weather Rev., 135, 3248–3259, 2007. a
Keil, C. and Craig, G.: A displacement and amplitude score employing an optical
flow technique, Weather Forecast., 24, 1297–1308, 2009. a
Koch, J., Jensen, K. H., and Stisen, S.: Toward a true spatial model evaluation
in distributed hydrological modeling: Kappa statistics, Fuzzy theory, and EOF-analysis benchmarked by the human perception and evaluated against a
modeling case study, Water Resour. Res., 51, 1225–1246,
https://doi.org/10.1002/2014WR016607, 2015. a
Koch, J., Siemann, A., Stisen, S., and Sheffield, J.: Spatial validation of
large-scale land surface models against monthly land surface temperature
patterns using innovative performance metrics, J. Geophys. Res.-Atmos., 121, 5430–5452, https://doi.org/10.1002/2015JD024482, 2016. a
Koch, J., Demirel, M. C., and Stisen, S.: The SPAtial EFficiency metric (SPAEF): multiple-component evaluation of spatial patterns for optimization of hydrological models, Geosci. Model Dev., 11, 1873–1886, https://doi.org/10.5194/gmd-11-1873-2018, 2018. a
Kullback, S. and Leibler, R. A.: On information and sufficiency, Ann.
Math. Stat., 22, 79–86, https://doi.org/10.1214/aoms/1177729694, 1951. a
Marzban, C. and Sandgathe, S.: Optical flow for verification, Weather
Forecast., 25, 1479–1494, https://doi.org/10.1175/2010WAF2222351.1, 2010. a
Mass, C. F., Ovens, D., Westrick, K., and Colle, B. A.: Does Increasing
Horizontal Resolution Produce More Skillful Forecasts?, B. Am. Meteorol. Soc., 83, 407–430,
https://doi.org/10.1175/1520-0477(2002)083<0407:DIHRPM>2.3.CO;2, 2002. a
Meijster, A., Roerdink, J. B. T. M., and Hesselink, W. H.: A General Algorithm for Computing Distance Transforms in Linear Time, in: Mathematical Morphology and its Applications to Image and Signal Processing, edited by: Goutsias, J., Vincent, L., Bloomberg, D. S., Computational Imaging and Vision, Springer, Boston, MA, 18, https://doi.org/10.1007/0-306-47025-X_36, 2000. a
Mittermaier, M. P.: A strategy for verifying near-convection resolving model forecasts at observing sites, Weather Forecast., 29, 185–204, 2014. a
Nychka, D., Furrer, R., Paige, J., and Sain, S.: fields: Tools for spatial
data, R package version 10.0, https://doi.org/10.5065/D6W957CT, available at: https://github.com/NCAR/Fields (last access: 11 February 2021), 2017. a
R Core Team: R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria,
available at: https://www.R-project.org/ (last access: 11 February 2021), 2018. a
Rezacova, D., Sokol, Z., and Pesice, P.: A radar-based verification of
precipitation forecast for local convective storms, Atmos. Res., 83,
221–224, https://doi.org/10.1016/j.atmosres.2005.08.011, 2007. a
Roberts, N. M. and Lean, H. W.: Scale-Selective Verification of Rainfall
Accumulations from High-Resolution Forecasts of Convective Events, Mon.
Weather Rev., 136, 78–97, 2008. a
Rossa, A. M., Nurmi, P., and Ebert, E. E.: Overview of methods for the
verification of quantitative precipitation forecasts, edited by: Michaelides, S. C., Springer, New York, NY, USA, 418–450, 2008. a
Skamarock, W. C., Klemp, J. B., Dudhia, J., Gill, D. O., Barker, M., Wang, W.,
and Powers, J. G.: A description of the Advanced Research WRF version 2,
Ncar tech. rep. ncar/tn-468+str, Boulder, CO, USA, 2005. a
Steinacker, R., Häberli, C., and Pöttschacher, W.: A transparent method
for the analysis and quality evaluation of irregularly distributed and noisy
observational data, Mon. Weather Rev., 128, 2303–2316, 2000. a
Venugopal, V., Basu, S., and Foufoula-Georgiou, E.: A new metric for comparing
precipitation patterns with an application to ensemble forecasts, J. Geophys.
Res., 110, D08111, https://doi.org/10.1029/2004JD005395, 2005. a
Wealands, S. R., Grayson, R. B., and Walker, J. P.: Quantitative comparison of
spatial fields for hydrological model assessment – some promising
approaches, Adv. Water Resour., 28, 15–32, 2005. a
Weniger, M., Kapp, F., and Friederichs, P.: Spatial Verification Using Wavelet
Transforms: A Revie, Q. J. Royal Meteor. Soc., 143,
120–136, https://doi.org/10.1002/qj.2881, 2016. a
Wikle, C. K., Zammit-Mangion, A., and Cressie, N.: Spatio-Temporal Statistics
with R, Chapman and Hall/CRC, Boca Raton, Florida, USA, 396 pp.,
available at: https://spacetimewithr.org (last access: 11 February 2021), 2019.
a
Zhang, J., Craigmile, P. F., and Cressie, N.: Loss function approaches to
predict a spatial quantile and its exceedance region, Technometrics, 50, 216–227, https://doi.org/10.1198/004017008000000226, 2008. a
Zinner, T., Mannstein, H., and Taferner, A.: Cb-TRAM: Tracking and monitoring
severe convection from onset over rapid development to mature phase using
multi-channel Meteosat-8 SEVIRI data, Meteorol. Atmos. Phys., 101, 191–210,
https://doi.org/10.1007/s00703-008-0290-y, 2008. a
Short summary
Verifying high-resolution weather forecasts has become increasingly complicated,
and simple, easy-to-understand summary measures are a good alternative. Recent work has demonstrated some common pitfalls with many such summaries. Here, new summary measures are introduced that do not suffer from these drawbacks, while still providing meaningful information.
Verifying high-resolution weather forecasts has become increasingly complicated,
and simple,...
