The nonparametric estimation of the power spectral density of uniformly-spaced data with missing samples is revisited. Classical estimators, such as the standard periodogram and the Lomb-Scargle periodogram, are biased when samples are missing. The classical method to obtain an asymptotically-unbiased estimator is to take the finite Fourier transform of the standard unbiased estimator of the autocorrelation function. However, the latter estimator is not necessarily positive semidefinite, so its finite Fourier transform can yield negative power spectral density values at some frequencies. To avoid this problem, Gao et al.2021)Gao et al.2021)Gao et al.2021)
An understanding of Earth's past climate can help put current and future changes into historical context. Widely used tree ring-based drought atlases generally target the Palmer Drought Severity Index or other metrics of soil moisture and/or drought risk. These indices reflect contemporaneous meteorological conditions, and it is possible to extract information about temperature and precipitation given the existing reconstructions. Here, we present a fully Bayesian inverse method that infers a joint posterior for monthly mean temperature and precipitation given tree ring-based PDSI reconstructions from the North American Drought Atlas. The method is skillful at reconstructing early twentieth century conditions when compared to instrumental measurements from the CRU TS dataset. Moreover, the reconstructions can capture the complex temporal and multivariate covariance structure between monthly regional temperatures and precipitation. By reconstructing regional temperature and precipitation for the last millennium, we identify the driest and wettest years and decades in each region. Our results highlight the unique nature of the 1930s Dust Bowl drought in central Kansas and the late twentieth century pluvial in the North American southwest.
Fine-scale oceanic fronts are ubiquitous and ephemeral physical features that separate contrasting water masses, creating significant heterogeneity in the physical seascape and plankton distributions. Because phytoplankton community composition (PCC) is a key driver of marine ecosystem functioning, understanding the extent to which fine-scale fronts influence PCC is a critical challenge. However, studying PCC across and within fronts is particularly difficult due to data scarcity and high biophysical variability. We developed a tailored statistical model to characterize PCC within an oceanic front we studied in the Mediterranean Sea. We modeled the frontal community as a finite mixture model with three components: two communities of adjacent water masses and a potential front-adapted community. Each component was further considered as a discrete mixture of an unknown number of multivariate Gaussian sub-components. First, we used an Expectation–Maximization algorithm to estimate the Gaussian parameters and determine the optimal number of sub-components based on in situ datasets of the PCC within a frontal zone and its adjacent water masses. Second, a hierarchical Bayesian approach was applied to estimate the weight of all components within the frontal dataset. Our analysis suggests that within the front a new community component, distinct from those in adjacent water masses, accounts for 70 % of the frontal community, indicating that a specific phytoplankton community can emerge in fine-scale oceanic fronts. Despite the limited number of frontal observations, our Bayesian modelling approach provides statistical evidence of the front's influence on phytoplankton community composition, effectively overcoming data scarcity and high variability.
Accurate modelling of extreme precipitation is vital for predicting future risks and informing adaptation strategies. Here, we compare and evaluate six different extreme value statistical models for hourly to 48 h extreme precipitation in southern Germany, with a primary focus on duration-dependent Generalized Extreme Value (dGEV) distributions. To assess model performance, particularly in capturing tail behavior, we utilize the 50-member single model initial-condition large ensemble of the Canadian Regional Climate Model version 5 for the period 1980–2019. The large sample size of 2000 simulated years enables a robust sampling of extreme quantiles. Using a sub-sampling strategy with 30 to 100 years, we compare the efficacy of Bayesian methodology, in particular Bayesian hierarchical models, against frequentist models (L-moments and Maximum Likelihood Estimation – MLE) in representing the tail risk of 100-year return levels based on limited sample sizes. Hierarchical models allow us to give special emphasis on the dimensionality of the GEV shape parameter, a critical factor for tail behavior. Our findings reveal that a shape parameter varying over durations but fixed across space is beneficial for the prediction of the 100-year return level. The resulting Intensity-Duration-Frequency (IDF) curve shows the highest accuracy and smallest confidence intervals proving its robustness. Compared to the standard GEV estimated by L-moments, our proposed model can reduce the relative error of the 100-year return level from 18.1  % to 8.8  % based on a 30-year sample size. Furthermore, our analysis reveals fundamental limitations of the Anderson-Darling test for extreme value model selection, demonstrating its poor correlation with predictive skill for upper quantiles – a critical finding for climate risk applications.
Soil moisture–temperature coupling (SM–T) significantly influences the frequency and amplitude of heat extremes. It describes how variations in soil moisture affect surface air temperature conditions and vice versa. This study aims to determine the spatial extent and duration of SM–Tin situT
We present a novel spatiotemporal clustering algorithm to extract spatiotemporal events based on their intensity. Our algorithm proceeds in two steps: (1) extracting intensity structures that are spatiotemporally consistent and, (2) separating individual events. We apply the algorithm to a novel drought index over the continental United States from 1980–2021 and show that it captures historical drought events over the continental United States and their spatiotemporal extents. 
We present a novel flexible bi-level spatiotemporal clustering algorithm to extract events based on their intensity and spatiotemporal structures. Our algorithm consists of using (i) a novel space-time k-means clustering to obtain spatiotemporally coherent intensity clusters, and (ii) a density-based spatial clustering of applications with noise (DBSCAN) to spatiotemporally section the intensity clusters into individual events. We discuss the development of the algorithm, the selection, tuning and meaning of the parameters within each step, as well as its validation. Finally, we apply the algorithm to a spatiotemporal drought index, standardized vapor pressure deficit drought index (SVDI), over the continental United States (US) from 1980–2021 and show that it captures historical drought events over the continental United States and their spatiotemporal extents.
The aim of this study is to provide a probabilistic gust analysis for the region of Germany that is calibrated with station observations and with an interpolation to unobserved locations. To this end, we develop a spatial Bayesian hierarchical model (BHM) for the post-processing of surface maximum wind gusts from the COSMO-REA6 reanalysis. Our approach uses a non-stationary extreme value distribution for the gust observations at the top level, with parameters that vary according to a linear model using COSMO-REA6 predictor variables. To capture spatial patterns in surface extreme wind gust behavior, the regression coefficients are modeled as 2-dimensional Gaussian random fields with a constant mean and an isotropic covariance function that depends only on the distance between locations. In addition, we include an elevation offset in the distance metric for the covariance function to account for differences in topography. This allows us to include data from mountaintop stations in the training process and to utilize all available information. The training of the BHM is carried out with an independent data set from which the data at the station to be predicted are excluded. We evaluate the spatial prediction performance at the withheld station using Brier score and quantile score, including their decomposition, and compare the performance of our BHM to climatological forecasts and a non-hierarchical, spatially constant baseline model. This is done for 109 weather stations in Germany. Compared to the spatially constant baseline model, the spatial BHM significantly improves the estimation of local gust parameters. It shows up to 5 %
Stochastic weather generators are efficient statistical models producing synthetic weather series by replicating key statistical properties without the computational cost of physical models. However, for applications requiring temperature simulation over a large area, challenges arise due to non-stationarity over time and spatial-temporal dependencies. This paper introduces a daily stochastic weather generator for temperature with arbitrary spatial resolution. The non-stationarity issue is addressed using a decomposition method to separate deterministic terms (trends and seasonality), from the stochastic part representing the underlying climate variability. We extend the existing local decomposition method to extrapolate to any point in space. The spatial-temporal dependence is modeled through a Gaussian field with a non-separable covariance function, accommodating complex interactions between time and space. Our generator, calibrated on a few French weather stations, is validated using several spatio-temporal indicators. First, we evaluate the generator's performance at the fitting stations, comparing simulated and observed indicators. Subsequently, we compare our spatial simulations to a high resolution gridded observation dataset. Results demonstrate that the proposed generator accurately captures the observed spatio-temporal statistics, even for extreme events such as large scale persistent heat waves.
We present a lightweight stochastic weather generator (SWG) based on a multisite hidden Markov model (HMM) trained on a large area with French weather station data. Our model captures spatiotemporal precipitation patterns with a strong emphasis on seasonality and the accurate reproduction of dry and wet spell distributions. The hidden states serve as interpretable large-scale weather regimes, learned directly from the data without requiring exogenous inputs. Compared to existing approaches, it offers a robust balance between interpretability and performance, particularly for extremes. The model architecture enables seamless integration of additional weather variables. Finally, we demonstrate its application to future climate scenarios, highlighting how parameter evolution and extreme event distributions can be analyzed in a changing climate.
Studies that investigate the effects of meteorological fluctuations on varying multi-disciplinary outcomes often depend on analysis of high-frequency sensor data from automatic monitoring stations in different locations. The validation of such spatial time series requires attention given that they are susceptible to multiple forms of error. Existing validation techniques tend to cater to detection of only one form of outlier in isolation, lack robustness, or fail to optimally leverage the strong between-series correlation that often prevails in high-frequency meteorological data exhibiting multiple seasonalities. To address these shortcomings, two adaptations were made to an existing procedure, for more powerful outlier detection in strongly correlated high-frequency time series, using a distributional approach. The modified technique was tested in a simulation study and was also applied to a real univariate spatial set of hourly air temperature series from the South African Air Quality Information System. In both instances, the effectiveness of the technique in detecting outliers was assessed relative to procedures lacking either or both adaptations. The results show the modified procedure to be most comprehensive in the simultaneous detection of multiple forms of error, including solitary spikes, shifts in the series mean, and irregularities in the diurnal pattern. Furthermore, the method is generalizable to any
The Australian Bureau of Meteorology uses linear methods and provides seasonal forecasts expressed as the probability of exceeding median rainfall. I use expanded methods, including more ocean data and deep learning neural networks that provide nonlinear estimates of the rainfall as measured by rain gauges.
Forecasting rainfall into the next season remains highly challenging and is normally presented in terms of probabilities rather than the expected rainfall as measured by rain gauges. I show here that, in favourable cases, for the selected times of the year and selected geographical regions, it is possible to obtain useful quantitative forecasts of rainfall with a series of relatively simple steps. One such instance explored in this work is the prediction of austral springtime rainfall in SE Australia regions predominantly based on the surrounding ocean surface temperatures during the winter.In the first stage, I search for predictors by exploring correlations between the target rainfall and ocean surface temperatures at earlier times. In addition to standard ocean climate indicators such as El Niño or the Indian Ocean Dipole, other typical patterns of variation are captured in terms of the temperatures of selected ocean areas. When characteristic patterns of correlation are discovered, they are included in the predictor selection in the form of expansion in terms of the empirical orthogonal functions (EOFs). EOF expansions can provide very strong signals. For example, in the case of the Indian Ocean, during the winter, the dominant EOF shows a stronger correlation with future rainfall than the commonly used Indian Ocean Dipole.The technical part of the forecast model is provided by deep learning artificial neural networks, where I use the information sources with the strongest correlation in relation to the historical rainfall data as the inputs. The networks are trained on past rainfall data, and the output is a quantitative forecast based on the current state of the predictors. The resulting hindcasts appear to be accurate for September and October and less reliable for November. I also present model forecasts for rainfall during the 2024 austral spring in the selected SE Australia regions.
In the study of the global climate, ocean temperature estimates use sea surface temperature (SST) anomalies instead of marine air temperature (MAT) anomalies. A key question to ask is whether biases result from this choice. In this article we employ hierarchical statistical models to investigate spatiotemporal differences between SST and MAT and their anomalies in the tropical Pacific. The analysis uses observations from the Tropical Atmosphere Ocean (TAO) buoy network and the ERA5 data product. Our spatiotemporal modeling approach accounts for missing data in the observation network and allows for full uncertainty quantification. Our findings indicate evidence that SST and MAT are interchangeable in the tropical Pacific when we calculate seasonally adjusted monthly anomalies.
By the end of 2023, renewable sources covered 63.4 % of the total electric-power demand of Chile, and, in line with the global trend, photovoltaic (PV) power showed the most dynamic increase. Although Chile's Atacama Desert is considered to be the sunniest place on Earth, PV power production, even in this area, can be highly volatile. Successful integration of PV energy into the country's power grid requires accurate short-term PV power forecasts, which can be obtained from predictions of solar irradiance and related weather quantities. Nowadays, in weather forecasting, the state-of-the-art approach is the use of ensemble forecasts based on multiple runs of numerical weather prediction models. However, ensemble forecasts still tend to be uncalibrated or biased, thus requiring some form of post-processing. The present work investigates probabilistic forecasts of solar irradiance for regions III and IV in Chile. For this reason, eight-member short-term ensemble forecasts of solar irradiance for the calendar year 2021 are generated using the Weather Research and Forecasting (WRF) model; these are then calibrated using the benchmark ensemble model output statistics (EMOS) method based on a censored Gaussian law and its machine-learning-based distributional regression network (DRN) counterpart. Furthermore, we also propose a neural-network-based post-processing method, resulting in improved eight-member ensemble predictions. All forecasts are evaluated against station observations for 30 locations in the study area, and the skill of post-processed predictions is compared to the raw WRF ensemble. Our case study confirms that all studied post-processing methods substantially improve both the calibration of probabilistic forecasts and the accuracy of point forecasts. Among the methods tested, the corrected ensemble exhibits the best overall performance. Additionally, the DRN model generally outperforms the corresponding EMOS approach.
This paper presents a pioneering study on the inference of boxplot-valued data using both Bayesian and frequentist approaches within a multivariate framework. This approach leverages complex yet intuitive representations to make large datasets more manageable and enhance their interpretability, which is invaluable in the age of big data. Boxplot-valued data are particularly important due to their ability to capture the inherent variability and distributional characteristics of complex datasets.In our study, we propose novel methodologies for parameter estimation and density estimation for boxplot-valued data and apply these techniques to climatological data. Specifically, we utilize data from the Berkeley Earth Surface Temperature Study, which aggregates 1.6 billion temperature reports from 16 pre-existing archives affiliated with the Lawrence Berkeley National Laboratory. Our methods are validated through extensive simulations comparing the efficiency and accuracy of Bayesian and frequentist estimators.We demonstrate the practical applicability of our approach by analyzing summer average temperatures across various European countries. The proposed techniques provide robust tools for analyzing complex data structures, offering valuable insights into climatic trends and variations. Our study highlights the advantages and limitations of each inferential method, offering guidance for future research and applications in the field of climatology.
Drought is one of the recurring natural phenomena affecting the socio-economic and environmental well-being of southern Ethiopia's society. The availability of insufficient ground-based rainfall observatory networks is limiting drought-monitoring and early-warning investigations. The main objective of this study is to analyze spatial and temporal drought characteristics using high-resolution satellite-based rainfall products for the 1991–2022 period in the Southern Nations, Nationalities, and Peoples (SNNP) region of Ethiopia. The satellite-based rainfall product used in this study was selected after the evaluation of three satellite products, namely the Africa Rainfall Climatology version 2 (ARC2), the Climate Hazards Group InfraRed Precipitation with Stations (CHIRPS), and the Tropical Applications of Meteorology using SATellite and ground-based observations (TAMSAT), against station-based rainfall for the study area space and time domains. The statistical metrics of correlation coefficient (CORR), bias (BIAS), percent bias (PBIAS), mean error (ME), mean absolute error (MAE), root mean square error (RMSE), and Nash–Sutcliffe efficiency coefficient (NSE) were used to compare and evaluate the satellite rainfall products. Accordingly, the CHIRPS shows the highest CORR of 0.96 and the highest BIAS of 1.02, which is very near to the perfect value (BIAS = 1), followed by the TAMSAT. Hence, the CHIRPS-based satellite rainfall product was used to assess the spatio-temporal patterns of meteorological drought based on the 3-month and 12-month standardized precipitation index (SPI). The results successfully grasped the known historical and recent droughts of 2022, 2021, 2015, 2014, 2010, 2009, and 2000. A high intensity and a high severity of drought were noted in the SPI-3, while the least occurrences of extreme events were recorded in the SPI-12. Additionally, severe drought situations were detected in the drought-prone areas in the southern and southeastern parts of the SNNP region. Finally, the study concludes that, to construct grid-based drought-monitoring tools for the development of early-warning systems, the CHIRPS rainfall product can be used as an additional source of information.
Proper scoring rules are an essential tool to assess the predictive performance of probabilistic forecasts. However, propriety alone does not ensure an informative characterization of predictive performance, and it is recommended to compare forecasts using multiple scoring rules. With that in mind, interpretable scoring rules providing complementary information are necessary. We formalize a framework based on aggregation and transformation to build interpretable multivariate proper scoring rules. Aggregation-and-transformation-based scoring rules can target application-specific features of probabilistic forecasts, which improves the characterization of the predictive performance. This framework is illustrated through examples taken from the weather forecasting literature, and numerical experiments are used to showcase its benefits in a controlled setting. Additionally, the framework is tested on real-world data of postprocessed wind speed forecasts over central Europe. In particular, we show that it can help bridge the gap between proper scoring rules and spatial verification tools.
A number of recent climate studies have used univariate parametric statistical models to estimate return periods of extreme weather events based on the method of maximum likelihood. Using simulations over multiple training datasets, we find that using maximum likelihood gives predictions of extreme return levels that are exceeded more often than expected. For instance, when using the generalised extreme value distribution (GEVD) with 50 annual data values, fitted using maximum likelihood, we find that 200-year return levels are exceeded more than twice as often as expected; i.e. they are exceeded in more than 1 in 100 simulated years. This bias, which we refer to as a predictive coverage probability (PCP) bias, would be expected to lead to unreliable predictions. We review the theory related to Bayesian prediction using right Haar priors which gives an objective way to incorporate parameter uncertainty into predictions for some statistical models and which eliminates the bias. We consider a number of commonly used parametric statistical models and give the right Haar priors in each case. Where possible, we give analytical solutions for the resulting predictions. Where analytical solutions are not possible, we apply either an asymptotic approximation for the Bayesian prediction integral or ratio of uniforms sampling. For the fully parameterised GEVD and the generalised Pareto distribution with a known location parameter, neither of which have a right Haar prior, we test a number of methods and find one that gives big reductions in the PCP bias relative to maximum likelihood predictions. Finally, we revisit the De Bilt extreme temperature example considered in a number of previous studies and generate revised, and shorter, estimates for the return period of the 2018 heatwave. Software for fitting predictive distributions with parameter uncertainty has been developed by the first author and will be available as an R package.
We present a novel non-stationary regional weather generator (nsRWG) based on an auto-regressive process and marginal distributions conditioned on climate variables. We use large-scale circulation patterns as a latent variable and regional daily mean temperature as a covariate for marginal precipitation distributions to account for dynamic and thermodynamic changes in the atmosphere, respectively. Circulation patterns are classified using ERA5 reanalysis mean sea level pressure fields. We set up the nsRWG for the central European region using data from the E-OBS dataset, covering major river basins in Germany and riparian countries. The nsRWG is meticulously evaluated, showing good results in reproducing at-site and spatial characteristics of precipitation and temperature. Using time series of circulation patterns and the regional daily mean temperature derived from general circulation models (GCMs), we inform the nsRWG about the projected future climate. In this approach, we utilize GCM output variables, such as pressure and temperature, which are typically more accurately simulated by GCMs than precipitation. In an exemplary application, the nsRWG statistically downscales precipitation from nine selected models from the Coupled Model Intercomparison Project Phase 6 (CMIP6), generating long synthetic but spatially and temporally consistent weather series. The results suggest an increase in extreme precipitation over the German basins, aligning with previous regional analyses. The nsRWG offers a key benefit for hydrological impact studies by providing long-term (thousands of years) consistent synthetic weather data indispensable for the robust estimation of probability changes in hydrologic extremes such as floods.
This paper presents a statistical analysis of air temperature data from 32 stations in Italy and the UK up to 2000 m above sea level from 2002 to 2021. The data came from both highland and lowland areas in order to evaluate the differences due to both location and elevation. The analysis focused on detecting trends at annual and monthly timescales, employing ordinary least-squares (OLS), robust S-estimator regression, and Mann–Kendall (MK) and Sen's slope methods. Hierarchical clustering (HCA) using dynamic time warping (DTW) was then applied to the monthly data to analyze the intra-annual pattern similarity of trends within and across the groups.Two different regions of Europe were chosen because of the different climate and temperature trends – namely, the northern UK (smaller trends) and the northwest Italian Alps (larger trends). The main novelty of the work is to show that stations with similar locations and altitudes have similar monthly slopes by quantifying them using DTW and clustering. These results reveal the nonrandomness of different trends throughout the year and between different parts of Europe, with a modest influence of altitude in wintertime. The findings revealed that group average trends were close to the National Oceanic and Atmospheric Administration (NOAA) values for the areas in Italy and the UK, confirming the validity of analyzing a small number of stations. More interestingly, intra-annual patterns were detected commonly at the stations of each of the groups and are clearly different between them. Confirming the different climates, most highland and lowland stations in Italy exhibit statistically significant positive trends, while in the UK, both highland and lowland stations show statistically nonsignificant negative trends. Hierarchical clustering in combination with DTW showed consistent similarity between monthly patterns of means and trends within the group of stations and inconsistent similarity between patterns across groups. The use of the 12 distance correlation matrices (dcor) (one for each month) also contributes to what is the main result of the paper, which is to clearly show the different temporal patterns in relation to location and (in some months) altitude. The anomalous behaviors detected at 3 of the 32 stations, namely Valpelline, Fossano, and Aonoch Mòr, can be attributed, respectively, to the facts that Valpelline is the lowest-elevation station in its group; Fossano is the southernmost of the Italian stations, with some sublittoral influence; and Aonoch Mòr has a large number of missing values.In conclusion, these results improve our understanding of temperature spatio-temporal dynamics in two very different regions of Europe and emphasize the importance of consistent analysis of data to assess the ongoing effects of climate change. The intra-annual time patterns of temperature trends could also be compared with climate model results.
Event attribution methods are increasingly routinely used to assess the role of climate change in individual weather events. In order to draw robust conclusions about whether changes observed in the real world can be attributed to anthropogenic climate change, it is necessary to analyse trends in observations alongside those in climate models, where the factors driving changes in weather patterns are known. Here we present a quantitative statistical synthesis method, developed over 8 years of conducting rapid probabilistic event attribution studies, to combine quantitative attribution results from multi-model ensembles and other, qualitative, lines of evidence in a single framework to draw quantitative conclusions about the overarching role of human-induced climate change in individual weather events.