Temperature profiles are from radiosondes where a sensor package carried aloft by a balloon transmits data back to a ground station. Vertical resolution of the data as available in global databases is variable on the order of 100 m. We chose four of the very few available radiosonde stations that measure four times a day instead of only twice in order to capture the diurnal cycle. These four stations are spread throughout Germany in flat to hilly terrain with launch times at 00:00, 06:00, 12:00, and 18:00 UTC: Bergen (52.82∘ N, 9.92∘ E), Lindenberg (52.21∘ N, 14.12∘ E), Idar-Oberstein (49.69∘ N, 7.33∘ E) and Kuemmersbruck (49.43∘ N, 11.90∘ E). All the stations are located at the mid-latitudes and none of the sites is characterized by complex terrain. The average distance between the stations is approx. 300 km. Quality-controlled temperature and pressure data (see , for details) were accessed from the freely available Integrated Global Radiosonde Archive of the National Oceanic and Atmospheric Administration .

We use forecasts from the EPS of the European Centre for Medium Range Weather Forecasts (ECMWF) with 50 perturbed ensemble members and one unperturbed control run . The model is discretized in a 0.25∘×0.25∘ horizontal grid on 91 vertical levels. Since data from individual ensemble members on model levels are not archived at ECMWF, we saved real-time forecast data from 15 September 2016 to 15 September 2019 for 25 lead times at 6-hourly time intervals out to +144 h (6 d). The forecasts are interpolated bi-linearly to the radiosonde locations in the horizontal. Due to the use of a hybrid coordinate system in the ECMWF forecasts, the pressures of the 51 ensemble members at each model level differ somewhat. Therefore, observations and NWP forecasts are linearly interpolated to the arithmetic mean of the 51 pressures at each model level.

In the remainder, the radiosonde measurements Tobs are used as response variables. For univariate statistical postprocessing only the sample mean T‾ens and sample standard deviation SDens of the ensemble temperature forecasts are used as the ensemble predictors, with the assumption that the ensemble members can be adequately described by a Gaussian distribution. For multivariate postprocessing model-level temperatures of all ensemble members are used.

Nonhomogeneous Gaussian regression is a postprocessing technique for ensemble forecasting systems. The observed temperature y is assumed to follow a Gaussian distribution, 1y∼N(μ,σ), determined by two parameters, μ and σ. The expectation of a Gaussian distributed random variable is E(y)=μ. Thus, μ is often referred to as expectation or mean. σ is often denoted as the standard deviation. However, in order to make these parameters easier to distinguish from the sample mean and sample standard deviation, we call μ and σ the location and scale parameter, which is also common in the statistical literature. To condition the Gaussian distribution on covariates derived from the NWP model, the location μ is expressed by a linear model of the ensemble (sample) mean T‾ens and the scale parameter σ by the sample standard deviation of the NWP ensemble members SDens: 2μ=β0+β1⋅T‾ens and log⁡(σ)=γ0+γ1⋅SDens. Originally, estimated the coefficients β⋆ and γ⋆ with a sliding training window in order to allow for their seasonal variation. An NGR, which can exploit the full data set and have coefficients that may vary not only seasonally but also by arbitrary other factors (NGRVC), can be constructed with GAMLSS models. Then a sum of several nonlinear functions replaces the intercepts and linear coefficients in this very recent extension of NGR cf.. Here, the location parameter μ is chosen as 3μ=f1|hh(p)+f2(doy)+f3(p,doy)︸varying interceptβ0+f4|hh(p)+f5(doy)+f6(p,doy)︸varying linear coefficientβ1⋅T‾ens. Both the intercept and the linear coefficient are allowed to vary smoothly with functions of pressure p, season (day of year doy) and their interaction. The |hh in the index of the function for the pressure dependence indicates that this function is conditional on the time of the radiosonde ascent (00:00, 06:00, 12:00, and 18:00 UTC in the current case) in order to account for diurnal variations. The terms f4|hh, f5 and f6 provide a varying linear coefficient for the ensemble mean T‾ens and thus allow the coefficient β1 in Eq. () to vary smoothly. The functional terms f1–f3 act on the scale of the additive predictor (here temperature in degrees) and serve as varying intercept.

A similar model is chosen for the logarithm of the scale parameter, 4log⁡(σ)=f1|hh(p)+f2(doy)+f3(p,doy)︸varying interceptγ0+f4|hh(p)+f5(doy)+f6(p,doy)︸varying linear coefficientγ1⋅SDens, where the functions f1–f3 act on the logarithmic scale and allow the coefficient γ0 in Eq. (3) to vary smoothly. Functions f4–f6 provide a varying linear coefficient of the logarithm of the standard deviation and allow γ1 of Eq. () to vary smoothly. Note that the smooth functions f⋆ in Eqs. () and (), which are identically named for simplicity, might vary in their form; e.g., the day of the year term in Eq. () might have a different functional form than the day of the year term in Eq. (). For more details on the relation between the classical sliding window approach and this smooth model approach, the reader is directed to .

This section provides an alternative method to remove the constraint that NGR models have to be fitted to each location separately. In the current case location refers to a particular level of the vertical profiles and not to multiple sites as in previous applications . Thus, the SAMOS models in the current study are fitted to the data of each site individually.

The motivation behind SAMOS is that with many locations and/or for operational use fitting many models might become computationally expensive. However, one single NGR-like regression model can be fitted to all locations at once when in a first step all site-specific characteristics are (nearly) eliminated from observations and ensemble data. The model is fitted not to the direct values, but rather to standardized anomalies formed by subtracting the respective climatological mean and dividing by the climatological standard deviation. This SAMOS approach is described in .

Here, we estimate the climatological properties of the observations with a GAM(LSS); for the expected value as 5μ=f1|hh(p)+f2(doy)+f3(p,doy), with similar terms to Eq. (). Only an intercept is estimated for the logarithm of the scale parameter σ, which technically reduces the GAMLSS to a GAM. One set of (μ, σ) has to be fitted for the radiosonde measurements, another one for the EPS data.

The pressure levels of the observational radiosondes and the EPS data are not the same. We linearly interpolated the spatially better resolved observations to the pressure levels of the EPS data (see Sect. ). Then the standardized anomalies of the observations and the ensemble are linked by a simple linear regression of the original NGR Eq. ().

Both NGRVC (Sect. ) and SAMOS (Sect. ) are univariate postprocessing methods; i.e., only marginal distributions are predicted. To account for dependence structure of the vertical levels, ensemble copula coupling ECC, is employed, which uses the dependence structure contained in the vertical profiles of the raw ensemble via its order statistics.

The ECC procedure consists of three steps.

Sample quantiles from the predicted marginal distributions.

In step 1 temperature quantiles have to be sampled from the univariate distributions obtained by either NGRVC (Sect. ) or SAMOS (Sect. ) for each level. The number of sampled quantiles matches the number of ensemble members m. Three different approaches are commonly employed for sampling these quantiles.

ECC-Q transforms m equally spaced probabilities with the quantile function (i.e., the inverse of the cumulative distribution function) of the predictive distribution.

Several metrics are used to evaluate the performance of the forecasting methods in achieving the goal of probabilistic forecasts: to maximize the sharpness of the predictive distribution subject to calibration. The metrics are applied out-of-sample using 5-fold cross-validation to test on independent data and avoid overfitting.

Calibration and sharpness of univariate predictions are evaluated using the probability integral transform (PIT) histogram and the sharpness diagram, respectively . Calibration refers to the statistical consistency between predicted distributions and observations. A calibrated forecast leads to a uniform PIT histogram. When the prediction is not available as a full probability distribution, but as samples of a distribution as in the case of the raw ensemble, one can assess calibration with the rank histogram. The rank histogram displays the rank of the observation in a vector merging observations and samples of the predictive distribution. If the observation is indistinguishable from the samples, the rank histogram is uniform; i.e., the forecast is calibrated.

Sharpness is an attribute of the forecast alone and refers to the concentration of the predictive distributions. This attribute is revealed in the sharpness diagram which displays the width of the central prediction interval of the predictive distributions. The aim of forecasting schemes is to obtain calibrated distributions that are as sharp as possible .

The rank histogram and the PIT histogram are common tools to verify the calibration of the forecasts in the univariate case. introduce four different types of their extensions to assess multivariate calibration. further investigates these methods in a simulation study and concludes that one should always use a combination of different multivariate rank histograms. In this study we employ the average rank histogram (ARH) and the band-depth histogram (BDH). The ARH provides a measure of ascending rank. In contrast, the BDH assesses the centrality of the observation within the forecast ensemble. One can thus not spot from the BDH whether a potential bias is positive or negative, which one can identify from the ARH.

As numerical measures we apply the continuous ranked probability score (CRPS) and the log score (LS) for univariate predictions . The LS is the negative log-likelihood. The CRPS can be interpreted as integral over all possible Brier scores and is a generalization of the mean absolute error for point forecasts .

The energy score ES, and the variogram score VS, are applied as numerical measures for assessing the performance of the multivariate predictions. The ES is the multivariate generalization of the CRPS. Like the CRPS, the ES gives absolute values only and is negatively orientated, meaning lower values are better then higher values. It addresses calibration of multivariate forecasts in terms of location and scale, i.e., mean and spread. However, some studies suggest that the ES is not sufficiently sensitive to misspecifications of the correlation structure e.g.,. Thus, introduced the VS as an alternative multivariate score. We employ the VS with an order of 0.5, which was found to discriminate well between correct and incorrect correlations and between calibrated and overdispersive/underdispersive forecasts cf. Eq. 3 in. The VS is an absolute measure too, and its skill is negatively orientated.

All scores are computed out-of-sample by 5-fold cross-validation using the package for the R environment for statistical computing .

The validation of the numerical scores was done by bootstrapping to estimate the uncertainty in the scores. This was applied by drawing a random sample in the size of the original data with replacement and averaging it. This procedure was repeated 500 times, and the 500 values are presented as box-and-whisker plots. In order to preserve the vertical dependence structure while bootstrapping, we considered a vertical profile to be the smallest indivisible unit: rather than bootstrapping temperatures on vertical levels and merging these samples to new profiles, we bootstrapped vertical temperature profiles as a whole for different times, locations and lead times.

Ensemble copula coupling (Sect. ) is used to restore the vertical correlation structure of the temperature profiles contained in the raw ensemble to the univariately corrected temperatures at each level. To better understand the three methods for sampling from the raw ensemble (ECC-Q, ECC-R, ECC-T), they are first demonstrated with two case studies before their forecast performance is evaluated for the whole data set.

The performance of the different correction methods is evaluated both graphically with rank histograms and numerically with the energy score and variogram score (cf. Sect. ) for the whole period 15 September 2016–15 September 2019.

Figure  assesses the calibration of forecasts over all locations, lead times and pressure levels graphically through average (ARH) and band-depth (BDH) multivariate rank histograms. The strong excess at low ranks in both histograms suggests that the raw ensemble profiles are systematically too warm. The pronounced ∩ shape of the average rank histogram for the quantile samples of univariate NGRVC correction (second column in Fig. ) indicates overdispersion. This is due to the construction of the NGRVC-Q ensemble, which assumes a correlation of unity between the temperatures at different levels . All three versions of the multivariate ensemble copula coupling reduce this overdispersion, but fail to eliminate it.

Figure  shows the univariate performance as evaluated with the CRPS (first column) and compares it to the multivariate performance evaluated with the energy score, which is the multivariate generalization of the CRPS (second column), and with the variogram score, which weights the representation of the correlation structure more strongly (third column). Lower values of all three scores are better. When evaluated over all lead times and vertical levels (first row) the univariate metric of the CRPS score in Fig.  obviously shows the largest improvement for the univariate postprocessing method (NGR-Q). However, the CRPS score for the multivariate ensemble copula coupling methods is on par or only slightly worse. When the multivariate metrics of energy score (Fig. a) and variogram score (Fig. c) are used, the univariate one falls decisively behind even the raw ensemble, as already seen in the rank histograms (Fig. ) and the winter case study (Fig. ).

When the scores are stratified into three pressure ranges (second row of Fig. ) it can be seen in all three metrics that the lowest layer next to the surface is the most difficult one for the raw ensemble. However, this is also the layer where the postprocessing achieves the largest improvements in all three scores. Unsurprisingly, the univariate postprocessing has poorer multivariate metrics of energy and variogram scores than the raw ensemble (with the exception of the energy score of the lowest layer). Improvements from the copula coupling methods in the lowest layer are much larger than in the two layers higher up. One result for which we have no explanation is why the energy score of the raw ensemble is better for the layer between 850 and 700 hPa, which is still partly in the boundary layer for at least part of the times, than for the layer above the altitude of the 700 hPa level. The variogram score, which more strongly weights the structure of the profile, does not show such a behavior.

A stratification by lead time (third row of Fig. ) shows first the expected worsening of the raw ensemble scores for longer lead times. Again, the multivariate scores of the univariately postprocessed (NGR-Q) ensemble are worse than for the raw ensemble. While the copula coupling methods still improve the raw ensemble for a +72 h forecast, 144 h are too far into the future to still achieve an improvement over the raw ensemble data.

The last row of Fig.  shows the vertical temperature gradient, which is only a derived quantity but not the response variable of the postprocessing itself. Therefore it should not come as a surprise that the univariate postprocessing with NGR-Q performs worse than the raw ensemble for the univariate CRPS metric. The copula coupling methods manage to improve the energy score over the raw ensemble. However, they cannot beat the raw ensemble in the variogram score. Since the variogram score weights the correlation aspect of the multivariate performance more strongly, this indicates that if one is interested in accurate temperature gradient forecasts it might be advisable to directly postprocess the gradients themselves instead of deriving them from the postprocessed temperatures.