According to the climatological histogram of Fig. , a unimodal distribution is clearly not appropriate in relation to model visibility. One has to handle low-visibility values separately; there is a second hump at medium to large visibility, and censoring is required at the maximal reported value xmax.

Let g(x|κ,θ) and G(x|κ,θ) denote the probability density function (PDF) and cumulative distribution function (CDF) of a gamma distribution Γ(κ,θ), with shape κ>0 and scale θ>0, respectively, while the notations h(x|μ,σ2) and H(x|μ,σ2) are used for the PDF and the CDF of a normal distribution N0(μ,σ2), with location μ and scale σ>0 left truncated at zero. Furthermore, denote the PDFs of the censored versions of these laws using gc(x|κ,θ) and hc(x|μ,σ2), that is gc(x|κ,θ):=g(x|κ,θ)I{x<xmax}+(1-G(xmax|κ,θ))I{x=xmax},hc(x|μ,σ2):=h(x|μ,σ2)I{x<xmax}+(1-H(xmax|μ,σ2))I{x=xmax}, where IA denotes the indicator function of a set A.

The proposed predictive distribution of visibility is a mixture of censored gamma and censored truncated normal distributions: 1p(x|κ,θ,μ,σ2,ω)=(1-ω)gc(x|κ,θ)+ωhc(x|μ,σ2), where both the weight ω∈[0,1] and the parameters of the component distributions depend on the ensemble forecast. In particular, ω=1/(1+exp⁡(-γf‾ENS)), which is a smooth monotone function of f‾ENS. We remark, that this specific form of the weight is a result of a detailed data analysis, where various parametric functions of the ensemble forecast with range  [0,1]  had been tested, including the forecast-independent weight (see, e.g. ). Furthermore, the mean m=κθ and variance v=κθ2 of the uncensored gamma distribution Γ(κ,θ) are given as m=a0+a12fHRES+a22fCTRL+a32f‾ENS+a4B1(d)+a5B2(d) and v=b0+b12SENS2, while the location and scale of the truncated normal distribution N0(μ,σ2) are expressed as μ=α0+α12fHRES+α22fCTRL+α32f‾ENS+α4B1(d)+α5B2(d) and σ=β0+β12SENS, where functions B1(d) and B2(d) are annual base functions 2B1(d):=sin⁡(2πd/365)andB2(d):=cos⁡(2πd/365) addressing seasonal variations in the mean and location (see, e.g. ), and d denotes the day of the year. Note that modelling location and scale parameters of a parametric predictive distribution as affine functions of the ensemble members and the ensemble variance or standard deviation, respectively, is quite typical in post-processing; see, e.g. or (, Sect. 8.3.2). However, one should admit that model performance is highly dependent on the choice of these link functions.

Following the optimum-score principle of , model parameters γ,a0,a1,…,a5,α0,α1,…,α5,b0,b1,β0,β1∈R are estimated by optimizing the mean value of an appropriate proper-scoring rule, namely the logarithmic score (see Sect. ), over an appropriate training dataset comprising past forecast–observation pairs. In the case where the high-resolution or control forecast is not available, one obviously sets a1=α1=0 or a2=α2=0.

The BMA predictive distribution of visibility X based on 52-member ECMWF ensemble forecasts is given by 3p(x|f1,f2,…,f52,θ1,θ2,…,θ52)=∑k=152ωkh(x|fk;θk), where ωk is the weight, and  h(x|fk;θk) is the component PDF corresponding to the kth ensemble member with parameter vector θk to be estimated with the help of the training data. Note that weights form a probability distribution (ωk≥0,k=1,2,…,52 and ∑k=152ωk=1), and ωk represents the relative performance of the forecast fk in the training data.

In the BMA model of , the conditional PDF h(x|fk;θk) is based on the square root of the forecast fk and consists of two parts. The first models the point mass at the maximal reported visibility xmax using logistic regression as follows: 4logitP(X=xmax|fk)=log⁡P(X=xmax|fk)P(X<xmax|fk)=π0k+π1kfk1/2.

The second part provides a continuous model of visibility given that it is less than xmax  using a beta distribution with shape parameters α,β>0 and support [0,xmax] defined by the following PDF: q(x|α,β):=(x/xmax)α-1(1-x/xmax)β-1B(α,β)xmax,x∈[0,xmax], where B(α,β) is the beta function. Given the kth ensemble member fk, the mean xmaxα/(α+β) and standard deviation xmaxαβ/((α+β)α+β+1) of the corresponding beta distribution are expressed as 5mk=ϱ0k+ϱ1kfk1/2andsk=c0+c1fk1/2, respectively. Note that variance parameters in Eq. () are kept constant for practical reasons. On the one hand, this form reduces the number of unknown parameters to be estimated and helps in avoiding overfitting. On the other hand, as argued by and , in a more general model allowing member-dependent variance parameters c0k and c1k, these parameters do not vary much from one forecast to another.

Now, the conditional PDF of visibility given the kth ensemble member fk is h(x|fk;θk)=P(X<xmax|fk)q(x|fk)I{X<xmax}+P(X=xmax|fk)I{X=xmax}, where P(X=xmax|fk) is defined by Eq. (), q(x|fk) denotes the beta distribution with the mean and standard deviation specified by Eq. (), and θk:=(π0k,π1k,ϱ0k,ϱ1k,c0,c1).

Parameters π0k and π1k are estimated from the training data by logistic regression, and the mean parameters ϱ0k and ϱ1k are obtained using linear regression connecting the visibility observations of less than xmax to the square roots of the corresponding ensemble members; on the other hand, for estimating weights ωk and variance parameters c0 and c1, one uses the maximum-likelihood approach with the EM algorithm to maximize the likelihood function. For more details, we refer the reader to and note that, following, again, , we do not distinguish between the exchangeable ensemble members f3,f4,…,f52 and assume ω3=ω4=…=ω52 and θ3=θ4=…=θ52. Furthermore, if some of the ensemble forecasts are missing then the corresponding weights should be set to zero.

The parameters of the mixture and BMA predictive PDFs described in Sect.  and , respectively, are estimated separately for each individual lead time. For a given day d and lead time ℓ, the estimation is based on training data (observations and matching forecasts with the given lead time) from an n d long time interval between calendar days d-ℓ-n+1 and d-ℓ; that is, one considers data of the latest n calendar days when the date of validity of the ℓ days ahead forecasts precedes the actual day d. The optimal length of the rolling training period is determined by comparing the predictive performance of post-processed forecasts for various lengths n.

As both investigated datasets consist of forecast–observation pairs for several SYNOP stations, one can consider different possibilities for the spatial composition of the training data. The simplest and most parsimonious approach is regional modelling , where all investigated locations are treated together and share a single set of model parameters. Regional models allow extrapolation of the predictive distribution to locations where only forecasts are available (see, e.g. ); nonetheless if the ensemble domain is too large and if the stations have quite different characteristics then this approach is not really suitable and, as demonstrated by, e.g. or , might even fail to outperform the raw ensemble. In contrast, local models result in distinct parameter estimates for different locations as they are based only on the training data of each particular site. In this way, one can capture local characteristics better so that local models usually outperform their regional counterparts as long as the amount of training data is large enough. Thus, one needs much longer training windows than in the regional case. For instance, suggest 720, 365, and 1816 d rolling training periods for EMOS modelling of temperature, wind speed, and precipitation accumulation, respectively. Finally, the advantages of regional and local parameter estimations can be combined through the use of semi-local techniques, where either the training data of a given location are augmented with the data of sites with similar characteristics or the ensemble domain is divided into more homogeneous subdomains, following which, within each subdomain, regional modelling is performed. In the case studies of Sect. , besides local and regional parameter estimation, we also consider the clustering-based semi-local approach suggested by . For a given date of the verification period, to each observation station, we first assign a feature vector depending on both the station climatology and the forecast errors of the raw ensemble mean over the training period. In particular, similarly to , we consider 24-D feature vectors comprising 12 equidistant quantiles of the empirical CDF of the training observations and 12 equidistant quantiles of the empirical CDF of the mean of the corresponding exchangeable members. Based on these feature vectors, the stations are grouped into homogeneous clusters using k-means clustering (see, e.g. , Sect. 16.3.1), and, within each cluster, regional estimation is performed, resulting in a separate set of model parameters for each cluster. As the training period rolls ahead, the feature vectors are updated, and the locations are dynamically regrouped.

It is advised that forecast skill be evaluated with the help of proper-scoring rules (see, e.g. ), which can be considered to be loss functions aiming to maximize the concentration (sharpness) of the probabilistic forecasts subject to their statistical consistency with the corresponding observations (calibration). One of the most used proper-scoring rules is the logarithmic score (LogS; ), which is the negative logarithm of the predictive PDF evaluated at the validating observation. The other very popular proper score is the continuous ranked probability score (CRPS; , Sect. 9.5.1). For a forecast provided in the form of a CDF F and a real value x representing the verifying observation, the CRPS is defined as 6CRPS(F,x):=∫-∞∞[F(y)-I{y≥x}]2dy=E|X-x|-12E|X-X′|, where IH denotes the indicator function of a set H, while X and X′ are independent random variables distributed according to F with the finite first moment. Both the LogS and the CRPS are negatively oriented scores; that is, smaller values mean better predictive performance. In most applications, the CRPS has a simple closed form (see, e.g. ); however, this is not the case for the predictive distributions corresponding to the mixture and BMA models of Sect.  and , respectively. In such cases, based on the representation on the right-hand side of Eq. (), which also implies that the CRPS can be reported in the same units as the observation, one can consider the Monte Carlo approximation of the CRPS based on a large sample drawn from F (see, e.g. ). In the case studies of Sect. , the predictive performances of the various forecasts with a given lead time are compared using the mean CRPS over all forecast cases in the validation period.

Furthermore, the forecast skill of the competing forecasts with respect to dichotomous events can be quantified with the help of the mean Brier score (BS; , Sect. 9.4.2). For a predictive CDF F and in the event where the observed visibility x does not exceed a given threshold y, the BS is defined as BS(F,x;y):=[F(y)-I{y≥x}]2. Thus, the CRPS is just the integral of the BS over all possible thresholds.

For a probabilistic forecast F, one can assess the improvement with respect to a reference forecast Fref in terms of a score S by using the corresponding skill score , defined as SSF:=1-S‾FS‾Fref, where S‾F and S‾Fref denote the mean score values corresponding to forecasts F and Fref, respectively. Skill scores are positively oriented (the larger the better), and, in our case studies, we report the continuous ranked probability skill score (CRPSS) and the Brier skill score (BSS).

Calibration and sharpness can also be investigated by examining the coverage and average width of (1-α)100%,α∈]0,1[, central prediction intervals (intervals between the lower and upper α/2 quantiles of the predictive distribution). Coverage is defined as the proportion of validating observations located in this interval, and, for a properly calibrated predictive distribution, this value should be around (1-α)100%. Note that level α is usually chosen to match the nominal coverage of (K-1)/(K+1)100% of a K-member ensemble, which allows a direct comparison with the raw forecasts.

Further simple tools for assessing the calibration of probabilistic forecasts are the verification rank histogram (or Talagrand diagram) of ensemble predictions and the probability integral transform (PIT) histogram of forecasts given in the form of predictive distributions. The Talagrand diagram is the histogram of the ranks of the verifying observations with respect to the corresponding ensemble forecasts (see, e.g. , Sect. 9.7.1), and, in the case of a properly calibrated K-member ensemble, the verification ranks should be uniformly distributed on {1,2,…,K+1}. The PIT is the value of the predictive CDF evaluated for the verifying observation with possible randomization in the points of discontinuity (see, e.g. , Sect. 9.5.4). The PIT values of calibrated predictive distributions follow a standard uniform law, and, in this way, the PIT histogram can be considered to be the continuous counterpart of the verification rank histogram.

Furthermore, the mean and the median of the predictive distributions, as well as the ensemble mean and median, can be considered to be point forecasts for the corresponding weather variable. As the former optimizes the root mean squared error (RMSE) while the latter optimizes the mean absolute error (MAE), we use these two scores to evaluate the accuracy of point predictions .

Finally, some of the skill scores are accompanied by 95 % confidence intervals based on 2000 block bootstrap samples obtained using the stationary bootstrap scheme, with mean block length derived according to . In this way, one can get insight into the uncertainty in verification scores and the significance of score differences.

In this case study, the predictive performances of the competing forecasts are compared using data of the calendar year 2021. For the 51-member ECMWF ensemble (control forecast and 50 exchangeable members), the mixture model has 15 free parameters to be estimated, and the comparison of the forecast skill of regional models based on training periods with lengths of 100, 150, … , 350 d reveals that the longest considered training period results in the best predictive performance. This 350 d training window is also kept for local and semi-local modelling, where the 13 locations are grouped into six clusters. Semi-local models with three, four, and five clusters have also been tested; however, these models slightly underperform compared to the chosen one. Furthermore, as mentioned, the 11 parameters of the BMA model are estimated regionally using 25 d rolling training periods, which means a total of 325 forecast cases for each training step. Hence, the data-to-parameter ratio of the regional BMA approach (325/11=29.5) is slightly above the corresponding ratio of the local mixture model (350/15=23.3). Note that, in this case study, validating visibility observations are reported up to 75 km; hence, the support of all investigated post-processing models is limited to the 0–75 km interval with a point mass at the upper bound.

Figure a indicates that, in terms of the mean CRPS, all investigated forecasts considerably outperform the raw visibility ensemble. Note that the clearly recognizable oscillation in the CRPS can be explained by the four different observation times per day, and the raw ensemble exhibits the strongest dependence on the time of the day, having the highest skill at 06:00 UTC. Parametric models are superior to climatology only for shorter forecast horizons, and their advantage gradually fades with the increase in the lead time. The difference between post-processed forecasts and climatology is more visible in the CRPSS values of Fig. b. Skill scores of the locally and semi-locally trained mixture models are positive for all lead times, and the difference between these forecasts is negligible. Up to 192 h, the Mixed-R approach also outperforms climatology, and it is clearly ahead of the BMA, which results in positive CRPSS values only for shorter forecast horizons (6–126 h). This ranking of the competing methods is also confirmed by Table , providing the overall mean CRPS values of calibrated and climatological forecasts as proportions of the mean CRPS of the raw ECMWF visibility ensemble. We remark that a similarly good performance in terms of climatology with respect to raw and BMA post-processed visibility forecasts of the University of Washington mesoscale ensemble was observed by .

In Fig. , the CRPSS values of the best-performing locally trained mixture model are accompanied by 95 % confidence intervals, which helps in assessing the significance of the differences in CRPS. The superiority of the Mixed-L forecast over climatology in terms of the mean CRPS is significant at a 5 % level up to 150 h, and between 42 and 174 h, it significantly outperforms the BMA model as well. We remark that, for the latter approach, after 96 h, the CRPSS with respect to climatology fails to be significantly positive (not shown).

The analysis of the Brier skill scores plotted in Fig.  slightly tones the picture of the performance of post-processed forecasts. For visibility not exceeding 1 km, none of the competitors outperform climatology (Fig. a), and calibrated predictions are superior to raw ensemble forecasts only for short lead times. With the increase in the threshold, the positive effect of post-processing is getting more and more pronounced, and the ranking of the different models starts matching the one based on the mean CRPS. From the competing calibration methods, the locally and semi-locally trained mixed models consistently display the highest skill, and, for the largest threshold value of 10 km, they outperform climatology up to 204 h (Fig. d).

The verification rank and PIT histograms of Fig.  again illustrate the primacy of climatology over the raw ensemble and the improved calibration of post-processed forecasts. Raw ECMWF visibility forecasts are under-dispersive and tend to overestimate the observed visibility; however, these deficiencies improve with the increase in the forecast horizon. Climatology results in almost-uniform rank histograms with just a minor under-dispersion, and there is no visible dependence on the forecast horizon. Unfortunately, none of the four investigated post-processing models can completely eliminate the bias of the raw visibility forecasts, which is slightly more pronounced for longer lead times.

Furthermore, the coverage values of the nominal 96.15 % central prediction intervals depicted in Fig. a are fairly consistent with the shapes of the corresponding verification rank and PIT histograms. The under-dispersion of the raw ensemble is confirmed by its low coverage, which shows an increasing trend and a clear diurnal cycle and ranges from 25.58 % to 65.21 %. As one can observe from the corresponding curve of Fig. b, the improvement of the ensemble coverage with the increase in the forecast horizon is a consequence of the increase in spread, which results in expanding central prediction intervals. The price of the almost-perfect coverage of climatology with a mean absolute deviation from the nominal value of 0.53 % is the much wider central prediction interval. The coverage values of post-processed forecasts decrease with the increase in the lead time; the corresponding mean absolute deviations from the nominal 96.15 % are 3.55 % (Mixed-L), 3.21 % (Mixed-C), 3.55 % (Mixed-R), and 3.39 % (BMA). However, this negative trend in coverage, especially in the case of the locally and semi-locally trained mixed models, is combined with increasing average width, which can be a consequence of the growing bias.

Finally, in terms of the RMSE of the mean forecast, all post-processing approaches outperform both the raw ensemble and the climatology for all lead times (see Fig. a); however, their advantage over climatology is negatively correlated with the forecast horizon. Mixed-L and Mixed-C approaches result in the lowest RMSE values, followed by the Mixed-R and BMA forecasts (see also Fig. b), the order of which perfectly matches the ranking based on the mean CRPS (Fig. b) and the mean BS for all studied thresholds (Fig. ).

Since, in the EUPPBench benchmark dataset, the 51-member ECMWF ensemble forecast is augmented with the deterministic high-resolution prediction, the mixture model (Eq. ) has 17 free parameters to be estimated, whereas, for the BMA predictive PDF (Eq. ), the parameter vector is 16-D. As mentioned, BMA modelling is based on 25 d regional training, while, to determine the optimal training-period length for mixed models, we again compare the skill of the regionally estimated forecasts based on rolling training windows of 100, 150, …, 350 d. In the case of the EUPPBench visibility data, the skill of the different models is compared with the help of forecast–observation pairs for the calendar year 2018. In contrast to the previous case study, where the longest tested training period of 350 d is preferred, here, the 100 d window results in the best overall performance. Using the same training-period length, we again investigate local modelling and semi-local estimation based on four clusters. Taking into account the maximal reported visibility observation in the EUPPBench benchmark dataset, now the mixed and BMA predictive distributions have point masses at 70 km.

Again, Fig. a shows the mean CRPS of post-processed, raw, and climatological EUPPBench visibility forecasts as functions of the lead time, while in Fig. b, the CRPSS values of the mixed and BMA models with respect to the 52 d climatology are plotted. Similarly to the case study of Sect. , climatological and post-processed forecasts outperform the raw ensemble by a wide margin; however, now the advantage of post-processing over climatology is not so obvious, and the ranking of the calibration methods also differs. From the four investigated models, the Mixed-C model results in the lowest mean CRPS for all lead times but 12 h; nevertheless, even this approach shows negative skill against climatology for lead times corresponding to 12:00 UTC observations. Local modelling (Mixed-L) is competitive only for short forecast horizons, which might be explained by the short training period leading to numerical issues during parameter estimation due to the low data-to-parameter ratio. In general, the skill scores of all post-processing methods show a decreasing trend, with the BMA having the mildest slope. Based on Table , providing the improvement in the overall mean CRPS over the raw ECMWF ensemble, one can establish a ranking of Mixed-C – BMA – Mixed-L – Climatology – Mixed-R. Note that the improvements provided here are much larger than the ones in Table . In terms of the mean CRPS, the 52-member EUPPBench visibility forecasts are behind the more recent 51-member ensemble predictions studied in Sect. . This dissimilarity in forecast performance is most likely due to the consecutive improvement in the ECMWF IFS; however, it might also be related to the difference in the forecast domains (see Fig. ).

Furthermore, according to Fig. , even for 00:00, 06:00, and 18:00 UTC observations, the advantage of the best-performing Mixed-C model over climatology is significant at a 5 % level only up to 48 h, whereas the difference in skill from the BMA approach is significant at 6, 24, and 30 h only. Note that the CRPSS of the BMA model with respect to climatology is significantly positive at 5 % only for lead times of 18, 24, 40, and 48 h (not shown).

The Brier skill scores of Fig.  lead us to similar conclusions as in the previous case study. For the lowest threshold of 1 km, all forecasts underperform in relation to climatology (Fig. a); however, the skill of post-processed predictions improves when the threshold is increased. For the 3, 5, and 10 km thresholds, the ranking of the various models is again identical to the ordering based on the mean CRPS (see Fig. b), with the Mixed-C approach exhibiting the best overall predictive performance, closely followed by the BMA model. For the largest threshold of 10 km, up to 54 h, climatology is outperformed even by the least skilful Mixed-L approach (see Fig. d), whereas the leading semi-locally trained mixed model results in a positive BSS up to 102 h.

The verification rank histograms of the raw EUPPBench visibility forecasts depicted in Fig.  show a much stronger bias than the corresponding panel of Fig. , while the improvement with the increase in the forecast horizon is less pronounced. Climatology is also slightly biased but in the opposite direction, whereas the verification rank histograms of all post-processed forecasts are closer to the desired uniform distribution than in the case study of Sect. . Here the locally trained mixed model exhibits the strongest bias; however, neither the verification rank histograms of climatology nor the PIT histograms of the calibrated forecasts indicate a visible dependence on the forecast horizon.

The fair calibration of climatological and post-processed forecasts can also be observed in Fig. a, displaying the coverage values of the nominal 96.23 % central prediction intervals. Semi-locally and regionally trained mixed models and climatology result in almost perfect coverage, closely followed by the BMA model; the corresponding mean absolute deviations from the nominal value are 0.69 %, 0.90 %, 0.72 %, and 1.81 %. The Mixed-L model is slightly behind its competitors, with a mean absolute deviation of 3.00 %, whereas the maximal coverage of the raw EUPPBench ensemble does not reach 60 %. Note that the ranking of the various predictions, the increasing coverage of the raw ensemble, and the lack of a visible trend in the coverage values of post-processed forecasts and climatology are pretty much in line with the shapes of the corresponding histograms of Fig. . In general, the average widths of the investigated 96.13 % central prediction intervals (Fig. b) are rather consistent with the matching coverage values. Nevertheless, one should remark that the best-performing Mixed-C model results in sharper predictions than climatology and the Mixed-R and BMA approaches.

Finally, according to Fig. , in terms of the RMSE of the mean, we see a similar behaviour and ranking of the different forecasts as in the case of the mean CRPS (see Fig. ), while the raw ensemble is clearly behind the other forecasts, with the increase in the lead time climatology becoming more and more competitive, especially at forecast horizons corresponding to 12:00 UTC observations. However, up to 30 h, both locally and semi-locally trained mixed models result in a lower RMSE than the climatological forecast, and the Mixed-C approach consistently outperforms all other calibration methods for all lead times but 12 h.