The large-scale atmospheric flow has attracted the attention of climate scientists since the days of Gilbert Walker almost a century ago (see, e.g., Walker and Bliss, 1932, Rossby, 1940, Horel and Wallace, 1981, and the review of Hannachi et al., 2017, and the references therein). An important driving force behind this interest is the prospect of obtaining a better understanding of the different mechanisms involved and using this knowledge to predict the system on long lead times. Of particular interest is the challenge posed by the extratropical atmospheric variability, which exhibits both nonlinearity and considerable complexity.

The extratropical large-scale circulation can be described by a number of interacting teleconnection patterns, including the North Atlantic Oscillation (NAO) and the Pacific North America (PNA) pattern; see e.g. Wallace and Gutzler (1981), Hannachi et al. (2017) and Feldstein and Franzke (2017). These teleconnections are characterized by typically large spatial scales and low-frequency variability (timescales of more than 10 d; Feldstein, 2000, 2002, 2003; Franzke and Feldstein, 2005), which provides the potential for longer term predictability compared, for example, to synoptic scales.

Being one of the main teleconnection patterns of the Northern Hemisphere (NH), the NAO controls much of the atmospheric variability, particularly over the North Atlantic region, the Mediterranean and the Eurasian continent. It also interacts with other teleconnections, such as the PNA and the El-Niño–Southern Oscillation (ENSO), to produce remote responses. The NAO is known to have a direct, strong impact on surface weather and climate through changes in the atmospheric mass and shifts of the jet stream. It varies on a wide range of timescales, ranging from days to decades and longer (Woollings et al., 2014, 2015). Descriptions of the physical characteristics of the NAO can be found in Benedict et al. (2004), Franzke et al. (2004), Stendel et al. (2016), Hannachi and Stendel (2016) and the references therein.

Several meteorological centres issue regular forecasts of the NAO for various purposes (e.g. NOAA CPC, https://www.cpc.ncep.noaa.gov/products/precip/CWlink/pna/nao_index_ensm.shtml, last access: 5 October 2020). These forecasts are based on dynamical numerical weather prediction (NWP) models. Probabilistic or statistical models are also used to issue such forecasts on short and extended lead times. Statistical models have the advantage of being much simpler than their dynamical analogues, but this advantage comes at the expense of neglecting some interactions with other processes.

The NAO has a number of characteristic features. Its probability density function (PDF) is non-Gaussian, both in the bulk and in both tails of the distribution. Furthermore, the NAO has non-zero autocorrelation for short and long lags (see, e.g., Önskog et al., 2018). A good forecasting model for the NAO should, in principle, be able to reproduce these main properties. The non-Gaussianity implies, in particular, that linear models such as autoregressive (AR) models cannot reproduce the main features. Note, however, that linear models with non-Gaussian noise can of course produce non-Gaussianity (Franzke 2017; Önskog et al., 2018). The NAO is a nonlinear phenomenon and is related to synoptic Rossby wave-breaking (Benedict et al., 2004; Franzke et al., 2004) and regime behaviour (Hannachi et al., 2017). Based on this observation, it is reasonable to expect that nonlinear probabilistic models can be better suited to fit the NAO time series and be able to reproduce the properties mentioned above.

In a previous article (Önskog et al., 2018), we studied the properties of the time series of the daily NAO index, in particular the station-based time series published by Cropper et al. (2015), and found that the distribution of the NAO index has clear non-Gaussian features and long-range dependence (Franzke et al., 2020). An autoregressive model with non-standardized t-distributed noise, taking the values of the index during the last three days into account (AR), provided a good model for the daily NAO index on timescales of up to 2 weeks. By investigating forecasts of the future distribution of the NAO (considering both the expected value and quantiles), we found that some, but not all, properties were well described by the model. Features that the model was unable to replicate included the long-range dependence on timescales of the order of 20 d or more, the different timescales of the positive and negative phases of the NAO and the fact that the negative tail of the NAO distribution is fatter than the positive tail. In this article, we study nonlinear time series models and derive models that reproduce all these features of the daily NAO index. In addition to considering other classes of models compared to our previous article (Önskog et al., 2018), we also develop the test statistics used to compare and evaluate the models further.

Our aim here is to develop a statistical NAO model with which we will be able to identify important dynamical processes controlling the NAO. In order to make progress towards this aim, we here address the following specific research questions: i.

What are the nonlinearities and state-dependencies of the NAO distribution?

Our article is organized as follows. First, in Sect. , we shortly describe the time series of the winter NAO used in the article. Next, in Sect. , we investigate the statistical properties of this time series and describe some of the analyses that we carry out to evaluate the performance of the models that we propose. In Sect. , we briefly review the results on linear models obtained in Önskog et al. (2018) and also provide the rationale behind the model improvement. Section  defines four different nonlinear models for the NAO, including self-excited threshold AR models and state-dependent nonlinear models and models with nonlinear noise. Section  discusses the simulation of climate noise using the models derived, and a concluding discussion is presented in the final section.

In this study, we are using the daily time series of the NAO index published by Cropper et al. (2015). The index is calculated from actual sea level pressure (SLP) observations on Iceland and the Azores, but reanalysis data have been used to fill in the gaps (1888–1905, 1940–1941 and on 145 other occasional days) in the observations. These station-based data are freely available (https://zenodo.org/record/9979#.V_9RG037X4i, last access: 14 March 2019). In the generation of the time series, the seasonality was removed by applying a tension spline method in which a daily annual cycle (of mean SLP and the SLP standard deviation) was interpolated from monthly values and forced so that the average of the daily values of the curve for each month was equal to the monthly means. For details regarding the generation of the time series, we refer to the original source (Cropper et al., 2015).

We have restricted the present study to the winter season, which we here define as starting on 1 December and ending on 28 February (excluding 29 February for all leap years). The study is based on data for 142 consecutive winters from 1872/1873 to 2013/2014. We have chosen the 71 winters for which the month of December falls on an odd year (1873/1874, 1875/1876 and so on) as our training data for fitting of the models. The remaining 71 winters (1872/1873, 1874/1875 and so on) constitute our testing data used for validation of the models. Both training and testing data consist of 6390 data points each. We have chosen to spread the training and testing data evenly during the time period to reduce the effect of any non-stationarity in the data. Note that some of the nonlinear models studied in Sect.  below incorporate values of the NAO index during seasons other than the winter and during the period 1850–1871 as input.

The analyses of the models that we propose for the NAO will focus on the extent to which the models capture the distribution, autocorrelation and timescale of the NAO. The ability of the models to reproduce the distribution of the NAO will be visualized by means of density plots and Q–Q plots but also quantified in terms of the Kullback–Leibler divergence (Kullback, 1959; Kullback and Leibler, 1951; Cover and Thomas, 2006; Lacasa et al., 2012; Kowalski et al., 2011). The symmetrized Kullback–Leibler divergence (KLD) between two probability densities P,Q is defined as follows: 1DKL(P,Q)=DKL(P|Q)+DKL(Q|P),whereDKL(P|Q)=∫-∞∞P(x)log⁡P(x)Q(x)dx and similarly for DKL(Q|P). The KLD is non-negative for all choices of P,Q and provides a measure of the difference between two distributions. Note that DKL(P,P)=0. We will use the KLD as a measure of the distance between the empirical distribution of the NAO and the distributions simulated from the models proposed in this article. In this setting, both P and Q are discrete and are used to calculate the KLD between two discrete distributions, and we have used the default density kernel in R to obtain corresponding smooth probability densities and then the KLD function in R to calculate the KLD.

It should be noted that the comparisons based on KLD that we carry out are somewhat flawed by the fact that the simulated distributions are not fixed but influenced by statistical simulation error. However, in this study each simulated distribution is based on almost 1.3×107 values, and for different realizations of the same model, the KLD between the empirical and the simulated distributions varies by an amount of the order of 10-4, which is negligible compared to the variation between different models. Furthermore, if we let P and Q be the empirical probability density of the training and testing data, respectively, we obtain DKL(P,Q)=3.2×10-3. We do not expect the KLD between the NAO distribution simulated by any of the models proposed and the empirical NAO distribution for the testing data to fall much below this bound, but we note that slightly smaller values could occur, either due to statistical error or due to the distribution predicted by the model being, by chance, closer to the distribution of the testing data than to the distribution of the data it was trained on.

It is well known that the NAO distribution is non-Gaussian in the sense that both the skewness and excess kurtosis of its distribution are non-zero. Moreover, the values of higher moments and, hence, the size of the departure from normality, tend to depend on the current state of the NAO. In Fig. , we have plotted the values of the sample standard deviation, skewness and excess kurtosis as a function of the sample mean for each of the 71 winters in the testing data. We can see that the relations between higher moments and the first moment are close to linear, and Fig.  includes point estimates and confidence intervals for the intercept and the slope of the corresponding linear regression lines. The signs of the intercept and slope are significant for all three higher moments (see also the left plot in Fig. ). In the coming sections, we will investigate how well the proposed models reproduce these linear relations.

A simple measure for the persistence of NAO events is provided by the sample autocorrelation function (ACF). Figure  contains plots of the sample ACF for six choices of lags as a function of the sample mean for each of the 71 winters in the testing data. The sample ACF clearly decreases with increasing lag, but there is also a clear dependence on the sample mean, especially for lags up to 1 week and for lags of the order of 1 month. This can be interpreted as the negative state of the NAO being more persistent than the positive state, and also that the negative state has a negative autocorrelation for timescales around 1 month. The negative phase being more persistent than the positive phase is consistent with the dynamics of the NAO in that the negative phase corresponds to anticyclonic circulation, which is typically more persistent than cyclonic circulation corresponding to the positive NAO phase. In the right plot of Fig.  we have summarized the signs of confidence intervals of the intercept and slope of regressions of the sample ACF versus the winter mean for lags up to 45 d. In the coming sections, we will investigate how well the proposed models reproduce the sample ACF and its dependence on the winter mean.

In this section, we consider a couple of classes of nonlinear time series models and investigate whether they provide a better fit of the daily winter NAO index than standard linear AR models.

The models investigated in this article only use present and past values of the NAO index to compute the future NAO index and not any other covariates. For this reason, the predictive skill of these models will be inferior to that of dynamical models taking many other variables than the NAO index itself into account. To give an example, comparing the forecast skill of one of the nonlinear statistical models proposed in this article to that of the dynamical forecast model issued by NOAA (ftp://ftp.cpc.ncep.noaa.gov/cwlinks/norm.daily.nao.ensf.z500.b14sep2001_current, last access: 17 May 2019), we see that the mean absolute error (MAE) of the 1 d forecast of the statistical model is similar to the MAE of the 6 d forecast of the NOAA model. However, in this context, we stress that the comparison between the dynamical and statistical models is flawed by the fact that the NAO index used by NOAA is based on reanalysis data which are differently normalized than the station-based data used in this article and, as a result, we cannot directly compare the results of the two classes of models. Moreover, the reanalysis data-based NAO index, being based on an empirical orthogonal function analysis, is much smoother than the station-based NAO index and this will affect the predictability of the two types of NAO indices.

We next investigate if simulations of the models proposed in this article generate surrogate time series which have the same properties as the daily winter NAO index on timescales other than days. To this end, we have simulated n=10000 winters of daily NAO data with the AR and ESETAR model with GARCH noise, respectively. Although the ESDNAR model with GARCH noise generally reproduces the daily NAO index slightly better than the ESETAR model with GARCH noise, we have chosen to use the latter model here as it contains terms with longer lags, which could help capturing long-term relationships. To obtain starting values for each of the winters, we have run the models for 2 months. We have calculated weekly, monthly and winter means of the simulated NAO and compared these means to those of the observed NAO index in terms of PDF and sample ACF. Figure  shows that the ESETAR model with GARCH noise reproduces the distribution of weekly means of the daily winter NAO index slightly better than the AR model and that both models perform similarly with respect to the sample ACF. The models slightly underestimate the sample ACF, but the difference is not significant.

Figures  and show that the sample ACF for the monthly means is significantly different from zero for lags of 1 and 4 months, with sample ACF values just above 0.1. For the winter means, the sample ACF is not significantly different from zero for any choice of lag. The lag 1 sample ACF for the monthly means is captured by both models, but not the lag 4 value. We note that the term corresponding to the mean of the winter of 7 years ago does not give rise to any significantly non-zero sample ACF, neither for the winter means nor for the monthly means, and this is reflected by the absence of non-zero sample ACF for the winter NAO for all interannual lags. The ESETAR model with GARCH noise provides a slightly better fit of the NAO PDF than the AR model. It is interesting to note that the distribution of the observed winter means almost appears to be bimodal, but this is probably an effect of too short a time series. A Kolmogorov–Smirnov test for normality has p value 0.21, so we cannot reject the hypothesis that the distribution of winter means is normal.

In this study we have shown that the NAO can be well described by nonlinear autoregressive models. Different models have different benefits and drawbacks, but in general the performances of the various nonlinear models are fairly similar. Putting all the analyses together, the time series model that provides the best description of the station-based daily winter NAO index is of polynomial form with GARCH-type noise. This model is able to reproduce the skewness and fat tails of the observed NAO index and the autocorrelation and timescales of the positive and negative phases of the NAO. The nonlinear and state-dependent NAO model gives some indication of dominant timescales for the NAO. Our model elucidated that the interannual variability mainly affects the positive phase of the NAO, while the negative phase is more affected by processes acting on timescales of 1–3 weeks. These findings are in agreement with the recent results of Caian et al. (2018), who analysed the interannual link between Arctic sea ice and the NAO using reanalysis data and climate model simulations. It was found, in particular, that anomalous Arctic sea ice, associated with a quasi-steady positive gradient of sea ice anomalies about coastal line, act to force precisely the positive NAO phase on interannual timescales.

Interestingly, the ESETAR model for the NAO index reveals significant association with slow timescales of the order of 7 years. A similar long-range behaviour for the NAO has previously been observed in several studies on the spectrum of the NAO, which found peaks of the NAO spectrum in the broad range of 7–10 years (e.g. Gámiz-Fortis et al., 2002; Wunsch 1999). This extended range dependence could be an imprint of the Hurst phenomenon (e.g. Franzke et al., 2020) and has been observed in various branches of natural sciences, such as hydrology (Mandelbrot and Wallis, 1968) and the atmospheric circulation (Franzke et al., 2015b). However, this long timescale did not show up in the sample ACF of the NAO time series, and it is unclear if this is due to the limited length of the data or that the dependence is not manifested in the sample ACF.

The better performance of the SDNAR model compared to the SETAR model might suggest that it is not possible to straightforwardly represent the nonlinearity by conditional linear models as was proposed by Horenko (2010), O'Kane et al. (2013), Franzke et al. (2015a) and Risbey et al. (2015). The idea, in these studies, is that the nonlinear behaviour of the atmospheric circulation can be captured by piecewise linear models. While models in these studies are not exactly similar to the SETAR class of models, they share the underlying idea.

The NAO is not an isolated and self-sustained phenomenon. It interacts with many other processes, including mainly the bottom boundary forcing. The sea surface temperature, for example, varies on a wide range of timescales, with the particular predominance of low frequencies, and can affect the NAO on those scales. Land surface and, in particular, sea ice interactions, can also affect NAO variability on those same scales. A comprehensive probabilistic model for the NAO that is able to provide a better long-term prediction should possibly include those processes as predictors, in addition to the nonlinear terms considered here. This is beyond the scope of this paper and is left for future research.