This study validates the near-surface temperature and precipitation output from decadal runs of eight atmospheric ocean general circulation models (AOGCMs) against observational proxy data from the National Centers for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR) reanalysis temperatures and Global Precipitation Climatology Project (GPCP) precipitation data. We model the joint distribution of these two fields with a parsimonious bivariate Matérn spatial covariance model, accounting for the two fields' spatial cross-correlation as well as their own smoothnesses. We fit output from each AOGCM (30-year seasonal averages from 1981 to 2010) to a statistical model on each of 21 land regions. Both variance and smoothness values agree for both fields over all latitude bands except southern mid-latitudes. Our results imply that temperature fields have smaller smoothness coefficients than precipitation fields, while both have decreasing smoothness coefficients with increasing latitude. Models predict fields with smaller smoothness coefficients than observational proxy data for the tropics. The estimated spatial cross-correlations of these two fields, however, are quite different for most GCMs in mid-latitudes. Model correlation estimates agree well with those for observational proxy data for Australia, at high northern latitudes across North America, Europe and Asia, as well as across the Sahara, India, and Southeast Asia, but elsewhere, little consistent agreement exists.

Atmospheric ocean general circulation models (AOGCMs or simply
GCMs) are being developed by various scientific organizations to study
climate science, including the human impact on climate change. Recently, the
World Climate Research Programme organized the Coupled Model Intercomparison
Project Phase 5

Toward this end, we perform a

There have been several works that validate climate model output using
various statistical methods. For instance,

Precipitation and near-surface air temperature were chosen for this study
because they are two of the most important climate model output fields, as
well as the two variables most commonly downscaled

In the literature for statistical analysis with climate model output and
observations, climate variable fields are often considered individually as
univariate spatial fields. For example,

There are few studies of the cross-dependence of two climate variables from
climate model output.

In this paper, our focus is to validate CMIP5 ensembles by investigating bivariate properties of climate variables and compare them across output from multiple climate models as well as observation proxies. Our goal is to perform validation on more than just means and variances of temperature and precipitation fields. In particular, we are interested in how cross-correlation of surface temperature and precipitation compares across model ensembles and observational proxy data. Considering the cost of running each climate model, validating climate models through various statistics in addition to simple means and variances is valuable. To the extent that each climate model accurately represents the true nature of Earth's climate, any statistics beyond means and variances should be comparable across multi-model ensembles, as well as corresponding observational proxies. Furthermore, smoothness and cross-correlation are among the key important quantities in describing the underlying distribution of the climate processes, so we also compare local smoothness of each variable across model ensembles and proxies.

The rest of the paper is organized as follows. Section 2 describes the two types of data used in this study, observational proxy data and GCM output. Section 3 introduces the statistical methodology used to estimate the statistical model parameters. Section 4 summarizes results and Sect. 5 makes recommendations for further study.

Near-surface air temperature values for 1981–2010 are taken from the
NCEP/NCAR reanalysis data, provided by the Earth System Research Laboratory
in the National Oceanic and Atmospheric Administration
(

Temperature (

NCEP reanalysis data is based on a system that uses forecasts and hindcasts
to fill in the gaps between missing data, which works well for fields such as
temperature.

Precipitation data was taken from the GPCP

The temperature values and precipitation values are defined at slightly
different grid points, so each field had to be adjusted to reconcile them to
the same

Output from eight GCMs were obtained from the Program in Climate Model
Diagnosis and Intercomparison (PCMDI) server
(

The data from these runs were used to compute 30-year seasonal averages for boreal summer (JJA) and boreal winter (DJF), again, for the years 1981–2010. Decadal runs were not available for HAD-GEM2-ES, the high-resolution Hadley Centre Earth Systems model, so data from part of one historical run was used, specifically December 1972–August 2003.

All precipitation values were cube-root transformed. The seasonal average
temperature fields from one GCM, that of the Geophysical Fluid Dynamics
Laboratory (GFDL), are shown in the top half of Fig.

Names of modeling institutes and sources for observational proxy-data data.

1000 (precipitation)

Observed proxy data as well as climate model simulation data are analyzed
using a two-step process. First, we fit a regression mean structure
(Sect.

Region definitions

Clearly the isotropic assumption is not reasonable on the globe as a whole
due to the spatially varying nature of the processes' spatial dependence
structure. Hence, the data are first blocked into 31 regions, that is,
21 land regions as defined by

Climate region definitions.

Before modeling the spatial dependence structure of the two climate
variables, we first filter the mean structure of each of the two fields,
temperature and cube-root transformed precipitation, for each region
separately, using simple linear regression. We write,

Residuals after mean filtering from observational proxy data (DJF),
in eastern North America (ENA):

Mean filtering coefficients by region number (land regions only) for
temperature intercept

North America and Greenland JJA residuals: GFDL for

North America and Greenland DJF residuals: GFDL for

To choose the appropriate mean structure, in addition to Eq. (

Figure

The abscissa is the region number (1

Sample region (WNA) parameter point estimate (asymptotic standard error) values.

Parameter estimates for observational proxy data and GCMs for each season by latitude band.

Figures

We denote bivariate data consisting of the residuals from
Eq. (

For modeling

The covariance parameters are the variance,

Here,

A parsimonious version of the bivariate Matérn function imposes a condition
on the covariance parameters:

Co-located correlation is the spatial correlation between the precipitation
and temperature fields after having been averaged over time, which is
fundamentally distinct from the more commonly computed temporal correlation
at each location (as in

Many of the results are presented using box plots of point estimates;
Figs.

Parameter estimates by source for each season by latitude band,
for JJA (left column) and DJF (right column).

Parameter estimates by source for each season by latitude band, for JJA (left column) and DJF (right column).

Figure

Note that box plots for

Figure

Correlation coefficient maps:

Figure

For Mid-S JJA and North DJF, GCMs underestimate precipitation variance,
consistent with

To explore precipitation variance for individual institute's GCMs,
Fig.

When fit simultaneously with precipitation residuals using a common spatial
scale, the NCEP temperature residual field tends to have a smoothness
coefficient of about 1.0, consistent with results of previous studies

Because each GCM is evaluated at its native resolution (Table

The estimates for the smoothness parameter of the precipitation field in
Fig.

Comparing the smoothness coefficients for temperature and precipitation,
those for precipitation are mostly larger than those for temperature,
consistent with results of

Estimates for the co-located cross-correlation parameter between
precipitation and temperature fields are given in Fig.

Maps of these correlation estimates are shown for observational proxy data
and two GCMs (GFDL and NCAR) in Fig.

The values for the maps in Fig.

Finally, the estimated correlation length, which is constrained to be the
same for both fields' covariance as well as the cross-covariance function,
ranges from about 200 to 2000 km (see Figs.

We present an approach for statistical models of the joint distribution of temperature and precipitation accounting for spatial dependence structure. Using a parsimonious bivariate covariance model, we compute spatial coefficients describing variation and smoothness for each field individually, as well as spatial scale and co-located correlation between the fields. Our results generally agree with previous work with several notable exceptions. Temperature variance increases with latitude in boreal winter while remaining small in summer; GCMs track these well, though with some tendency to overestimate the variance of reanalysis data. Precipitation variance decreases with increasing latitude and GCMs match observed values well except for southern mid-latitudes. For DJF GPCP data, precipitation variance seems to increase slightly with latitude, but decreases with latitude for GCMs. It is not clear for any latitude band or season that GCMs consistently underestimate or overestimate this variance. Smoothness estimates for temperature fields tend to be smaller than corresponding smoothness estimates for precipitation, and both tend to decrease with increasing latitude. Observational proxies, particularly in the tropics, tend to have a wider range, and smoother fields, than GCMs. Smoothness estimates for precipitation are almost entirely larger than those for temperature, both for proxies and all models collectively. For high latitudes, GCMs are collectively estimated to have about the same smoothness for precipitation and temperature. Overall, GCMs tend to predict smoothness coefficients near those predicted by observation proxies except in the tropics, where GCMs tend to predict less smooth fields for both temperature and precipitation. Co-located cross-correlations from GCMs largely agree among models but differ from estimates from observation proxies in many regions, particularly in equatorial and mid-latitude zones. Temporal correlations are sensitive to any mean, seasonal variation, or trend that may not have been fully removed and, to some extent, this is true for spatial correlation estimates, though we have worked hard to minimize this. Since the aim of this study is to validate GCMs against observational proxy data, having the “right” mean structure model is not the dominant concern, but rather the consistent application across all GCMs and proxy data. Given this perspective, it is clear that GCMs are not fully consistent with one another or with proxies, except for a few regions. Exploration of this aspect of this modeling effort deserves additional attention, as well as a more complete treatment of all ocean regions.

To improve the latitudinal resolution and minimize the effects of specific mean structure filters, it may be possible to redefine spatial regions to smaller, more geographically appropriate areas, but only as GCM model resolution grows. Using some form of non-stationary covariance formulation would be a more elegant way than regional “chunking” to address this problem.

Several parameter estimates for equatorial zones show differences between JJA
and DJF, notably

Finally, the statistical methods used here for two fields can be naturally extended to multivariate climate variables, for example, to quantify the cross-correlations of more than two variables, accounting for their spatial dependence.

Once the mean structure from each field (cube-root precipitation or
temperature) is subtracted separately as done in Sect.

The log-likelihood function,

In general, the Hessian matrix is defined as

Practically, the Nelder–Mead algorithm returns the observed Fisher information, so the asymptotic standard errors are obtained by computing the square root of the diagonal elements of its inverse.

R. Philbin obtained data and reformatted it, developed and executed all code, prepared the manuscript and developed summaries. M. Jun designed study idea and indicated data source, provided core algorithm, consulted on technical details, read and fine-tuned drafts.

The authors thank G. North and R. Saravanan for their very constructive and
helpful comments regarding this work. This publication is based in part on
work supported by award no. KUS-C1-016-04, made by King Abdullah University
of Science and Technology (KAUST). Also, Mikyoung Jun's research was
supported by NSF grant DMS-1208421. The authors acknowledge the World Climate
Research Programme's (WCRP) Working Group on Coupled Modelling (WGCM), which
is responsible for CMIP and thank the climate modeling groups (listed in
Table