Evaluation of CMIP5 Climate Models Using Historical Surface Air Temperatures in Central Asia

Abstract

Using historical data compiled by the Climate Research Unit, spatial and temporal analysis, trend analysis, empirical orthogonal function (EOF) analysis, and Taylor diagram analysis were applied to test the ability of 24 Coupled Model Intercomparison Project Phase 5 (CMIP5) climate models to accurately simulate the annual mean surface air temperature in central Asia from the perspective of the average climate state and climate variability. Results show that each model can reasonably capture the spatial distribution characteristics of the surface air temperature in central Asia but cannot accurately describe the regional details of climate change impacts. Some of the studied models, including CNRM-CM5, GFDL-CM3, and GISS-E2-H, could better simulate the high- and low-value centers and the contour distribution of the surface air temperature. Taylor diagram analysis showed that the root mean square errors of all models were less than 3, the standard deviations were between 8.36 and 13.45, and the spatial correlation coefficients were greater than 0.96. EOF analysis showed that the multi-model ensemble can accurately reproduce the surface air temperature characteristics in central Asia from 1901 to 2005, including the rising periods and the fluctuations of the north and south inversion phases. Overall, this study provides a valuable reference for future climate prediction studies in central Asia.

1. Introduction

Climate system models are considered important tools to simulate and predict the future impacts of climate change [1]. Many developed climate models are able to accurately simulate the main features of global warming under natural and human forces [2]. Variations among climate models tend to make the results of climate predictions uncertain, often requiring the assessment of the simulation capabilities of different models [3,4,5,6]. During the last century, the increased concentration of greenhouse gases has been an important cause of global warming [7]. Temperatures in the northern hemisphere increased by approximately 1 °C in the 20th century [8]. As a result of global warming, a series of problems are caused, such as water availability [9,10,11], food security [12,13], changes in ecological environments [14,15,16], biological diversity [17,18], and impacts on human health [19,20,21]. Addressing climate change and its impacts has become an important topic in recent decades, particularly for policymakers who need the latest climate change information to harmonize between man and nature.

Researchers have focused on climate prediction models for northern Eurasia [22], East Asia [23], central Africa [24], and China [25]; however, few studies have been performed in Central Asia (CA). CA is one of the driest regions in the world with a fragile ecological environment. Most of CA is covered by dry land, including the Gobi Desert. Therefore, water is an important resource for human survival and ecosystem stability in this region. The ecosystems in the arid regions of CA are sensitive to climate change [26]. The impacts of global warming in CA include the increase in the amount of winter precipitation [27] and changes in the size of the inland lakes [28]. There is a lack of systematic research on the historical surface air temperatures (SATs) for this important arid region, including their evolution in the future, and their spatial and temporal distribution characteristics.

Therefore, this work aims to evaluate the ability of 24 climate models to simulate historical SATs in CA to address three main questions: (1) How effective are the climate models in simulating the observed data? (2) How well does the multi-model ensemble (MME) of the 24 climate models reproduce the observed data? (3) What is the spatial and temporal distribution of the observed and simulated data? Taylor’s diagram and empirical orthogonal function (EOF) analyses are applied to assess the quality of each climate model’s simulation capabilities. The results provide a reliable reference for future climate prediction studies in CA.

2. Studied Area and Data

2.1. Studied Area

The study area in Figure 1 measures approximately four million square kilometers in CA, has a population of approximately 60 million inhabitants, and includes five former Soviet republics: Kazakhstan, Uzbekistan, Kyrgyzstan, Turkmenistan, and Tajikistan [29]. CA extends up to the Caspian Sea in the west, with China in the east, Russia in the north, and Iran, Afghanistan, and Pakistan in the south [30]. Located in the hinterland of Eurasia, far from the ocean, CA has a typical continental climate with scarce precipitation and widely varying temperatures.

2.2. Data

Monthly temperature data were obtained from the Climate Research Unit (CRU TS v4.01, CRU) from meteorological stations [31]. The CRU-compiled data is available for 1901–2016 with a horizontal resolution of 0.5° × 0.5°. In this study, this observed data for 1901–2005 was used to align the climate models from the Coupled Model Intercomparison Project Phase 5 (CMIP5).

The data used in the 24 CMIP5 models for 1901–2005 were obtained from the Global Climate Model Data Archive; the climate models, along with their horizontal resolutions and sources, are summarized in Table 1 (available at http://cmip-pcmdi.llnl.gov/cmip5/index.html (accessed on 11 July 2020). For comparison with the data observed by the CRU, all models were interpolated to a 2.5° × 2.5° grid to facilitate the intercomparison. The 2.5° × 2.5° resolution is the mean resolution of all models (including CRU data).

3. Methodology

Climate models have different horizontal resolutions; this could affect the comparison results. To counter this effect, the climate model data were first converted to a 0.5° × 0.5° grid through bilinear interpolation.

The Taylor diagram was then applied to compare the climate model assessment studies [32]. Taylor diagrams show the correlation statistics of multiple climate models, such as the correlation coefficient, root mean square error (RMSE), and standard deviation, and thus can be used to summarize a model’s accuracy and consistency.

EOF analysis, similar to principle component analysis, is widely used in atmospheric research to reduce the dimension of the SAT field by dividing the variables into several major physical modes [33,34,35]. Using EOF analysis, the climate field can be expressed using spatial and temporal variables. The first several modes represent the dominant modes of a variable and capture the main information about its changes. EOF analysis can be expressed as follows:

(1)${\phi }_{ij}= {\displaystyle \sum }_{k=1}^m{U}_{ki}{Z}_{kj}$

where i = 1, … m, j = 1, …, n, m is the number of grids, n is length of time, φ_ij is the i^th component of the j^th random vector for the centralized and normalized data, U_ki represents the eigenvector values of the i^th site in the t^th component, also known as EOFs, and Z_kj is the time coefficient for each eigenvector, also known as PCs. The first several EOFs can best represent the spatial characteristics of the SAT.

4. Results

4.1. Climatology

The spatial distributions of the 24 CMIP5 climate models and the observed annual mean SAT by the CRU for 1901‒2005 are presented in Figure 2. According to the observed temperature data (Figure 2y), the annual mean SAT in most parts of the studied area was above 0 °C. Areas maintaining an annual mean SAT of >10 °C included the southeastern region of Uzbekistan and most of Turkmenistan. The south-central region of Turkmenistan demonstrated the highest annual mean SAT at approximately 15 °C, whereas the regions with the lowest annual mean SAT were in the northern part of Tajikistan and eastern Kyrgyzstan.

When compared with SAT data provided by the CRU, all studied models accurately simulated the decreasing distribution of SAT from southwestern Turkmenistan to northeastern Kazakhstan. Similarly, they each accurately simulated the high-value center in Turkmenistan and the low-value center in Tajikistan. However, large variations were observed in the range of the high-SAT and low-SAT regions; the models most accurately portraying the SAT spatial distribution included CNRM-CM5 (Figure 2f), GFDL-CM3 (Figure 2j), GISS-E2-H (Figure 2l), GISS-E2-R (Figure 2m), MRI-CGCM3 (Figure 2v), NorESM (Figure 2w), and NorESM‒ME (Figure 2x). Models were classified as having a relatively good simulation effect if they could accurately simulate the high-value and low-value SAT centers.

The anomaly in the annual mean SAT of the observational data provided by the CRU and the 24 CMIP5 climate models is shown in Figure 3. Three warming periods between 1901 and 2005 (in the 1970s and early 21st century) were apparent in the studied area. Although most of the studied climate models reasonably reproduced the rising SAT trends in the area, differences were present among the models as well as between the observed data and the models as most climate models overestimated the annual mean SAT. In general, the differences among the models were mainly due to the differences in their description of physical processes, such as water vaporization, land surface, cloud and radiation response, and feedback mechanisms. Moreover, each model also has great uncertainty in the description of each of these processes. The bias among the models is the result of the differences and uncertainties in their process descriptions.

A Taylor diagram of the 24 CMIP5 climate models is shown in Figure 4. Most of the studied models demonstrated a similar ability to simulate the SAT. All models had an RMSE near 3; the three with the lowest RMSE were CNRM-CM5, CSIRO-Mk3-6-0, and MIROC5 with an RMSE of 2.89, 2.88, and 2.87, respectively. All models had spatial correlation coefficients >0.96 and a standard deviation of 8.36–13.45; GISS-E2-H and GISS-E2-R had the lowest standard deviations. The standard deviations of BCC-CSM-1(m) and MRI-CGCM3 are closer to the observations.

4.2. Spatial Characteristics of the Annual Mean Surface Air Temperature

The spatial distribution characteristics of the climate tendency rates across CA are shown in Figure 5. According to the data recorded by the CRU (Figure 5a), the climate tendency rates increased by 0.05 °C–0.17 °C per decade and varied by region. The greatest climate tendency rates were observed in the northern and western parts of Kazakhstan and were ≧0.16 °C per decade, whereas the lowest climate tendency rate of ≦0.08 °C per decade was observed in southern Tajikistan.

The MME presented a similar but slightly lesser warming trend (Figure 5b), ranging from 0.05 °C to 0.09 °C per decade. Higher increases were observed in the north than in the south, and the high-value center was located in northern Kazakhstan.

The difference between the MME-predicted and observed data (i.e., Figure 5a,b) is shown in Figure 5c; these differences, which were mostly negative, ranged from –0.10 °C to 0.02 °C per decade. The greatest differences were observed for central and western parts of Uzbekistan and northern and eastern parts of Kazakhstan, where the predicted climate tendency rates were lower than the observed rates. Positive differences (i.e., model overpredictions) were concentrated in southern Tajikistan, with a maximum value of 0.02 °C per decade.

4.3. EOF Analysis of the Surface Air Temperature in Central Asia

To further evaluate the performance of the CMIP5 climate models, EOF analysis was performed on the observed data and MME. In the first several modes of the observed data, the variance contribution rate was 99.46%, and only the first mode passed the NORTH test. Similarly, only the first mode of the MME passed the NORTH test, and the variance contribution rate was 99.95%. The spatial first mode distribution of the observed data is shown in Figure 6a; most of the studied area demonstrated positive values. The high-value center was located in the southern region of Turkmenistan and the low-value center was located in the eastern regions of Tajikistan and Kyrgyzstan. Similar results were evident from the first mode of the MME, shown in Figure 6b. The spatial distribution of the MME was similar to that of the observed data. Furthermore, the SAT showed a consistent overall trend over the past 100 years.

The time coefficients, which represent the variations in the spatial mode over time, corresponding to the first mode of the observed data and the MME, are shown in Figure 6c,d, respectively. Both time coefficients of the observed data and MME showed significant upward trends and reached the significant level of 0.01. This indicates that MME can simulate the same uptrend seen in the observed data.

Overall, EOF analysis indicated that MME can reasonably reproduce the spatial distribution and time variation characteristics of the SAT in the studied region. Therefore, MME is feasible for future projection of the spatial and temporal SAT variations.

5. Discussion

The MME ensemble of the 24 evaluated CMIP5 climate models simulated the spatial and temporal variations in the annual mean SAT in the studied region of CA well. Thus, it is feasible to use the MME to estimate the spatial and temporal distribution characteristics and the evolution of the SAT in future studies. In this work, only a simple average method was used to calculate the MME; the MME could thus be further improved using filtering patterns according to the statistical values of each climate model and assigning different weight coefficients.

Owing to their different calculation processes, different climate models have different horizontal resolutions, making it difficult to describe the atmospheric physical and chemical processes well and resulting in large simulation deviations [36]. Selecting the appropriate horizontal resolution of a climate model to ensure that it can be used to solve the scientific problem concerned has thus become a focus of climate simulation research using climate models. Future research should be done to consider the effect of different horizontal resolutions of the climate model on simulated SAT trends.

CA is located in the hinterland of the Eurasian continent and presents a complex terrain and large differences between the eastern and western regions. Different climate models use different topographic data, which leads to SAT errors in the interpolation process. After interpolation, the deviation in the SAT values is proportional to the altitude and terrain complexity. Employing the terrain effect correction can significantly reduce this deviation, particularly in mountainous and desert areas such as the Tianshan and Pamir mountains in the east and the large desert in the mid-west region [37]. Future work should therefore aim to correct the SAT data based on actual topographic data.

6. Conclusions

This paper assesses the capability of 24 CMIP5 climate models to simulate observed SAT data in CA from 1901 to 2005. Most of the studied climate models accurately reproduced the space and time patterns of the annual mean SAT, although some variations among the climate models were present. Three main conclusions were drawn:

(1) Although most of the 24 studied CMIP5 climate models captured the spatial temperature distribution in the studied region of CA, they tended to slightly overestimate the annual mean SAT. Similar results have been found when applied in northern Eurasia [22], the Arctic [38], the northern hemisphere [39], and globally [40]. This may be because some climate models overestimate the response to the increasing greenhouse gas concentrations [41]. Future efforts should thus focus on improving the accuracy of these models.

(2) Taylor diagram analysis demonstrated that the RMSE of all models was less than 3, the standard deviation was between 8.36 and 13.45, and the spatial correlation coefficient was greater than 0.96. The 24 studied climate models can well simulate the warming trend in CA as they replicated the observed climate tendency rates at between 0.05 °C and 0.09 °C per decade. However, the overall simulated climate tendency underestimated the observed value in most regions.

(3) EOF analysis indicated that the MME accurately simulates the spatial and temporal variation in SAT increases in CA as well as the north–south anti-phase fluctuations. Therefore, the MME can be used to estimate the temporal and spatial distribution characteristics of the SAT in CA.

In general, the 24 studied CMIP5 models exhibited different capabilities in simulating the SAT variations in CA from 1901 to 2005. Uncertainty is introduced in SAT simulations due to the varying horizontal resolutions used in different climate models. As climate models are an important tool for studying future climate change effects, the accuracy of a climate model must be demonstrated before it can be employed to estimate future climate change scenarios. Accurate climate models can provide a reference for climate change mitigation as well as disaster reduction and prevention in CA.

Author Contributions

Conceptualization, Y.X. and Z.T.; Formal analysis, Z.T. and M.G.; Investigation, Y.Y.; Methodology, M.Y.; Resources, R.Y.; Validation, X.C. and M.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Strategic Priority Research Program of the Chinese Academy of Sciences, the Pan-Third Pole Environment Study for a Green Silk Road, grant number XDA20060303.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Results for individual Coupled Model Intercomparison Project Phase 5 (CMIP5) models (Taylor et al., 2012) are available for download from the database at https://esgf-node.llnl.gov/search/cmip5/ (accessed on 11 July 2020). The CRU temperature dataset was obtained from the Climatic Research Unit at the University of East Anglia (http://www.cru.uea.ac.uk/data, accessed on 11 July 2020).

Acknowledgments

We wish to thank the four anonymous reviewers, whose valuable comments and suggestions helped us to improve our manuscript. We would like to acknowledge the Climate Research Unit (CRU) for the free climate data. We thank the help of World Climate Research Programme’s Working Group on Coupled Modelling, which is responsible for CMIP.

Conflicts of Interest

The authors declare that they have no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Table

Generate image description

Figure 1. Elevation map of the studied area in central Asia (CA) (Generated by ArcGIS 10.2, URL: http://www.esri.com/software/arcgis/arcgis-for-desktop (accessed on 17 June 2015), the elevation data were obtained from the SRTM 90 m Digital Elevation Database v4.1, URL: https://cgiarcsi.community/data/srtm-90m-digital-elevation-database-v4-1/ (accessed on 20 August 2020).

Generate image description

Figure 2. Spatial distributions of the annual mean surface air temperature (SAT), 1901‒2005, where (a–x) represent the 24 studied CMIP5 climate models as detailed in Table 1 and (y) represents the observational data by the Climate Research Unit (CRU).

Generate image description

Figure 3. Annual mean SAT anomaly of the observed (CRU) data and 24 CMIP5 climate models, 1901–2005.

Generate image description

Figure 4. Taylor diagram comparing the observed data with the annual mean SAT as simulated by the 24 studied climate models for 1901–2005. Each red dot represents a climate model, excluding observation.

Generate image description

Figure 5. Spatial distributions of the annual mean SAT trends of (a) the observed data (by the CRU), (b) the multi-model ensemble (MME), and (c) the MME minus the CRU data, 1901–2005.

Generate image description

Figure 6. Spatial distributions of the first mode of the annual mean SAT of (a) the observed data and (b) the MME, with first time coefficients for (c) the observed data and (d) the MME.

Table 1

Description of the Coupled Model Intercomparison Project Phase 5 (CMIP5) climate models used.

Model	Modeling Center	Horizontal Resolution(Latitude × Longitude)
(a) BCC-CSM1.1	BCC, China	2.8125° × 2.8125°
(b) BCC-CSM1.1(m)	BCC, China	1.125° × 1.125°
(c) BNU-ESM	GCESS, China	2.8125° × 2.8125°
(d) CanESM2	CCCMA, Canada	2.8125° × 2.8125°
(e) CCSM4	NCAR, USA	1.25° × 1°
(f) CNRM-CM5	CNRM-CERFACS, France	~1.4° × 1.4°
(g) CSIRO-Mk3-6-0	CSIRO-QCCCE, Australia	1.875° × 1.875°
(h) FGOALS-g2	LASG-CESS, China	3° × 2.8125°
(i) FIO-ESM	FIO, China	~5.625° × 1.4°
(j) GFDL-CM3	NOAA GFDL, USA	4° × 1.25°
(k) GFDL-ESM2G	NOAA GFDL, USA	4° × 1.25°
(l) GISS-E2-H	NASA GISS, USA	2.5° × 2.5°
(m) GISS-E2-R	NASA GISS, USA	2.5° × 2.5°
(n) HadGEM2-ES	MOHC, UK	~2.5° × 0.9°
(o) IPSL-CM5A-LR	IPSL, France	3.75° × 1.875°
(p) IPSL-CM5A-MR	IPSL, France	2.5° × 1.25°
(q) MIROC5	MIROC, Japan	~1.4° × 1.4°
(r) MIROC-ESM-CHEM	MIROC, Japan	2.8125° × 2.8125°
(s) MIROC-ESM	MIROC, Japan	2.8125° × 2.8125°
(t) MPI-ESM-LR	MPI-M, Germany	1.875° × 1.875°
(u) MPI-ESM-MR	MPI-M, Germany	1.875° × 1.875°
(v) MRI-CGCM3	MRI, Japan	1.125° × 1.125°
(w) NorESM1-M	NCC, Norway	2.5° × 1.875°
(x) NorESM1-ME	NCC, Norway	2.5° × 1.875°