Evaluation of Temperature and Precipitation Simulations in CMIP6 Models Over the Tibetan Plateau

Abstract

This study presents a comprehensive evaluation of the mean and extreme surface air temperature and precipitation in the Coupled Model Intercomparison Project Phase 6 (CMIP6) multimodel ensemble simulations over the Tibetan Plateau. Simulations from 29 climate models are compared against gridded observations for the period 1961–2012. Results show that models reasonably reproduce the overall spatial patterns of the 1961–2012 averages of seasonal mean temperature and precipitation. Nevertheless, models tend to underestimate mean temperature and overestimate precipitation accumulations. To be specific, the multimodel mean estimates of seasonal average temperatures over the plateau are 0.5°C–2.6°C colder than observed, while the corresponding estimates of seasonal precipitation accumulations are 218% (spring: March–May), 76% (summer: June–August), 129% (autumn: September–November), and 533% (winter: December–February) of those observed. As in observations, models also well capture increasing trend of mean temperature and precipitation in all seasons but underestimate the rates of increasing trends for both variables in all seasons. Models’ ability to simulate temperature and precipitation extremes is also evaluated in terms of a set of chosen extreme indices defined by the Expert Team on Climate Change Detection and Indices. On average, models tend to underestimate the averages of annual maximum daily maximum temperature, annual minimum daily minimum temperature, frost days, warm nights, and consecutive dry days but overestimate the corresponding averages of annual maximum 5‐day precipitation accumulations and simply daily intensity. Generally, models are able to simulate the signs of the trends in extreme indices but underestimate their magnitudes and misrepresent spatial patterns.

Introduction

The Intergovernmental Panel on Climate Change (IPCC) reported a warming of about 1.0°C in global mean surface temperature during the past 100 years (e.g., IPCC SR15, Masson-Delmotte et al., 2018). Along with this global warming, temperature and precipitation extremes have also changed around the world (e.g., Fischer et al., 2014; Ji & Kang, 2015; Kharin et al., 2018; Lorenz et al., 2019; Piras et al., 2016), causing profound effects on ecological, economic, and societal processes (e.g., Allison et al., 2009; Easterling et al., 2000; Wheeler & Von Braun, 2013; Ye et al., 2018; Z. Zhang et al., 2020). To mitigate the adverse impacts of climate change, reliable future climate change information is urgently needed.

The most common approach to investigating future climate change is to use general circulation models (GCMs), which can simulate the historical climate and project the future climate of the Earth under a suite of different possible emission scenarios (Cai et al., 2018; Gusain et al., 2020). Climate models have evolved over the past decades with increasingly better representation of the complex climate system. Nevertheless, state-of-the-art climate models are not perfect yet, largely because our understanding of the complex climate system remains rather limited (Reichler & Kim, 2008; Sun et al., 2015). For example, despite that models participating in the Coupled Model Intercomparison Project (CMIP) Phase 5 have incorporated new components such as dynamic vegetation, terrestrial, and marine carbon cycles and indirect effects of aerosols compared to the prior generation models in CMIP3 (Taylor et al., 2012), they still have remarkable biases in representing impact-relevant local to regional climate conditions (e.g., Berg & Sheffield, 2019; Gautam & Mascaro, 2018; Sharma et al., 2018).

Recently, the new sixth phase CMIP simulations from more than 30 climate models developed by different institutions around the world have become available. The new phase CMIP has incorporated further improved climate modules and thus is expected to yield improved representation of the Earth's climate system (Eyring et al., 2016). Evaluation of these new simulations with respect to the observed mean and extreme climates is drawing extensive research efforts (e.g., Catalano et al., 2020; Chen et al., 2020; D. Jiang et al., 2020; Rivera & Arnould, 2020; Xin et al., 2020), as it can inform the extent to which these models can be trusted for projecting future climate change and how they can be further improved. These studies demonstrated that the CMIP6 GCMs have an improvement in simulating the temporal and spatial patterns of the climate variables comparing to the previous phase (i.e., CMIP5).

The Tibetan Plateau (TP) is the highest and largest plateau in the world, with an area of more than 2.5 million km² and an average elevation of above 4,000 m (Frauenfeld et al., 2005). It has been found to have profound impacts on regional and even global climates through its thermal forcing mechanisms (Ma et al., 2018; Rao et al., 2019). It is also the source of major Asian rivers (e.g., Yellow, Yangtze, Mekong, Salween, Brahmaputra, Ganges, and Indus), which are supporting more than 1.4 billion people living downstream (Immerzeel et al., 2010). In recent decades, TP has experienced higher warming rate than the global mean (e.g., X. Liu & Chen, 2000; Rangwala & Miller, 2012; Thompson et al., 2000; B. Wang et al., 2008; Yao et al., 2019), which has induced discernable changes such as glacier shrinking and thinning, permafrost thawing, and shortening soil frozen period (e.g., Kang et al., 2010; Shen et al., 2015; K. Yang et al., 2011; Yao et al., 2012). Reliably simulating the climate and its change over the TP is of importance for informing necessary adaptation and mitigation strategies for these changes. Previous studies have evaluated the ability of climate models in CMIP5 to reproduce different near surface climate variables over the TP (e.g., Q. Hu et al., 2014; Salunke et al., 2019; Su et al., 2013; Xu et al., 2017; You et al., 2016). Overall, these results indicate that the CMIP5 simulations reasonably capture the climatological spatial patterns of annual and seasonal mean temperatures and precipitation over the TP but with cold bias in temperature and wet bias in precipitation; they correctly reproduce the increasing trends of mean temperature and precipitation, while they fail to simulate the spatial distribution of the trends in mean temperature and precipitation. It is natural to ask how well the new-generation CMIP6 models perform in simulating not only the mean climate but also extreme events over the TP before using them to project future TP climate. However, there are limited studies investigating this question (e.g., Lun et al., 2021; Zhu & Yang, 2020), in particular considering temperature and precipitation extremes.

The purpose of this study is, therefore, to assess the performance of the latest generation CMIP6 climate models in simulating seasonal mean and extreme temperature as well as precipitation over the TP with respect to a gridded observational data set for the period 1961–2012. The remainder of the paper is organized as follows. Section 2 introduces the data and methods used in this study. Evaluations of selected models are presented in Section 3. The study is concluded with a summary in Section 4.

Data and Methods Observations and Model Simulations

In this study, the TP boundary is defined as above the 2,500 m contour line using the National Aeronautics and Space Administration Shuttle Radar Topographic Mission 90m DEM (G. Zhang et al., 2014). Because of the limitations of temporal discontinuity and spatial representation in ground-station observations, gridded observations are generally preferred for model evaluation (Alexander et al., 2006; Kiktev et al., 2003). We, therefore, chosen a gridded CN05.1 observational data set of daily near surface temperature and precipitation on a horizontal resolution of 0.5° × 0.5°developed by Wu and Gao (2013) as the reference for model evaluation. This data set is created by interpolating observations from 2,416 stations in China. CN05.1 has been widely used in assessing the performance of regional and global climate model simulations (Li et al., 2018; X. Wang et al., 2016; Y. Yang et al., 2019; You et al., 2015). Over the TP, 283 stations are involved in the interpolation (72°E–105°E, 25°N–41°N) (X. Wang et al., 2017). There are in total 1,033 grid cells within the TP (Figure 1). Daily mean, maximum, and minimum surface air temperature and daily precipitation for the period of 1961–2012 are used.

Simulations of daily near surface air temperature and precipitation for the period 1961–2012 from 29 CMIP6 models were acquired from the CMIP6 data archive available online at https://esgf-node.llnl.gov/search/cmip6/. More detailed information about each model is provided in Table 1. We define the attributes Experiment ID (historical), Variable (pr, tas, tasmax, tasmin), and Frequency (day) to select the models. Note that for each model, we used all available ensemble members (Table S1). For example, the three simulation variants (e.g., r1i1p1f1, r2i1p1f1, r3i1p1f1) of daily precipitation and temperature outputs of BCC-CSM2-MR are all used.

Table 1 The Information of the 29 CMIP6 Global Climate Models Evaluated in This Study

Number	Model name	Modeling center	Resolution (longitude × latitude)
1	BCC-CSM2-MR	Beijing Climate Center and China Meteorological Administration, China	1.125° × 1.125°
2	BCC-ESM1	Beijing Normal University, China	2.8125° × 2.8125°
3	CanESM5	Canadian Centre for Climate Modeling and Analysis, Canada	2.8125° × 2.8125°
4	CNRM-CM6-1	National Centre for Meteorological Research and European Centre for Research and Advanced Training in Scientific Computation, France	1.40625° × 1.40625°
5	CNRM-CM6-1-HR	National Centre for Meteorological Research and European Centre for Research and Advanced Training in Scientific Computation, France	0.5° × 0.5°
6	CNRM-ESM2-1	National Centre for Meteorological Research and European Centre for Research and Advanced Training in Scientific Computation, France	1.40625° × 1.40625°
7	EC-Earth3-Veg	EC-Earth consortium, Europe	0.703125° × 0.703125°
8	FGOALS-f3-L	LASG, Institute of Atmospheric Physics, Chinese Academy of Sciences, China	1.25° × 1°
9	FGOALS-g3	LASG, Institute of Atmospheric Physics, Chinese Academy of Sciences, China	2° × 2.25°
10	GFDL-CM4	National Oceanic and Atmospheric Administration Geophysical Fluid Dynamics Laboratory, United States	1.25° × 1°
11	GFDL-ESM4	National Oceanic and Atmospheric Administration Geophysical Fluid Dynamics Laboratory, United States	1.25° × 1°
12	HadGEM3-GC31-LL	Met Office Hadley Centre, UK	1.875° × 1.25°
13	HadGEM3-GC31-MM	Met Office Hadley Centre, UK	0.83333° × 0.55556°
14	INM-CM4-8	Institute for Numerical Mathematics, Russia	2° × 1.5°
15	INM-CM5-0	Institute for Numerical Mathematics, Russia	2° × 1.5°
16	IPSL-CM6A-LR	Institute Pierre Simon Laplace, France	2.5° × 1.25874°
17	KACE-1-0-G	National Institute of Meteorological Sciences/Korea Meteorological Administration, Korea	1.875° × 1.25°
18	MIROC-ES2L	University of Tokyo, National Institute for Environmental Studies and Japan Agency for Marine-Earth Science and Technology, Japan	2.8125° × 2.8125°
19	MIROC6	University of Tokyo, National Institute for Environmental Studies and Japan Agency for Marine-Earth Science and Technology, Japan	1.40625° × 1.40625°
20	MPI-ESM-1-2-HAM	Max Planck Institute for Meteorology, Germany	1.875° × 1.875°
21	MPI-ESM1-2-HR	Max Planck Institute for Meteorology, Germany	0.9375° × 0.9375°
22	MPI-ESM1-2-LR	Max Planck Institute for Meteorology, Germany	1.875° × 1.875°
23	MRI-ESM2-0	Meteorological Research Institute, Japan	1.125° × 1.125°
24	NESM3	Nanjing University of Information Science and Technology, China	1.875° × 1.875°
25	NorCPM1	University of Bergen, Norway	2.5° × 1.875°
26	NorESM2-LM	University of Bergen, Norway	2.5° × 1.875°
27	NorESM2-MM	Norwegian Climate Centre, Norway	1.25° × 0.9375°
28	SAM0-UNICON	Seoul National University, Korea	1.25° × 0.9375°
29	UKESM1-0-LL	Met Office Hadley Centre, UK	1.875° × 1.25°

Note. Experiment ID: historical; Variable: pr, tas, tasmax, tasmin; Frequency: day.

Temperature and Precipitation Extreme Indices

Four temperature extreme indices and three precipitation extreme indices defined by the Expert Team on Climate Change Detection and Indices (ETCCDI; Frich et al., 2002) were adopted in this study. They are annual maximum of daily maximum temperature (TXx), annual minimum of daily minimum temperature (TNn), frost days (FD), warm nights (TN90p), annual maximum of consecutive 5-day precipitation accumulation (RX5day), consecutive dry-day index (CDD), and the simple daily intensity (SDII). Table 2 lists the detail definitions of these indices. They were selected for several reasons. First, they are closely related to hydrologic and agricultural extreme events (Sillmann et al., 2013; Terando et al., 2012). For example, FD can be relevant for agricultural and engineering practices; RX5day is often used to describe changes in potential flood risks, and CDD is often referred to as a drought indicator. Second, they are highly sensitive to global warming and have experienced noticeable changes over the TP (Z. Jiang et al., 2015; C. Liu et al., 2019; Song et al., 2014; Yin et al., 2019; You et al., 2018). For instance, significance increases in TNn, TNx, and TN90p and decreases in FD have been observed over the TP during the past decades. Third, these indices together reflect both intensity and duration of extreme temperature and precipitation events (Z. Jiang et al., 2012; Wu et al., 2020).

Table 2 Definitions of the Extreme Climate Indices Used in This Study

Index	Definition	Units
TXx	The annual maxima of daily maximum	°C
TNn	The annual minima of daily minimum	°C
FD	Number of days with minimum temperature <0°C	Days
TN90p	Annual percentage of days when TN > 90th percentile	%
RX5day	Annual maximum of 5 days precipitation accumulations	mm
CDD	Maximum number of consecutive dry (RR < 1) mm days	Days
SDII	The daily precipitation amount averaged over all wet days (defined as PR > 1.0 mm) in a year	mm/day

Data Processing

The CMIP6 models possess different horizontal resolutions. To facilitate comparison and evaluation, we regridded all simulations of daily temperature and precipitation to a common grid of 0.5° × 0.5° as in the observed data set using bilinear interpolation. These regridded daily simulations were then used to compute their seasonal means and extremes of daily temperature and precipitation. To compute multimodel ensemble statistics, such as the multimodel ensemble mean bias in an extreme index, biases in individual ensemble members of a given climate model are averaged first before taking the average among models. Doing so can avoid overweighting models with more ensemble members.

Model Performance Metrics

For assessing model performance in reproducing the overall spatial patterns of the long-term means and trends in an interested variable, we use the Taylor diagram (Taylor, 2001), which can provide an efficient summary of comparison between observations and simulations in terms of their spatial correlation, root-mean square error (RMSE), and the ratio of their spatial variations (measured by spatial standard deviations). The correlation coefficient and RMSE describe the degree of phase and amplitude agreement of the two fields, and the ratio of variances measures the degree of uniformity of the two fields. Better simulations have relatively higher correlations, low RMSEs, and closer spatial variations. For the ease of comparison across models in one Taylor diagram, model standard deviations are normalized by the observed standard deviation.

The performance in simulating long-term trends of climate variables is also a very important factor measuring the capability of models. The Sen's slope estimator was employed to estimate the trends in the studied means and extremes of daily precipitation and temperature, and the Mann–Kendall test with a significance level of 5% was used to evaluate whether the estimated trends are statistically significant. Those nonparametric methods have the advantage of not assuming any special form for the data distribution function, while having a power nearly as high as their parametric competitors. More details of the Sen's slope estimator and Mann–Kendall test methods, the reader is referred to Sen (1968) and Kendall (1975). Those methods are highly recommended by the World Meteorological Organization and have been widely applied to assess trend of hydrometeorological time series (Agbo et al., 2021; Da Silva et al., 2015; Nyikadzino et al., 2020; You et al., 2016).

Results Performance in Simulating Seasonal Mean Temperature and Precipitation Seasonal Mean Temperatures

Model's ability to simulating the 1961–2012 mean values of seasonal temperature over the TP was first evaluated. We focus on the annual and summer (June–August) mean temperatures in the main text and present results for other seasons in supplemental online materials. On average, the long-term means of annual and summer average temperatures over the plateau are about −1.7°C and 7.8°C during 1961–2012 based on observational data set, respectively. Figure 2 presents the regional mean biases in annual and summer average temperatures in individual CMIP6 models and multimodel ensemble mean relative to the corresponding observations. Overall, the CMIP6 models underestimate these long-term mean values, with a cold bias of −1.4°C for annual mean temperature and −0.5°C for summer mean temperature. For annual average temperature, 18 out of 29 models show cold biases ranging from −9.7°C to −0.3°C, while the remaining 11 models show warm biases up to 4.1°C. Biases in summer average temperature vary from −7.3°C to 4.2°C across the 29 models. It is interesting to note that models with good performance in simulating annual average temperature do not necessarily exhibit similarly good performance in summer. For example, the NorESM2-MM model performs best in simulating annual temperature but exhibits only moderate performance in simulating summer one.

Cold biases are also seen in other seasons, with average biases of −2.1°C for spring, −0.7°C for autumn, and −2.6°C for winter (Figure S1). Although temperature biases vary substantially across models and seasons, models generally tend to have smallest biases in summer and largest biases in winter. Among the 29 evaluated models, MPI-ESM-1-2-HAM, UKESM1-0-LL, MPI-ESM1-2-HR, KACE-1-0-G, and HadGEM3-GC31-LL are found to perform better than the other models for seasonal mean temperatures in terms of bias.

The spatial distributions of the observed and multimodel ensemble mean annual and summer average temperatures are shown in Figure 3. The observations show that the lowest annual mean temperatures occur in the central and the northwest part of the plateau, while the highest annual mean temperatures are found in the southeastern region. Not surprisingly, the general pattern exhibits a decreasing gradient from the southeast to the northwest. The spatial pattern of temperature in each season roughly resembles that of annual means. The CMIP6 models generally depict the same spatial patterns, with low temperatures in the central and northwest and high temperatures in the southeast. However, the models tend to underestimate the temperatures over the TP except for the south boundary when compared with the observations. The cold areas on the northwestern simulated by models are larger than that observed. In addition, there is a large variation in the spatial distribution of biases among models and across seasons (Figures S2–S4). Figure 4 shows the Taylor diagrams of the model simulations against observations in annual and summer average temperatures. It can be seen that the majority of models have spatial correlations between 0.50 and 0.84, indicating that the models have a reasonably well performance in simulating the overall spatial patterns of long-term averages of annual and summer temperatures during the study period. The ratios of variances mainly range from 1 to 2, indicating the simulated spatial variation is larger than observation. The normalized RMSEs are in the range of 0.75–1.50, indicating the relatively low amplitude of temperature biases. Five models, including GFDL-CM4, GFDL-ESM4, HadGEM3-GC31-MM, INM-CM4-8, and INM-CM5-0, are found to have relatively higher spatial correlations and lower RMSEs, while FGOALS-g3, CanESM5, and MIROC-ES2L show poor performance in these regards. Compared with the single-model result, the multimodel ensemble mean generally performs better in simulating the spatial distributions of annual and summer temperatures.

The trends of annual and summer average temperatures estimated by the Sen's slope are summarized in Table 3. Significance of these estimated trends is evaluated in term of the Mann–Kendal method. It is clear that all climate models can appropriately simulate the warming tendency of temperatures for all seasons but tend to underestimate the rates. For example, only three CMIP6 climate models simulate warming trends that are greater than 0.3°C/decade in annual average temperature, which are CanESM5 (0.83°C/decade), NorESM2-LM (0.31°C/decade), and NorESM2-MM (0.31°C/decade). The top five models with the smallest trend bias are NorESM2-LM, NorESM2-MM, CNRM-CM6-1-HR, HadGEM3-GC31-MM, and KACE-1-0-G. The Taylors diagram shows that most models have difficulty in simulating the spatial pattern of the trends (Figure 4). The spatial correlations in trends between models and observations are generally less than 0.6, the ratio of the variances mainly ranges from 0.5 to 1, and the normalized RMSEs of most models fall in the range of 0.7–1.4. Compared to other models, CNRM-ESM2-1, INM-CM5-0, HadGEM3-GC31-LL, HadGEM3-GC31-MM, and BCC-CSM2-MR perform relatively better in simulating the trends in annual and seasonal mean temperatures. On the contrary, FGOALS-g3, GFDL-ESM4, and NorESM2-MM exhibit the worst performance among others.

Table 3 The Trends of Annual and Summer Temperature and Precipitation From Observations (OBS) and CMIP6 GCMs Over the TP During 1961–2012

Models	Temperature (°C/decade)		Precipitation (mm/decade)
	Annual	Summer	Annual	Summer
OBS	0.33*	0.25*	7.70*	2.44
BCC-CSM2-MR	0.25*	0.26*	0.48	0.35
BCC-ESM1	0.21*	0.11*	1.25	2.73
CanESM5	0.83*	1.08*	11.92*	6.29*
CNRM-CM6-1	0.20*	0.16*	1.81*	1.15
CNRM-CM6-1-HR	0.29*	0.23*	2.85	0.89
CNRM-ESM2-1	0.20*	0.15*	8.48*	4.10*
EC-Earth3-Veg	0.26*	0.24*	9.55*	4.86
FGOALS-f3-L	0.22*	0.28*	8.56	1.74
FGOALS-g3	0.18*	0.18*	5.30	4.04
GFDL-CM4	0.24*	0.31*	5.69	2.73
GFDL-ESM4	0.09	0.14*	−6.74	−4.49
HadGEM3-GC31-LL	0.26*	0.27*	0.43	1.19
HadGEM3-GC31-MM	0.29*	0.27*	1.56	1.00
INM-CM4-8	0.20*	0.13*	−0.74	0.49
INM-CM5-0	0.11*	0.11*	2.26	2.09
IPSL-CM6A-LR	0.26*	0.25*	1.34	−0.18
KACE-1-0-G	0.29*	0.29*	8.37	3.36
MIROC6	0.10*	0.05	1.07	1.70
MIROC-ES2L	0.16*	0.13*	5.75	1.09
MPI-ESM-1-2-HAM	0.22*	0.15*	1.20	1.17
MPI-ESM1-2-HR	0.26*	0.22*	−2.34	−0.11
MPI-ESM1-2-LR	0.24*	0.20*	−1.32	1.12
MRI-ESM2-0	0.26*	0.25*	6.61*	4.06
NESM3	0.21*	0.20*	0.06	0.08
NorCPM1	0.23*	0.19*	5.02	1.16
NorESM2-LM	0.31*	0.22*	7.13	2.39
NorESM2-MM	0.31*	0.23*	6.57	0.62
SAM0-UNICON	0.26*	0.09	−1.05	−5.08
UKESM1-0-LL	0.21*	0.22*	2.33	1.48
MME	0.25*	0.23*	3.22	1.45

Note. Trend with a significance level greater than 95% is highlighted with the asterisk (*). The ensemble of the GCMs simulations is abbreviated to MME.

Seasonal Precipitation Accumulations

The long-term mean of annual precipitation accumulations over the TP is around 421 mm during 1961–2012 based on observational data set. Also, about 60% of annual precipitation (254 mm) occurs during summer (July–August), while precipitation in winter is very low. For CMIP6 simulations (shown in Figures 5 and S5), the multimodel mean estimates of seasonal precipitation accumulations over the TP are 125% (annual), 218% (spring), 76% (summer), 129% (autumn), and 533% (winter) of the corresponding observations. Models show wet biases ranging from 55.8% to 209.7% for the annual average precipitation and from 8.1% to 169.5% for summer average precipitation (Figure 5). Models with the least relative biases in annual and seasonal precipitation accumulations are HadGEM3-GC31-MM, CNRM-CM6-1-HR, NorESM2-MM, FGOALS-f3-L, and IPSL-CM6A-LR.

Figure 6 contrasts the spatial distributions of multimodel mean estimates of annual and seasonal precipitation accumulations with the corresponding observations. Precipitation decreases gradually from the southeast to the northwest in observations for both annual and summer precipitation accumulations. Generally, models can roughly reproduce these spatial patterns. However, compared with the observations, models tend to overestimate these precipitation accumulations over almost the TP, with the most pronounced overestimation occurring in the southeastern corner of the TP. In addition, models fail to reproduce the spatial patterns of spring and winter precipitation accumulations (Figures S6 and S7). The Taylor diagrams show that the spatial correlations for annual precipitation vary from 0.38 to 0.86, suggesting that the models can reproduce the spatial patterns of annual precipitation in the TP (Figure 7). It is noted that autumn precipitation has higher correlations than annual and other seasons (Figure S8). For the majority of models, the ratio of spatial standard deviation of modeled precipitation to that of observed ranges 1.5 and 2.7, indicating the simulated precipitation is more inhomogeneous in space than the observations. The five leading models are MRI-ESM2-0, NorESM2-MM, NorESM2-LM, HadGEM3-GC31-MM, and MIROC, while FGOALS-f3-L, FGOALS-g3, and MPI-ESM-1-2-HAM show relatively poor performance. The Taylor diagrams also show that multimodel ensemble mean simulations perform relatively better than individual models.

Table 3 shows that most models fail to simulate the wetting trend of precipitation, with only five exceptional models (CanESM5, EC-Earth3-Veg, CNRM-ESM2-1, MRI-ESM2-0, and CNRM-CM6-1) that show a significantly increasing trend. For summer precipitation, most models correctly simulate the increasing trend that is not significant, except for GFDL-ESM4, IPSL-CM6A-LR, MPI-ESM1-2-HR, and SAM0-UNICON. Overall, as shown in Figure 7, most models have difficulty in simulating the spatial patterns of trends in precipitation. The spatial correlations between modeled and observed annual precipitation accumulations are either negative or slightly positive (less than 0.3). A relatively higher spatial correlation is found in summer, indicating a relatively better model performance in simulating the spatial patterns of summer precipitation trends over the TP. The models CNRM-ESM2-1, INM-CM5-0, IPSL-CM6A-LR, MIROC-ES2L, and MPI-ESM1-2-LR appear to produce better results than other models. Again, the result shows that the multimodel ensemble mean is desirable for providing a more reliable estimation of precipitation trends than individual models.

Model Performance in Simulating Annual Extreme Temperature and Precipitation Temperature Extremes

Model biases of TXx, TNn, FD, and TN90p are presented in Figure 8. Similar to the mean temperature, most models present a cold bias for TXx and TN90p, while only half of models underestimated the value of TNn and FD. The biases vary from −11.1°C to 4°C for TXx, from −16.9°C to 13.7°C for TNn, from −82.4 to 65.6 days for FD, and from −5.6% to 13.4% for TN90p. The models SAM0-UNICON, NorESM2-LM, MIROC-ES2L, INM-CM4-8, and NorESM2-MM show relatively better performance in warm events (TXx), while KACE-1-0-G, MRI-ESM2-0, HadGEM3-GC31-LL, NorCPM1, and UKESM1-0-LL perform relatively better in cold events.

Warm extremes as described by TXx and cold extremes as described by TNn are characterized by low values in the northwest and high values in the southeast and northeast (Figure 9). Not surprisingly, cold extremes as described by FD display an opposite pattern. In general, the spatial patterns of TXx, TNn, and FD in models are roughly similar to those in observations. However, there are still obvious discrepancies, particularly over high elevation terrains. In these regions, the CMIP6 multimodel ensemble mean simulations show higher TXx and lower TNn values than the observations. Most models fail to capture the observed spatial pattern of TN90p (Figure S9). The Taylor diagrams show that there is reasonable correspondence between the CMIP6 simulations of TXx, TNn, and FD (Figure 10). The majority of models have spatial pattern correlations of 0.4–0.8; the ratios of model to observation standard deviations range from 1.0 to 1.9. GFDL-CM4, HadGEM3-GC31-MM, INM-CM4-8, INM-CM5-0, and NESM3 are found to show relatively better performance among all models.

Observations show that TXx, TNn, and TN90p have significantly increased during 1961–2012, and FD has significantly decreased (Table 4). Most models capture well the observed trends in these temperature extreme indices. Similar to seasonal mean temperatures, the trends in temperature extremes are also underestimated in the new-generation CMIP6 models. For example, the observed trend in TNn is 0.60°C/decade, which is substantially larger than the trend of 0.31°C/decade in the multimodel ensemble mean TNn series. The best model is UKESM1-0-LL for TXx and TN90p and NorESM2-MM for TNn and FD, respectively. As can be seen in Figure 10, the spatial correlations of trends in the temperature extremes between CMIP6 models and observations are relatively weak in most cases, ranging from −0.42 to 0.59. This indicates that the most models can hardly capture the spatial patterns of trends in temperature extremes over the TP. INM-CM4-8, MPI-ESM1-2-HR, MPI-ESM1-2-LR, NESM3, and NorESM2-LM are the five best models for warm extremes. MPI-ESM1-2-HR, INM-CM5-0, NESM3, NorESM2-LM, and NorESM2-MM are more skillful for clod extremes.

Table 4 The Trends of Extreme Climate Indices From Observations (OBS) and CMIP6 GCMs Over the TP During 1961–2012

Models	TXx (°C/decade)	TNn (°C/decade)	FD (days/decade)	TN90p (%/decade)	RX5day (mm/decade)	SDII (mm·day−1/decade)	CDD (days/decade)
OBS	0.21*	0.60*	−5.74*	7.36*	0.20	0.03	−2.35
BCC-CSM2-MR	0.29*	0.44*	−2.84*	5.29*	0.12	0.01	−0.11
BCC-ESM1	0.15	0.36*	−2.39*	3.31*	0.24	0.01	−0.11
CanESM5	0.73*	0.70*	−4.14*	18.84*	0.64*	0.05*	−1.25
CNRM-CM6-1	0.13*	0.21*	−1.42*	5.26*	0.08	0.01	−0.67
CNRM-CM6-1-HR	0.22	0.34	−1.03	2.98*	0.32	0.02	−0.40
CNRM-ESM2-1	0.13*	0.27*	−1.41*	4.28*	0.43	0.03	−0.63
EC-Earth3-Veg	0.22*	0.30	−3.36*	6.63*	0.51	0.04	−0.96
FGOALS-f3-L	0.28*	0.24	−3.43*	6.29*	0.13	0.02	−0.88
FGOALS-g3	0.12	0.20	−2.10*	4.58*	0.46	0.01	−0.22
GFDL-CM4	0.20	0.43	−4.24*	6.22*	0.96	0.02	−0.48
GFDL-ESM4	0.11	0.12	−1.33	2.01*	−0.69	−0.01	−0.07
HadGEM3-GC31-LL	0.28*	0.21	−3.93*	6.65*	0.18	0.01	−0.40
HadGEM3-GC31-MM	0.30*	0.52*	−3.45*	5.61*	0.27	0.02	−0.53
INM-CM4-8	0.13	0.30	−1.91*	4.68*	0.10	0.01	0.39
INM-CM5-0	0.11*	0.03	−1.63*	5.34*	0.07	0.01	−0.66
IPSL-CM6A-LR	0.20*	0.29*	−1.29*	5.93*	−0.01	0.00	−0.92
KACE-1-0-G	0.29*	0.40*	−2.86*	6.82*	0.68	0.03	−0.48
MIROC6	−0.01	0.22	−1.31	3.76*	−0.15	0.00	−0.03
MIROC-ES2L	0.07	0.18*	−2.12*	4.51*	0.20	0.01	−0.17
MPI-ESM-1-2-HAM	0.04	0.26	−3.46*	5.44*	0.10	0.01	−0.01
MPI-ESM1-2-HR	0.25*	0.35*	−3.29*	7.37*	0.09	0.01	−0.03
MPI-ESM1-2-LR	0.19*	0.45*	−3.25*	8.18*	0.03	0.00	−0.29
MRI-ESM2-0	0.12	0.38*	−3.78*	6.37*	0.25	0.03	−0.49
NESM3	0.28*	0.21*	−3.30*	7.11*	0.03	0.00	−0.11
NorCPM1	0.18*	0.23*	−3.55*	5.69*	0.04	0.00	−0.57
NorESM2-LM	0.16	0.38*	−4.03*	6.32*	1.03	0.03	−1.46
NorESM2-MM	0.24*	0.56*	−4.33*	4.20*	0.69	0.04	−0.64
SAM0-UNICON	0.04	0.21	−2.07*	2.22*	−0.51	−0.03	−0.57
UKESM1-0-LL	0.21*	0.18*	−2.99*	6.89*	0.29	0.02	−0.17
MME	0.19*	0.31*	−2.77*	5.82*	0.23	0.01	−0.44

Note. Trend with a significance level greater than 95% is highlighted with the asterisk (*).The ensemble of the GCMs simulations is abbreviated to MME.

Precipitation Extremes

As can be seen in Figure 11, all models, except CanESM5 and IPSL-CM6A-LR, display positive biases RX5day and SDII. The biases of RX5day range from 1.2 to 48.6 mm and typically less than 2.5 mm for SDII. The top five models with the least bias are UKESM1-0-LL, CanESM5, CNRM-CM6-1, NorCPM1, and FGOALS-f3-L. In contrast, most models show negative biases for CDD of −62.4 to −2.7 days, implying the number of wet days in models is overestimated. The models IPSL-CM6A-LR, CNRM-CM6-1, CanESM5, CNRM-ESM2-1, and EC-Earth3-Veg are found to perform better than the other models.

The RX5day and SDII decrease from the southeast to the northwest over the TP. Despite models reasonably reproduce the spatial patterns of RX5day and SDII, they tend to overestimate RX5day and SDII over most parts of the TP. Most models fail to capture the observed spatial pattern of CDD (Figure S10). Figure 10 shows that the majority of models for RX5day and SDII have spatial correlations between 0.40 and 0.80, have a ratio of spatial standard deviation larger than 0.9, and have RMSE ranging from 1.0 to 2.0. This indicates models have a reasonable performance in simulating the spatial distributions of RX5day and SDII. The best five models are UKESM1-0-LL, NorESM2-LM, HadGEM3-GC31-LL, CNRM-CM6-1, and HadGEM3-GC31-MM.

An increasing trend in RX5day and SDII and a decreasing trend in CDD can be seen in observations (Table 4). The models can capture the trends of precipitation extremes in recent decades. However, the trends of SDII and CDD are generally underestimated in the models. The Taylor diagrams show the coefficient between the modeled and observed extreme precipitation is very poor, with the values being smaller than 0.40. INM-CM5-0, IPSL-CM6A-LR, MPI-ESM1-2-HR, NorCPM1, and NorESM2-LM seem to produce better results than other models for RX5day and SDII.

Summary

In this paper, we evaluated the performance of 29 CMIP6 models in simulating the mean and extreme surface air temperature and precipitation over the TP with reference to high-resolution gridded observations for the period of 1961–2012. The results show that on average over the TP, the CMIP6 ensemble models underestimate seasonal mean temperatures for all seasons with regional mean biases ranging from −2.6°C to −0.5°C; for precipitation, all the evaluated CMIP6 models overestimate the means of seasonal precipitation accumulations, with the multimodel mean estimates of spring, summer, autumn, and winter mean precipitation being 218%, 76%, 129%, and 533% of the corresponding observations (Table 5). We also found that most of CMIP6 models can reasonably capture the spatial patterns of annual and seasonal mean temperatures and precipitation, with spatial pattern correlations between the multimodel ensemble mean values and observations mainly ranging from 0.38 to 0.86 across seasons.

Table 5 Summary Results of CMIP6 Models in Simulating Mean and Extremes Climate Over the TP

		Mean		Extreme
		Temperature	Precipitation	Temperature	Precipitation
Climatology	Mean	Cold biases: 0.5°C–2.6°C across seasons	Wet biases:76%–533% across seasons	Cold biases for all indices	Positive biases in RX5day and SDII; negative biases in CDD
		Best models: MPI-ESM-1-2-HAM, UKESM1-0-LL, MPI-ESM1-2-HR, KACE-1-0-G, HadGEM3-GC31-LL	Best models: HadGEM3-GC31-MM, CNRM-CM6-1-HR, NorESM2-MM, FGOALS-f3-L, IPSL-CM6A-LR	Best models: SAM0-UNICON, NorESM2-LM, MIROC-ES2L, INM-CM4-8, NorESM2-MM for TXx; KACE-1-0-G, MRI-ESM2-0, HadGEM3-GC31-LL, NorCPM1, UKESM1-0-LL for TNn and FD	Best models: UKESM1-0-LL, CanESM5, CNRM-CM6-1, NorCPM1, EC-Earth3-Veg for RX5day and SDII; IPSL-CM6A-LR, CNRM-CM6-1, CanESM5, CNRM-ESM2-1, EC-Earth3-Veg for CDD
	Spatial patterns	COR 0.5–0.84; RMSE 0.75–1.5; RSSD 1–2	COR 0.38–0.86; RMSE 1.3–2.6; RSSD 1.5–2.7	COR 0.4–0.8; RMSE 0.75–2; RSSD 1–1.9	COR 0.4–0.8; RMSE 0.8–2.2; RSSD 0.7–2.1
		Best models: GFDL-CM4, GFDL-ESM4, HadGEM3-GC31-MM, INM-CM4-8, INM-CM5-0	Best models: MRI-ESM2-0, NorESM2-MM, NorESM2-LM, HadGEM3-GC31-MM, MIROC	Best models: GFDL-CM4, HadGEM3-GC31-MM, INM-CM4-8, INM-CM5-0, NESM3	Best models: UKESM1-0-LL, NorESM2-LM, HadGEM3-GC31-LL, CNRM-CM6-1, HadGEM3-GC31-MM for RX5day and SDII; IPSL-CM6A-LR, CNRM-CM6-1, CanESM5, CNRM-ESM2-1, NorESM2-MM for CDD
Trend	Mean	Underestimate the rates	Fail to simulate significantly increasing trend	Underestimate the rates	Capture the trends of precipitation
		Best models: NorESM2-LM, NorESM2-MM, CNRM-CM6-1-HR, HadGEM3-GC31-MM, KACE-1-0-G	Best models: CanESM5, EC-Earth3-Veg, CNRM-ESM2-1, MRI-ESM2-0, CNRM-CM6-1	Best models: UKESM1-0-LL for TXx and TN90p, NorESM2-MM for TNn and FD	Best models: MIROC-ES2L and NorESM2-LM
	Spatial patterns	COR < 0.6; RMSE 0.7–1.4; RSSD 0.5–1	COR < 0.3; RMSE 1.1–2.6; RSSD 1–2.4	COR < 0.59; RMSE 1–2; RSSD 0.5–1.7	COR < 0.4; RMSE 0.8–2.2; RSSD 0.5–2.1
		Best models: CNRM-ESM2-1, INM-CM5-0, HadGEM3-GC31-LL, HadGEM3-GC31-MM, BCC-CSM2-MR	Best models: CNRM-ESM2-1, INM-CM5-0, IPSL-CM6A-LR, MIROC-ES2L, MPI-ESM1-2-LR	Best models: INM-CM4-8, MPI-ESM1-2-HR, MPI-ESM1-2-LR, NESM3, NorESM2-LM for TXx; MPI-ESM1-2-HR, INM-CM5-0, NESM3, NorESM2-LM, NorESM2-MM for TNn and FD	Best models: INM-CM5-0, IPSL-CM6A-LR, MPI-ESM1-2-HR, NorCPM1, NorESM2-LM

Note. The abbreviation for the spatial correlation coefficient, root-mean-square error, and ratio of spatial standard deviation is COR, RMSE, and RSSD, respectively.

In the context of global warming, TP was found to have experienced more rapid temperature change comparing to many other regions over the world (Rangwala & Miller, 2012; B. Wang et al., 2008; Yao et al., 2019). Our results indicate that most of the GCMs capture the warming trend of temperature over the TP, with the annual mean rate of 0.25°C/decade. However, the models underestimate the observed rate of temperature increase, especially in winter. Observations show that the annual precipitation over the TP has significantly increased during 1961–2012. Most models exhibit a positive trend, but the trend is often statistically insignificant. Furthermore, most models can hardly reproduce the spatial pattern of the temperature and precipitation trends.

In term of climate extreme indices, the CMIP6 models underestimate the 1961–2012 means of all the four temperature extreme indices and CDD and overestimate the means of RX5day and SDII. The spatial pattern correlations between the CMIP6 multimodel ensemble mean and the corresponding observations mainly fall between 0.4 and 0.8 for these indices, suggesting that the spatial variations of these extreme indices can also be captured. Generally, models are able to simulate the signs of the trends of these extreme indices but underestimate their magnitudes. Also, the spatial patterns of trends for these extreme indices are not well simulated.

It is important to note that among the 29 CMIP6 models, none of them is able to simulate all the features of climate over the TP reasonably. As shown in Table 5, a model performs well in some aspects but may fail in other aspects. On the whole, the models, namely HadGEM3-GC31-MM, INM-CM4-8, KACE-1-0-G, MPI-ESM1-2-HR, and NorESM2-MM, perform relatively better in simulating the mean and extreme temperature. For precipitation, CNRM-CM6-1-HR, HadGEM3-GC31-MM, NorESM2-MM, NorESM2-LM, and IPSL-CM6A-LR have a better performance. Additionally, in agreement with previous studies (Z. Jiang et al., 2015; Salunke et al., 2019; Sillmann et al., 2013; Su et al., 2013), the multimodel ensemble mean simulations generally outperform individual models. Therefore, the good models out of 29 models selected in this paper in temperature and precipitation can be used to create a weighted ensemble of models for better future projections of the TP.

Recent researches have shown that there are some improvements from the CMIP5 to CMIP6 GCMs in simulating the mean and extreme temperature and precipitation globally and in China as a whole (Chen et al., 2020; D. Jiang et al., 2020; Lun et al., 2021; Zhu & Yang, 2020). Although, it seems that the cold bias and wet bias are persistent features in different generations of CMIP models, implying that state-of-the-art global climate models suffer from a common deficiency in some aspects of their formulation (Su et al., 2013). In addition, the biases of CMIP6 GCMs in the TP are larger than that in China and the global average. For example, the GCMs produce average cold biases of 0.61°C for annual mean temperature over China, which is smaller than the bias over the TP (1.4°C in this study). The wet bias resulted in overestimation of RX5day by approximately 10.7 mm, and underestimation of CDD by approximately 9.2 days on average over global lands (Chen et al., 2020). These values are much larger in TP in this study, of approximately 19 mm for RX5day and 36 days for CDD.

Meanwhile, remarkably overestimation in winter precipitation over the TP can be seen in the CMIP6 models. The precipitation over the TP is mainly affected by westerlies in winter and only a small proportion of total annual precipitation occurs in the season. In the models’ simulation, drizzle occurs frequently and results in high precipitation in winter, which is a persistent problem in many climate models (Dai & Trenberth, 2004; Sun et al., 2016). Furthermore, precipitation processes in the TP are extremely complicated and are affected not only by local convective processes and large-scale circulation but also by topography (Feng & Zhou, 2012; X. Wang et al., 2016; M. Yang et al., 2007). However, the latter factor still has not been fully accounted for in the CMIP6 models due to their coarse resolution (Q. Hu et al., 2014; Pathak et al., 2019; Su et al., 2013; Yan et al., 2013). The model physics schemes (e.g., precipitation/convection parametrization schemes) also need to be developed to reproduce the most realistic results (G. Hu & Franzke, 2020; D. Jiang et al., 2020). A topographic correction can be used to achieve the improvements in precipitation. According to Adam et al. (2006), the topographic correction can only change 20.2% of precipitation in topographically influenced region. Thus, the overestimation of precipitation would not be fundamentally changed. Complete and accurate representations of precipitation physical processes in the mountainous regions are still a challenge for current global climate models. For temperature, the topographic correction can improve notably the performance of GCMs (Q. Hu et al., 2014). However, with a focus on the basic evaluations of the CMIP6 models, this work does not consider topographic correction of the GCMs outputs. If the CMIP6 simulations are used for other purposes such as atmospheric forcing data for hydrological models, the topographic corrections are necessary.

Acknowledgments

This study was supported by the National Natural Science Foundation of China (92047301).

Data Availability Statement

The original CMIP6 data can be accessed through the ESGF data portals online (https://esgf-node.llnl.gov/search/cmip6/), where the projections used in this study can be searched by the 29 GCMs, Experiment ID (historical), Variable (pr, tas, tasmax, tasmin), and Frequency (day). CN05.1 observational data set is available through Wu and Gao (2013, http://www.geophy.cn/CN/10.6038/cjg20130406).