HESS Opinions: The need for process-based evaluation of large-domain hyper-resolution models

Abstract

A meta-analysis on 192 peer-reviewed articles reporting on applications of the variable infiltration capacity (VIC) model in a distributed way reveals that the spatial resolution at which the model is applied has increased over the years, while the calibration and validation time interval has remained unchanged. We argue that the calibration and validation time interval should keep pace with the increase in spatial resolution in order to resolve the processes that are relevant at the applied spatial resolution. We identified six time concepts in hydrological models, which all impact the model results and conclusions. Process-based model evaluation is particularly relevant when models are applied at hyper-resolution, where stakeholders expect credible results both at a high spatial and temporal resolution.

Introduction

One of the famous paradoxes of the Greek philosopher Zeno of Elea ($\sim$ 450 BC) concerns a shot arrow : “If one shoots an arrow, and cuts its motion into such small time steps that at every step the arrow is standing still, the arrow is motionless, because a concatenation of non-moving pieces cannot create motion.” Only ages later, this reasoning could be refuted by the invention of integral and differential calculus by Newton and Leibniz , accepting infinitely small rates of change. Motion is a change of location over time, thus motion links time and space.

In hydrology, it is essential to understand and predict the motion of water within the Earth system, which implies that both space and time have to be considered. In hydrological models space can be accounted for by using distributed (spatially explicit) models, where space is “cut in small pieces”, to paraphrase Zeno. Different types of distributed hydrological models exist; distinguished roughly two different classes. The first class consists of distributed differential models. These models explicitly simulate lateral fluxes by means of differential equations. The second class are the distributed integral models, which consist of one-dimensional columns and ignore lateral fluxes between the columns (lateral fluxes can be accounted for with an extra routing scheme, although this does not allow for lateral re-distribution). These models have a wide application in land surface modelling . In this discussion we focus on the latter.

The constant development in computational power, the increased understanding of physical processes, and the increased availability of high spatial resolution hydrological information stimulated the development of increasingly complex and distributed hydrological models . Increasing the spatial resolution of global hydrological models (GHMs) has been labelled as one of the current “grand challenges” in hydrology by and , who call for global modelling at the so-called spatial hyper-resolution ($\sim \mathrm{1}$ km and smaller). Arguably, there is a growing societal need for hydrological information at the (sub-)kilometre scale. Whereas model products at the 1 or ${0.5}^{\circ }$ resolution may provide relevant information for policy makers at the (inter)national level, hyper-resolution results will become relevant for local water managers or even individual farmers (see e.g. ). The scientific challenge is not to simply provide information based on a model with default parameters, but to provide credible information that matches the actual situation in the field at a temporal resolution, which is consistent with the spatial resolution of the model. The temporal and spatial scales are linked through the characteristic speed (including both velocity and celerity; see ) of the involved hydrological processes , the so-called process scale; see Fig. . The Figure shows that there is a general tendency for the temporal process scale to decrease with the spatial process scale, although there is quite a broad bandwidth and local changes might occur stepwise. Policy makers might be able to deal with model products at a monthly resolution, whereas resource managers and farmers expect, at the spatial hyper-resolution, credible model products with a daily or hourly resolution.

The timescales and space scales of several hydrometeorological processes. Adapted from and , who based it on , , , and . The blue areas indicate the temporal and spatial resolution at which the VIC model has been applied, when it was initially developed (A) and presently (B). The dashed arrow pointing downwards shows the ambitions of spatial hyper-resolution modelling, whereas the dashed arrow pointing towards (C) shows the temporal and spatial resolution of hyper-resolution modelling if it follows the direction of characteristic velocity of hydrometeorological processes.

[Figure omitted. See PDF]

Although increasing the spatial resolution of hydrological models is claimed to provide the opportunity to improve physical process representation , almost every hydrological model requires calibration of the model parameters . Models can contain conceptual parameters, which have no directly measurable physical meaning and thus need calibration. In addition, the measurement scale of parameters which do have a physical meaning often differs from the model scale, making calibration necessary to determine the effective parameter values to account for sub-grid variability . responded to the hyper-resolution challenge by emphasizing that the focus of hydrologic modelling should be on determining and accounting for epistemic uncertainty and appropriate parameterizations at different spatial resolutions, rather than on maximizing the spatial resolution. Increasing the spatial resolution of the model (towards hyper-resolution) is not a solution to sub-grid variability, since many of the relevant processes take place on even smaller scales . Hence, despite their increasing spatial resolution, also GHMs require calibration in order to obtain effective parameters, and validation to determine model credibility. Even if a correct physical representation of hydrological processes is impossible, the goal of the model should be to mimic realism and hydrological processes as closely as possible . This implies that the models should be subject to a process-based calibration and validation procedure . Since different hydrological processes dominate at different scales (Fig. ), the temporal and spatial scales are linked. Because the spatial resolution of GHMs is currently being increased to meet societal needs , the temporal resolution should decrease accordingly to meet these needs. This should be reflected in the calibration and validation time interval of the model, in order to guarantee model credibility at the required temporal and spatial resolution.

Timescales

A short review of scientific literature about scaling issues provides the impression that the focus has mostly been on the spatial scale and/or resolution rather than on its temporal counterpart . Many concepts have been developed to describe representative areas and volumes . In soil physics, the representative elementary volume (REV) is an often used concept, which describes the volume for which a measurement can be considered representative . explored a similar concept with applications in hydrology, namely the representative elementary area (REA), the critical area at which the pattern of small-scale heterogeneity becomes unimportant. proposed the representative elementary watershed (REW), allowing for closure of the balance equations averaged over time and space. Similar concepts, which statistically integrate temporal variations, have not been reported in the literature. The lack of attention for the temporal scale, however, is remarkable because hydrological states and fluxes are mostly studied as a function of time. As an illustration of the lack of attention for the aspects of temporal scale, it should be noted that in the recent papers by and on spatial hyper-resolution modelling, the temporal resolution of these models is referred to only once. One of the reasons why the development of a representative elementary time step (RET) is more complex is that several different time concepts play a role in hydrological modelling.

As a guideline and first step for the discussion on time dimensions in hydrological models, we identify six time concepts, which in practice are often mixed up and misinterpreted. A distinction is made between scale, which is defined as a continuous variable, resolution, defined as discrete variable being a model property, and time interval, which is a discrete variable independent of the used model. The six concepts are

• the process timescale

• the input resolution

• the numerical resolution (time step)

• the output resolution (temporal resolution)

• the calibration/validation time interval

• the interpretation time interval.

First, the process timescale is defined, as the characteristic timescale of the hydrological process considered. This is the typical time period over which the process takes place. Infiltration excess overland flow, for instance, has a relatively short timescale, whereas regional groundwater flow has a longer timescale. The end user determines which process is most relevant in the modelling procedure.

Second, the temporal resolution of the input data or input resolution is relevant for the modelled process. The input resolution of the forcing data can differ from the output resolution of the model, and this can impact the results of the model. An example is given in the upper panels of Fig. , showing an application of the Green–Ampt infiltration model.

Application of the Green–Ampt infiltration scheme for different input resolutions (upper row), different numerical resolutions (middle row), and different output resolutions (lower row). For each set-up, the model was fed with the same extreme precipitation event of 32 mm of rain in 30 min (4 mm in first 5 min, 5 mm in 5–10 min, 7 mm in 10–20 min, 5 mm in 20–25 min and 4 mm in 25–30 min). The model parameters have been kept constant; saturated hydrologic conductivity $K_{\mathrm{s}}=0.044$ cm h${}^{-\mathrm{1}}$, initial soil moisture ${\mathit{\theta }}_{\mathrm{i}}=0.1$, saturated soil moisture ${\mathit{\theta }}_{\mathrm{s}}=0.5$, matric pressure at wetting front $\mathrm{\Psi }=22.4$ cm. Each of the three time concepts impacts the conclusions that are drawn from the model results, which shows that calibration and validation at the appropriate time interval is essential to resolve the processes taking place.

[Figure omitted. See PDF]

The numerical resolution (or the time step) of the model is the time interval over which the model calculates the states and the fluxes internally. A model can only deterministically resolve a process if the numerical resolution is higher than the characteristic timescale of the process. The panels in the second row of Fig.  show how the numerical resolution impacts model output for the process of ponding, which leads to different conclusions about ponding, based on the model output.

The output resolution (often referred to as simply temporal resolution) is the time interval at which the model output yields the states and fluxes. This time interval can be equal to the numerical resolution of the model, or aggregated from the numerical resolution. The modelled process can only be identified if the output time interval is shorter than the characteristic timescale of the process, which is shown in the lower panels of Fig. .

The calibration and validation time interval of the model is defined here as the time interval at which the model output is being confronted with observations. Calibration and validation of the model output can be conducted at another time interval than the output resolution, by aggregating the model output. Calibration and validation should be performed at a time interval smaller than or equal to the timescale of the process that is relevant for the end user.

Finally, the interpretation time interval is defined as the time interval at which the model output is eventually analysed or interpreted. This can be equal to the calibration time interval, or the model output can be further aggregated resulting in a larger interpretation time interval (e.g. from daily to monthly). Since the model has not been validated or calibrated on time intervals smaller than the calibration time interval, the credibility of the results will be unknown for time interval smaller than the calibration time interval.

It is critical to note that some of these time concepts are necessarily equal to or larger than related time concepts, sometimes for logical reasons (the output resolution cannot be higher than the numerical resolution) and sometimes for model credibility reasons (the interpretation time interval should not be smaller than the calibration time interval). It is also important to note that the first time concept, the process scale, explicitly links the temporal and the spatial scale . Conversely, the spatial resolution of a model will set a minimum temporal resolution determining which processes need to be resolved.

Example for VIC model studies

To illustrate the development of calibration/validation time interval and spatial resolution in large-domain hydrological modelling, we carried out a meta-analysis on the use of GHMs. The variable infiltration capacity (VIC) model was chosen for this analysis because it is widely used and therefore enough studies were available for a meta-analysis. The VIC model is mentioned explicitly in as a type of model being run at the spatial hyper-resolution. Sub-grid variability is parameterized as a distribution of responses without explicit treatment of the pattern. We believe this model is representative of the much larger class of global hydrological models.

The VIC model was initially constructed to couple climate model output to hydrological processes: it is capable of solving both the energy and the water balance. developed a horizontal routing model to couple the individual grid cells of the VIC model. This facilitated the distributed application of VIC for rainfall–runoff processes at large domains. No explicit definition of a spatial derivative or scale appears in the equations of the VIC model, the spatial resolution of the model only appears in the routing scheme through the horizontal flow velocity (see for a description of space–time representation in other distributed hydrologic models).

In our analysis we assembled 242 peer-reviewed studies that used the VIC model. Of these, 192 studies used the model in a distributed way and performed a calibration or validation on the model output (see Table  in Appendix A). Figure  presents a space–time perspective on the application of the VIC model during the past 2 decades. As expected, the spatial resolution at which the model is applied has increased steadily over the years (Fig. a). While the model was initially constructed for spatial resolutions of the order of 0.5 to 2${}^{\circ }$, it is now mostly applied at 1/8${}^{\circ }$ and smaller. The main driver for the increase in spatial resolution is the availability of high-resolution spatial data sets, such as that presented by . The increase in resolution, however, does not apply to the employed calibration and validation time interval. Figure b shows that the time interval at which the model has been calibrated and validated has remained steady over the years. Therefore, while the spatial resolution of the model has increased, the model output is still calibrated and validated at the original coarse time interval. Processes with a short timescale, which become more important when the spatial resolution increases, will likely be overlooked during the calibration and validation of the model if the time interval is too coarse. Several studies have already shown that calibration on a coarser time interval does not guarantee credible results for shorter time intervals . There are, however, examples of studies where the interpretation time interval is smaller than the calibration time interval, e.g. and .

The year of publication versus the highest spatial resolution of the VIC model that was used in the study (a), the smallest time interval on which the calibration and/or validation of the VIC model was performed (b), and the total number of grid cells in the study (c) based on 192 peer-reviewed studies. The grey lines in (c) show the slope of computational power increase according to Moore's law . The point size is proportional to the number of studies that were published in a certain year with a certain spatial or temporal resolution. If the spatial resolution was given in kilometres, it was assumed that ${\mathrm{1}}^{\circ }=100$ km. For the total number of grid cells, catchment size was divided by cell size, assuming that ${\mathrm{1}}^{\circ }=100$ km, unless the number of grid cells was explicitly given. Statistics (the mean and the standard deviation) have been obtained per year on logarithmically transformed data. With linear regression a line was fitted through the mean and the standard deviation.

[Figure omitted. See PDF]

Figure  indicates the initial development scale of the VIC model (A), the scale where it is heading to right now (B), and the direction where it should go in order to resolve relevant hydrometeorological processes (C). Therefore, the VIC model with a high spatial resolution should be calibrated and/or validated at a time interval short enough to catch the processes relevant at those particular spatial scales.

Two causes for the discrepancy in the joint development of spatial resolutions and calibration time intervals come to mind: lack of computational power, or a lack of (using) observations with a high temporal frequency. Figure c shows that the total number of grid cells that was used in the studies has on average increased over time. This is as expected: computational power has increased significantly over the years. According to Moore's law , computational power roughly doubles every 2 years. The grey lines in Fig. c indicate the corresponding slope in computational power on a log–log scale. The largest numbers of grid cells per year likely indicate the limit of technical capability. Overall, the trend in the studies, even in the higher quantiles, is much lower than the computational limit, suggesting that computational power is not a constraint for most studies. This implies that, presently, the main constraint for calibration and validation of distributed hydrological models at a certain time interval (Fig. b) is not the computational power, but the lack of (using) observations with a high temporal frequency. A possible explanation for this may be that many (global) studies rely on data from the Global Runoff Data Centre (GRDC), which are often available only at the monthly time interval. Also important is that for large basins, the typical application scale of VIC and other GHMs, flow is often regulated by dams for hydropower and flood control. Naturalized flows for these basins are often estimated at the monthly time interval. Our results reinforce the conclusion of that field observations should account for the spatial and temporal heterogeneity of hydrometeorological processes, and the statement from that in most cases, temporal resolution is fixed by the data collection procedure.

Problem statement and outlook

The meta-anlysis on VIC studies showed that the spatial resolution at which the model is applied has increased over the years, while the calibration time interval has remained steady (Fig. ). The examples are shown for the VIC model only, but we have the impression that the obtained trends apply for all GHMs. There is a general tendency to move towards higher spatial resolution in large-domain hydrological models (induced by e.g. ; ), whereas the available data for calibration and validation are model independent.

Although coarse temporal resolution data can be used to constrain model uncertainty, the ambition to move towards spatial hyper-resolution hydrological models with predictive capabilities should keep pace with the data that are required to run, calibrate, and validate the models. Increasing the spatial resolution of the model implies modelling different relevant hydrometeorological processes (there are some interesting developments concerning parameter transferability over spatial resolutions; see e.g. , , and ), which in turn requires calibration and validation to be performed on a smaller time interval. It requires a community effort to increase the availability of high temporal resolution data for calibration and validation of large-domain hydrological models. Especially for large-domain studies, where data collection from all the separate basins at different institutes and countries is very time consuming (explaining the success of the GRDC), the data need to be gathered at and accessible from one point. It should also be recognized that discharge data only, especially at a monthly timescale, do not provide sufficient information for a process-based model evaluation at the spatial hyper-resolution scale. Possible paths forward are the use of tracer data to identify different flow paths , the use of multiple objectives , and the use of satellite and remote sensing data , all at a representative spatial and temporal resolution.

We acknowledge that calibration and validation at the appropriate time interval is only one of the many challenges of spatial hyper-resolution hydrological modelling. Even with enough observations available for calibration and validation, disinformative data , correct subgrid parameterizations , and model structural uncertainty remain outstanding challenges. However, we believe that all these challenges can only be tackled if the models are subject to critical and process-based evaluation and validation . In the end, the goal is to model hydrological processes in an appropriate way .

Along with an increased spatial resolution of the model products, there will be a shift in users' expectations of those products. Whereas coarse-scale (0.5 to 1${}^{\circ }$) products may provide relevant information for policy makers at the national or state level, products at the spatial hyper-resolution (0.1 to 1 km) are potentially of interest to a much wider range of users, including for instance farmers that want to schedule their irrigation. At the sub-kilometre scale, new processes such as infiltration excess overland flow and ponding can (and should) be resolved, but at the same time these processes cannot be explicitly resolved at a daily or monthly time interval. Thus, the recent call for increasing the spatial resolution of distributed hydrological models should not focus solely on the spatial resolution, but should aim to increase the evaluation time interval simultaneously, at a balanced rate consistent with the characteristic timescales and space scales of the relevant hydrological processes (Fig. ). We believe that such a balanced approach will serve societal needs best.

Articles in the meta-analysis

All articles used to create Fig. , with their highest spatial resolution (Spat.; in degrees) and the time interval (Temp) used for calibration and validation.

Authors	Journal	Year	Title	Spat.	Temp.
Abdullah, F. A. and D. P. Lettenmaier	J. Hydrol.	1997	Application of regional parameter ...	1.000	monthly
Acharya, A., et al.	J. Hydrol.	2011	Modeled streamflow response ...	0.125	monthly
Adam, J. C., et al.	J. Geophys. Res.	2007	Simulation of reservoir influences ...	1.000	monthly
Agboma, C. O., et al.	J. Hydrol.	2009	Intercomparison of the total storage ...	0.300	monthly
Ahmad, S., et al.	Adv. Water Resour.	2010	Estimating soil moisture ...	0.125	daily
Andreadis, K. M. and D. P. Lettenmaier	Adv. Water Resour.	2006	Assimilating remotely sensed ...	0.125	daily
Arora, V. K. and G. J. Boer	J. Climate	2006	The temporal variability of ...	2.000	monthly
Ashfaq, M., et al.	J. Geophys. Res.	2010	Influence of climate model ...	0.125	daily
Bao, Z., et al.	J. Hydrol.	2012	Comparison of regionalization ...	0.250	monthly
Bao, Z., et al.	J. Hydrol.	2012	Attribution for decreasing ...	0.250	monthly
Bao, Z., et al.	Hydrol. Process.	2012	Sensitivity of hydrological ...	0.250	monthly
Bohn, T. J., et al.	Environ. Res. Lett.	2007	Methane emissions from ...	1.000	daily
Bohn, T. J., et al.	J. Hydrometeorol.	2010	Seasonal Hydrologic Forecasting ...	0.125	monthly
Bowling, L. C., and D. P. Lettenmaier	J. Hydrometeorol.	2010	Modeling the Effects of ...	0.125	hourly
Chang, J., et al.	Quaternary Int.	2014	Impact of climate change ...	0.500	daily
Cherkauer, K. A., and D. P. Lettenmaier	J. Geophys. Res.	1999	Hydrologic effects of frozen soils ...	0.500	daily
Christensen, N. S., et al.	Climatic Change	2004	The effect of climate change on ...	0.125	daily
Christensen, N. S., and D. P. Lettenmaier	Hydrol. Earth Syst. Sci.	2007	A multimodel ensemble approach ...	0.125	daily
Costa-Cabral, M., et al.	Climatic Change	2013	Snowpack and runoff response ...	0.125	monthly
Crow, W. T., et al.	J. Geophys. Res.	2003	Multiobjective calibration of ...	0.125	hourly
Cuo, L., et al.	J. Hydrol.	2013	The impacts of climate change ...	0.250	daily
Demaria, E. M. C. , et al.	J. Hydrol.	2013	Climate change impacts on ...	0.250	daily
Demaria, E. M. C., et al.	Int. J. River Bas. Manag.	2014	Satellite precipitation in ...	0.125	monthly
Díaza, A., et al.	Int. J. River Bas. Manag.	2013	Multi-annual variability of ...	0.125	daily
Drusch, M., et al.	Geophys. Res. Lett.	2005	Observation operators for the ...	0.125	daily
Eum, H., et al.	Hydrol. Process.	2014	Uncertainty in modelling the ...	0.063	daily
Fan, Y. et al.	J. Hydrometeorol.	2011	Verification and Intercomparison ...	0.125	monthly
Feng, X., et al.	J. Hydrometeorol.	2008	The Impact of Snow Model ...	0.125	daily
Ferguson, C. R., et al.	Int. J. Remote Sens.	2010	Quantifying uncertainty in ...	0.125	monthly
Ferguson, C. R., et al.	J. Hydrometeorol.	2012	A Global Intercomparison of ...	0.250	daily
Gao, H., et al.	J. Hydrometeorol.	2004	Using a Microwave Emission ...	0.125	daily
Gao, H., et al.	J. Hydrometeorol.	2006	Using TRMM/TMI to Retrieve ...	0.125	daily
Gao, H., et al.	J. Hydrometeorol.	2007	Copula-Derived Observation ...	0.125	daily
Gao, H., et al.	Int. J. Remote Sens.	2010	Estimating the water budget ...	0.500	monthly
Gao, Y., et al.	J. Geophys. Res.	2011	Evaluating climate change ...	0.125	monthly
Garg, V., et al.	J. Hydr. Eng.	2013	Hypothetical scenario?based ...	0.250	yearly
Gebregiorgis, A. and F. Hossain	J. Hydrometeorol.	2011	How Much Can A Priori Hydrologic ...	0.125	daily
Gebregiorgis, A. S., et al.	Water Resour. Res.	2012	Tracing hydrologic model ...	0.125	daily
Gu, H., et al.	Stoch. Environ. Res. Risk Ass.	2014	Impact of climate change ...	0.125	daily
Guerrero, M., et al.	Int. J. River Bas. Manag.	2013	Parana River morphodynamics ...	0.125	monthly
Guo, J., et al.	J. Hydrol.	2004	Impacts of different precipitation ...	0.125	daily
Guo, J., et al.	Proc. Env. Sci.	2011	Daily runoff simulation in ...	0.042	daily
Haddeland, I., et al.	Gephys. Res. Lett.	2006	Anthropogenic impacts on ...	0.500	monthly
Haddeland, I., et al.	J. Hydrometeorol.	2006	Reconciling Simulated Moisture ...	0.125	hourly
Haddeland, I., et al.	J. Hydrol.	2006	Effects of irrigation on the ...	0.500	daily
Hamlet, A. F., et al.	J. Climate	2005	Effects of Temperature and ...	0.125	monthly
Hamlet, A. F. and D. P. Lettenmaier	Water Resour. Res.	2007	Effects of 20th century warming ...	0.125	monthly
Hidalgo, H. G., et al.	J. Hydrol.	2013	Hydrological climate change ...	0.500	monthly
Hillarda, Y., et al.	Remote Sens. Environ.	2003	Assessing snowmelt dynamics ...	0.125	daily
Huang, M., et al.	J. Geophys. Res.	2003	A transferability study of model ...	0.130	daily
Hurkmans, R. T. W. L., et al.	Water Resour. Res.	2008	Water balance versus land ...	0.088	daily
Hurkmans, R. T. W. L., et al.	Water Resour. Res.	2009	Effects of land use changes ...	0.050	daily
Hurkmans, R., et al.	J. Climate	2010	Changes in Streamflow Dynamics ...	0.088	daily
Jayawardena, A. W., et al.	J. Hydrolog. Eng.	2002	Meso-Scale Hydrological ...	1.000	daily
Kam, J., et al.	J. Climate	2013	The Influence of Atlantic ...	0.125	daily
Lakshmi, V., et al.	Gephys. Res. Lett.	2004	Soil moisture as an ...	0.125	monthly
Li, J., et al.	J. Hydrometeorol.	2007	Modeling and Analysis ...	0.042	daily
Li, H., et al.	J. Hydrometeorol.	2013	A Physically Based Runoff ...	0.063	monthly
Liang, X. and Z. Xie	Adv. Water Resour.	2001	A new surface runoff ...	0.125	daily
Liang, X. and Z. Xie	Global Planet. Change	2003	Important factors in land? ...	0.125	daily
Liang, X., et al.	J. Geophys. Res.	2003	A new parameterization ...	0.125	daily

Authors	Journal	Year	Title	Spat.	Temp.
Liang, X., et al.	J. Hydrol.	2004	Assessment of the effects ...	0.031	daily
Liu, Z., et al.	Hydrol. Process.	2010	Impacts of climate change on ...	0.500	daily
Liu, L., et al.	J. of Flood Risk Manag.	2013	Hydrological analysis for water ...	0.010	daily
Liu, H., et al.	Hydrol. Earth Syst. Sci.	2013	Soil moisture controls on ...	0.500	monthly
Liu, X., et al.	Hydrol. Earth Syst. Sci.	2014	Effects of surface wind speed ...	0.250	monthly
Livneh, B., et al.	J. Climate	2013	A Long-Term Hydrologically ...	0.063	monthly
Lohmann, D., et al.	Hydrolog. Sci. J.	1998	Regional scale hydrology: ...	0.167	daily
Lu, X., and Q. Zhuang	J. Geophys. Res.	2012	Modeling methane emissions ...	0.333	daily
Lucas-Picher, P., et al.	Atmosphere-Ocean	2003	Implementation of a ...	0.405	monthly
Luo, Y., et al.	J. Hydrometeorol.	2005	The Operational Eta Model ...	0.125	monthly
Luo, L. and E. F. Wood	Gephys. Res. Lett.	2007	Monitoring and predicting ...	0.125	monthly
Luo, L. and E. F. Wood	J. Hydrometeorol.	2008	Use of Bayesian Merging ...	0.125	monthly
Lutz, E. R., et al.	Water Resour. Res.	2012	Paleoreconstruction of cool ...	0.063	monthly
Mao, D. and K. A. Cherkauer	J. Hydrol.	2009	Impacts of land-use change ...	0.125	monthly
Mao, D., et al.	Water Resour. Res.	2010	Development of a coupled ...	0.125	daily
Marshall, M., et al.	Climate Dynamics	2012	Examining evapotranspiration ...	1.000	monthly
Matheussen, B., et al.	Hydrol. Process.	2000	Effects of land cover change ...	0.250	monthly
Maurer, E. P., et al.	J. Geophys. Res.	2001	Evaluation of the land ...	0.125	monthly
Maurer, E. P., et al.	J. Climate	2002	A Long-Term Hydrologically ...	0.125	monthly
McGuire, M., et al.	J. Water Resour. Plan. Manage.	2006	Use of Satellite Data for ...	0.125	monthly
Meng, L. and S. M. Quiring	Int. J. Climatol.	2010	Observational relationship of ...	0.500	monthly
Miguez-Macho, G., et al.	B. Am. Meterol. Soc.	2008	Simulated water table ...	0.008	monthly
Miller, W. P., et al.	J. Water Res. Pl. Manag.	2012	Water Management Decisions ...	0.125	monthly
Minihane, M. R.	Phys. Chem. Earth	2012	Evaluation of streamflow ...	0.250	monthly
Mishra, V., et al.	J. Hydrometeorol.	2010	Parameterization of Lakes ...	0.125	daily
Mishra, V. and K. A. Cherkaue	Agric. For. Meteorol.	2010	Retrospective droughts in ...	0.125	monthly
Mishra, V., et al.	J. Hydrometeorol.	2010	Assessment of Drought due ...	0.125	monthly
Mishra, V., et al.	Int. J. Clim.	2010	A regional scale assessment ...	0.125	monthly
Mishra, V., et al.	Global Planet. Change	2011	Lake Ice phenology of ...	0.125	daily
Mishra, V., et al.	Global Planet. Change	2011	Changing thermal dynamics ...	0.125	daily
Mishra, V. and K. A. Cherkauer	J. Geophys. Res.	2011	Influence of cold season ...	0.125	daily
Mo, K. C.	J. Hydrometeorol.	2008	Model-Based Drought Indices ...	0.500	monthly
Mo, K. C., et al.	J. Hydrometeorol.	2012	Uncertainties in North American ...	0.500	monthly
Munoz-Arriola, F., et al.	Water Resour. Res.	2009	Sensitivity of the water resources ...	0.125	monthly
Nijssen, B., et al.	Water Resour. Res.	1997	Streamflow simulation for ...	0.500	monthly
Nijssen, B., et al.	J. Climate	2001	Global Retrospective Estimation ...	2.000	monthly
Nijssen, B., et al.	Climatic Change	2001	Hydrologic sensitivity of global ...	1.000	monthly
Niu, J., et al.	J. Hydrol.	2013	Impacts of increased CO${}_{\mathrm{2}}$ ...	1.000	monthly
Niu, J. and J. Chen	Hydrological Sciences Journal	2014	Terrestrial hydrological responses ...	1.000	daily
Niu, J. and B. Sivakumar	Stoch Environ Res Risk Assess	2014	Study of runoff response to ...	1.000	monthly
Niu, J., et al.	Hydrol. Earth Syst. Sci.	2014	Teleconnection analysis of ...	1.000	monthly
Null, S. E. and J. H. Viers	Water Resour. Res.	2013	In bad waters: Water year ...	0.125	monthly
O'Donnell, G. M., et al.	J. Geophys. Res.	2000	Macroscale hydrological modeling ...	0.500	monthly
Oubeidillah, A. A., et al.	Hydrol. Earth Syst. Sci.	2014	A large-scale, high-resolution ...	0.042	monthly
Ozdogan, M.	Hydrol. Earth Syst. Sci.	2011	Climate change impacts on ...	0.125	monthly
Pan, M. and E. F. Wood	J. Hydrometeorol.	2006	Data Assimilation for ...	0.500	daily
Parada, L. M. and X. Liang	J. Geophys. Res.	2004	Optimal multiscale Kalman ...	0.125	daily
Parada, L. M. and X. Liang	J. Geophys. Res.	2008	Impacts of spatial resolutions ...	0.125	daily
Park, D. and M. Markus	J. Hydrol.	2014	Analysis of a changing ...	0.125	daily
Parr, D. and G. Wang	Global Planet. Change	2014	Hydrological changes in the ...	0.030	daily
Qiao, L., et al.	Water Resour. Manag.	2014	Climate Change and ...	0.125	daily
Qin, S., et al.	Int. J. Remote Sens.	2013	Development of a hierarchical ...	0.030	daily
Raje, D. and R. Krishnan	Water Resour. Res.	2012	Bayesian parameter uncertainty ...	1.000	monthly
Raje, D., et al.	Hydrol. Process.	2014	Macroscale hydrological modelling ...	1.000	monthly
Ray, R. L., et al.	Remote Sens. Environ.	2010	Landslide susceptibility mapping ...	0.010	daily
Rhoads, J., et al.	J. Geophys. Res.	2001	Validation of land surface models ...	1.000	daily
Rosenberg, E. A., et al.	Water Resour. Res.	2011	Statistical applications of...	0.063	monthly
Rosenberg, E. A., et al.	Hydrol. Earth Syst. Sci.	2013	On the contribution of ...	0.125	daily
Saurral, R. I.	J. Hydrometeorol.	2010	The Hydrologic Cycle of the ...	0.125	monthly
Schaller, M. F. and Y. Fan	J. Geophys. Res.	2009	River basins as groundwater ...	0.125	monthly
Schumann, G. J.-P., et al.	Water Resour. Res.	2013	A first large-scale flood ...	0.250	monthly
Sheffield, J., et al.	J. Geophys. Res.	2003	Snow process modeling ...	0.125	daily
Sheffield, J., et al.	J. Hydrometeorol.	2012	Representation of Terrestrial ...	0.500	monthly
Shi, X., et al.	Environ. Res. Lett.	2011	The role of surface energy ...	1.000	weekly
Shi, X., et al.	J. Climate	2013	Relationships between Recent ...	1.000	monthly

Authors	Journal	Year	Title	Spat.	Temp.
Shrestha, R. R., et al.	Hydrol. Process.	2012	Modelling spatial and ...	0.063	monthly
Shrestha, K. Y., et al.	J. Hydrometeorol.	2014	An Atmospheric-Hydrologic ...	0.250	daily
Shrestha, R. R., et al.	J. Hydrometeorol.	2014	Evaluating Hydroclimatic ...	0.063	daily
Shrestha, R. R., et al.	Hydrol. Process.	2014	Evaluating the ability of a ...	0.063	monthly
Shukla, S., et al.	Hydrol. Earth Syst. Sci.	2012	Value of medium range ...	0.500	2-weeks
Shukla, S., et al.	Hydrol. Earth Syst. Sci.	2012	On the sources of global land ...	0.500	monthly
Sinha, T., et al.	J. Hydrometeorol.	2010	Impacts of Historic Climate ...	0.125	weekly
Sinha, T. and K. A. Cherkauer	J. Geophys. Res.	2010	Impacts of future climate ...	0.125	weekly
Sinha, T. and A. Sankarasubramanian	Hydrol. Earth Syst. Sci.	2013	Role of climate forecasts and ...	0.125	monthly
Slater, A. G., et al.	J. Geophys. Res.	2007	A multimodel simulation of ...	1.000	monthly
Sridhar, V., et al.	Climate Dynamics	2013	Explaining the hydroclimatic ...	0.125	monthly
Stephen, H., et al.	Hydrol. Earth Syst. Sci.	2010	Relating surface backscatter ...	0.125	daily
Su, F., et al.	J. Geophys. Res.	2005	Streamflow simulations of ...	1.000	monthly
Su, F., et al.	J. Geophys. Res.	2006	Evaluation of surface water ...	1.000	monthly
Su, F., et al.	J. Hydrometeorol.	2008	Evaluation of TRMM Multisatellite ...	0.125	daily
Su, F. and D. P. Lettenmaier	J. Hydrometeorol.	2009	Estimation of the Surface ...	0.125	monthly
Tang, C. and T. C. Piechota	J. Hydrol.	2009	Spatial and temporal soil ...	0.125	monthly
Tang, Q. and D. P. Lettenmaier	Int. J. Remote Sens.	2010	Use of satellite snow-cover ...	0.063	monthly
Tang, C., et al.	J. Hydrol.	2011	Relationships between ...	0.125	monthly
Tang, Q., et al.	J. Hydrometeorol.	2012	Predictability of Evapotranspiration ...	0.063	daily
Tang, C., et al.	Global Planet. Change	2012	Assessing streamflow sensitivity ...	0.063	monthly
Tang, C. and R. L. Dennis	Global Planet. Change	2014	How reliable is the offline ...	0.125	monthly
Vano, J. A. et al.	J. Hydrometeorol.	2012	Hydrologic Sensitivities of ...	0.125	monthly
VanShaar, J. R. et al.	Hydrol. Process.	2012	Effects of land-cover changes ...	0.125	monthly
Vicuna, S. et al.	J. Am. Water Resour. As.	2007	The sensitivity of California ...	0.125	monthly
Voisin, N., et al.	J. Hydrometeorol.	2008	Evaluation of Precipitation ...	0.500	monthly
Voisin, N.,et al.	Weather Forecast.	2011	Application of a Medium-Range ...	0.250	daily
Wang, A., et al.	J. Geophys. Res.	2008	Integration of the variable ...	0.125	monthly
Wang, J., et al.	Int. J. Clim.	2010	Quantitative assessment of climate ...	0.125	monthly
Wang, G .Q, et al.	Hydrol. Earth Syst. Sc.	2012	Assessing water resources in ...	0.500	daily
Werner, A. T., et al.	Atmosphere-Ocean	2013	Spatial and Temporal Change ...	0.063	daily
Wen, Z., et al.	Water Resour. Res.	2012	A new multiscale routing ...	0.031	daily
Wenger, S. J., et al.	Water Resour. Res.	2010	Macroscale hydrologic ...	0.063	daily
Wojcik, R., et al.	J. Hydrometeorol.	2008	Multimodel Estimation of ...	0.125	hourly
Wood, A. W., et al.	J. Geophys. Res.	2002	Long-range experimental ...	0.125	monthly
Wood, A. W., et al.	J. Geophys. Res.	2005	A retrospective assessment ...	0.125	monthly
Wu, Z., et al.	Atmosphere-Ocean	2007	Thirty-Five Year (1971–2005) ...	0.300	daily
Wu, Z. Y., et al.	Hydrol. Earth Syst. Sci.	2011	Reconstructing and analyzing ...	0.300	daily
Wu, H., et al.	Water Resour. Res.	2014	Real-time global flood ...	0.125	daily
Xia, Y., et al.	J. Geophys. Res.	2012	Continental-scale water ...	0.125	daily
Xia, Y., et al.	Hydrol. Process.	2012	Comparative analysis of ...	0.125	monthly
Xia, Y., et al.	Hydrol. Process.	2014	Evaluation of NLDAS-2 ...	0.125	daily
Xie, Z., et al.	J. Hydrometeorol.	2007	Regional Parameter Estimation ...	0.500	monthly
Yang, G., et al.	J. Hydrometeorol.	2010	Hydroclimatic Response of ...	0.125	daily
Yang, G., et al.	Landscape Urban Plan.	2011	The impact of urban development ...	0.125	daily
Yang, G. and L. C. Bowling	Water Resour. Res.	2014	Detection of changes in ...	0.125	daily
Yearsley, J.	Water Resour. Res.	2012	A grid-based approach for ...	0.063	daily
Yong, B., et al.	Water Resour. Res.	2010	Hydrologic evaluation of ...	0.063	daily
Yong, B., e al.	J. Hydrometeorol.	2013	Spatial-Temporal Changes of ...	0.063	daily
Yuan, F., et al.	Can. J. Remote Sens.	2004	An application of the VIC-3L ...	0.250	daily
Yuan, X., et al.	Hydr. Sci. J.	2009	Sensitivity of regionalized ...	0.500	monthly
Yuan, X., et al.	J. Hydrometeorol.	2013	Probabilistic Seasonal ...	0.250	monthly
Zeng, X., et al.	J. Hydrometeorol.	2010	Comparison of Land?Precipitation ...	0.125	monthly
Zhang, X., et al.	Phys. Chem. Earth	2012	Modeling and assessing ...	0.031	monthly
Zhang, B., et al.	Agr. Water Manage.	2012	Drought variation trends in ...	0.500	yearly
Zhang, B., et al.	Theor. Appl. Climatol.	2013	A drought hazard assessment ...	0.500	yearly
Zhang, B., et al.	Hydrol. Process.	2014	Assessing the spatial and ...	0.500	yearly
Zhang, X., et al.	J. Hydrometeorol.	2014	A Long-Term Land Surface ...	0.250	monthly
Zhang, B., et al.	Hydrol. Process.	2014	Spatiotemporal analysis of climate ...	0.500	yearly
Zhao, F., et al.	J. Hydrometeorol.	2012	Application of a Macroscale ...	0.050	daily
Zhao, X. and P. Wu	Natural Hazards	2013	Meteorological drought over ...	0.500	yearly
Zhao, Q., et al.	Env. Earth Sci.	2013	Coupling a glacier melt model ...	0.083	daily
Zhao, F., et al.	J. Hydrol.	2013	The effect of spatial rainfall ...	0.050	daily
Zhu, C. and D. P. Lettenmaier	J. Climate	2007	Long-Term Climate and ...	0.125	monthly
Ziegler, A. D., et al.	J. Climate	2003	Detection of Intensification in ...	2.000	monthly
Ziegler, A. D., et al.	Climatic Change	2005	Detection of time for plausible ...	0.125	monthly

Acknowledgements

The authors would like to thank Claudia Brauer and Massimiliano Zappa for their useful suggestions concerning a draft version of this paper. Edited by: E. Zehe