Evaluation of GPM IMERG Early, Late, and Final rainfall estimates using WegenerNet gauge data in southeastern Austria

Abstract

The Global Precipitation Measurement (GPM) Integrated Multi-satellite Retrievals for GPM (IMERG) products provide quasi-global (60${}^{\circ }$ N–60${}^{\circ }$ S) precipitation estimates, beginning March 2014, from the combined use of passive microwave (PMW) and infrared (IR) satellites comprising the GPM constellation. The IMERG products are available in the form of near-real-time data, i.e., IMERG Early and Late, and in the form of post-real-time research data, i.e., IMERG Final, after monthly rain gauge analysis is received and taken into account. In this study, IMERG version 3 Early, Late, and Final (IMERG-E,IMERG-L, and IMERG-F) half-hourly rainfall estimates are compared with gauge-based gridded rainfall data from the WegenerNet Feldbach region (WEGN) high-density climate station network in southeastern Austria. The comparison is conducted over two IMERG 0.1${}^{\circ }$ $\times$ 0.1${}^{\circ }$ grid cells, entirely covered by 40 and 39 WEGN stations each, using data from the extended summer season (April–October) for the first two years of the GPM mission. The entire data are divided into two rainfall intensity ranges (low and high) and two seasons (warm and hot), and we evaluate the performance of IMERG, using both statistical and graphical methods. Results show that IMERG-F rainfall estimates are in the best overall agreement with the WEGN data, followed by IMERG-L and IMERG-E estimates, particularly for the hot season. We also illustrate, through rainfall event cases, how insufficient PMW sources and errors in motion vectors can lead to wide discrepancies in the IMERG estimates. Finally, by applying the method of, we find that IMERG-F half-hourly rainfall estimates can be regarded as a 25 min gauge accumulation, with an offset of$+$40 min relative to its nominal time.

Introduction

The Global Precipitation Measurement (GPM) mission was launched in February 2014. This international mission is led by the National Aeronautics and Space Administration (NASA) and the Japan Aerospace and Exploration Agency (JAXA), as a successor to the Tropical Rainfall Measuring Mission (TRMM), to continue and improve satellite-based rainfall and snowfall observations on a global scale . The GPM mission consists of a core observatory satellite and a constellation of partner satellites to collect information from as many passive microwave (PMW) and infrared (IR) satellite platforms as available. Such a merged PMW–IR approach can mutually enhance the respective merits of individual PMW or IR satellite-based rainfall estimates; that is, IR satellite estimates can be adjusted with the greater accuracy of PMW data, and, conversely, PMW satellite estimates can be interpolated along cloud movements obtained by the high sampling rate of IR data .

Once observation data are received from the PMW and IR platforms, they are combined into half-hourly gridded fields through the Integrated Multi-satellite Retrievals for GPM (IMERG) system . The IMERG system mainly comprises the following rainfall retrieval algorithms: the Climate Prediction Center Morphing-Kalman Filter (CMORPH-KF) , the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks-Cloud Classification System (PERSIANN-CCS) , and the TRMM Multi-Satellite Precipitation Analysis (TMPA) . Processed differently based on user requirements in terms of data latency and accuracy (see Sect.  for details), the IMERG computes Early, Late, and Final runs (hereafter IMERG-E, IMERG-L, and IMERG-F runs).

Since the first release of IMERG-F data in April 2014, extensive studies have been devoted to the evaluation of the IMERG rainfall estimates compared to ground observations such as radars and gauges, or to other existing satellite rainfall data e.g.. For instance, demonstrated, through an intercomparison study between the data using a hydrological model, that the IMERG products can adequately substitute TMPA products, both statistically and hydrologically. Furthermore, presented a new validation approach for tracing rainfall errors to individual platforms or techniques within the IMERG system using ancillary variables provided in the products. Such analyses can provide useful information, not only for further improvements in processes of satellite rainfall retrieval but also for users in many relevant applications, from hydrological modeling and hazard studies to climate simulations .

In this study, we evaluate and compare the rainfall data generated by all three IMERG runs using rain-gauge-based gridded data from the WegenerNet Feldbach region (WEGN) high-density climate station network in southeastern Austria . Through this approach, the study aims to rigorously test the performance of IMERG runs and to explore differences between the data. The comparison is conducted over two 0.1${}^{\circ }$ $\times$ 0.1${}^{\circ }$ IMERG grid boxes, which are fully covered by 40 and 39 WEGN stations, respectively. We investigate the data during April–October in the years of 2014–2015, the first two years after the launch of the GPM Core Observatory. While IMERG-F data are available from April 2014, IMERG-E and IMERG-L data are only available from April 2015 at the time of writing, so the evaluation of these two runs is restricted to 2015. Note that all the IMERG data will eventually be retrospectively processed to the start of the TRMM era.

Even though gauge data are considered to be ground reference in many existing validation studies, it is acknowledged that gauge measurements are also subject to uncertainties in terms of areal representativeness, owing to a limitation in spatial coverage . Fortunately, this point is of much less concern for the WEGN data. Around 40 gauges in one IMERG grid ensure a high reliability of data within the domain area, considering that a much smaller number of gauges ranging from 5 to 15 gauges per 2.5${}^{\circ }$ $\times$ 2.5${}^{\circ }$ grid cell, depending on the study , has been suggested to guarantee a monthly error of less than 10 %. The variance reduction factor (VRF) , examined using half-hourly WEGN data from the 40 gauges grid box, is about 0.02 for 10 gauges (average of 40 random combinations), and little or no improvement is observed in the VRF beyond 10 gauges. Another concern is that the use of tipping-bucket gauges, as employed in the WEGN network, is associated with systematic errors caused by various factors such as wind speed and rainfall intensity . To this end, the WEGN data are adjusted by a correction factor described by , who found that WEGN tends to underestimate rainfall by about 10 % compared to reference gauges.

The paper is organized as follows. Following this introduction, Sect.  further introduces IMERG and WEGN data and Sect.  describes the methodologies adopted for the assessment of IMERG estimates. The results are detailed in Sect. , in terms of statistical evaluation and analysis of example rainfall events. Section  contains concluding remarks and plans for future studies.

Data

GPM IMERG satellite rainfall estimates

IMERG version 3 (V03) level 3 products are used in this study. The level 3 products include gridded rainfall and snowfall data, with 0.1${}^{\circ }$ $\times$ 0.1${}^{\circ }$ spatial resolution and 30 min temporal resolution, generated from combining PMW and IR data of the GPM constellation satellites, and calibrated by gauge analysis of the Global Precipitation Climatology Centre (GPCC) . The IMERG processing steps include (1) the CMORPH-KF for quality-weighted time interpolation (“morphing”) of PMW estimates following cloud motion vectors , (2) the PERSIANN-CCS for retrieving PMW-calibrated IR estimates , and (3) the TMPA for inter-satellite calibration and monthly gauge adjustment . A more complete data and algorithm description can be found in .

The IMERG system is run twice in near-real time (NRT), first to produce IMERG-E data about 6 h after nominal observation time for users who need a quick answer related to potential flood or landslide warnings, and second to produce IMERG-L data with approximately 18 h latency for users working in agricultural forecasting or drought monitoring. Once the monthly gauge analysis is received, the final IMERG cycle is run to create the IMERG-F data approximately 3 months after the observation month. Note that both IMERG-E and IMERG-L runs only use some of the IMERG processing steps. For instance, instantaneous PMW rainfall estimates are only propagated forward in time by the morphing scheme of the IMERG-E run, whereas both forward and backward morphing schemes are used in IMERG-L and IMERG-F runs. In this way, IMERG-L and IMERG-F runs are expected to better describe changes in the intensity and shape of rainfall features. For bias adjustment, the IMERG NRT runs use climatological gauge data, while the IMERG-F run ingests monthly GPCC gauge analyses, so the IMERG-F estimates are supposed to be the most accurate and reliable . In this study, we use the calibrated estimates (precipitationCal) for all IMERG runs.

IMERG version 4 (V04) products have recently been started to be released, but the new data do not yet cover the time ranges used in this study at the time of writing (as of April 2017). However, the different version should not lead to significant changes in our conclusions, since the main aim of this study is to evaluate the three different IMERG runs relative to each other. Most of the changes in V04 are applied to all three IMERG runs , so any improvements to the IMERG-F run should also result in a similar improvement to IMERG-E and IMERG-L runs. Furthermore, it is likely that the algorithmic and data differences between the runs (e.g., backward morphing and gauge adjustment) have a stronger influence than any differences between the versions.

WEGN gridded rain gauge data

The WEGN is a high-resolution network for weather and climate study and monitoring purposes, located in the Feldbach region, southeastern Austria . The region is part of the southeastern Alpine foreland, characterized by the river Raab valley and a moderate hilly landscape, with altitudes ranging from 260 to 600 m. The network comprises 153 weather stations in an area of about 300 km${}^{\mathrm{2}}$ (i.e., about one station per 2 km${}^{\mathrm{2}}$), collecting rainfall measurement data every 5 min (Fig. ). A total of 151 stations employ tipping-bucket gauges for rainfall measurements, and each gauge was equipped with one of three different sensors during the study period . Meanwhile, since a major sensor replacement in 2016, all WEGN tipping-bucket gauges have employed the same type of sensor .

WEGN climate station network in the Feldbach region (black square), southeastern Austria. The inset plot shows an enlarged view of the number of WEGN rain gauges that are located within GPM IMERG 0.1${}^{\circ }$ $\times$ 0.1${}^{\circ }$ grid cells. The two red-framed grid boxes (within 46.9–47.0${}^{\circ }$ N, 15.8–16.0${}^{\circ }$ E), containing 40 and 39 WEGN gauges each, are selected for the study; for brevity, these are also termed “Grid 15.85” and “Grid 15.95” based on mid-longitude.

[Figure omitted. See PDF]

Once the WEGN processing system receives “Level 0” raw observations, with a latency of 1–1.5 h, the Quality Control System produces “Level 1” station-level data. Then, only best quality Level 1 data are chosen to transfer into the Data Product Generator (DPG), and the DPG generates the general user data products, “Level 2” station time series, as well as 200 m $\times$ 200 m gridded data by an inverse-distance-weighted interpolated method; all missing and non-best Level 1 data are filled in by temporal and spatial interpolation as part of the DPG processing. All data products are available online at the WEGN web portal within 2 h latency. Since recently, based on the findings of , the Level 2 processing has started to apply a bias correction factor for part of the rain data. More information on the WEGN data processing system and data products can be found in and .

For the statistical comparisons and the study on rainfall events reported in Sect.  and , half-hourly WEGN gridded rainfall data are used, which are generated by summing up the basic (5 min) gridded data, for direct comparison with IMERG rainfall estimates on a WEGN grid points average to IMERG grid box basis. For Sect. , on interpreting temporal characteristics of the satellite estimates, WEGN gridded data with 5 min native resolution are used. For computing the area-averaged WEGN rainfall for each IMERG grid box, we simply take the arithmetic mean of all WEGN grid points that lie within the grid box.

Furthermore, we use a threshold value of 0.05 mm 30 min${}^{-\mathrm{1}}$ to define rain/no rain for removing false alarms in half-hourly data. Figure  shows that the number of WEGN half-hourly rainfall values retained after such threshold-clipping is only significantly reduced for a small number of gauges (fewer than about 30 out of 150 gauges) and that more than about 65 gauges are not affected at all. This suggests that the chosen threshold is reasonable, leaving a high amount of reliable half-hourly data, and that the WEGN half-hourly data exceeding 0.05 mm are very unlikely to be false alarms from the gauges' technical limitations. We also note that WEGN is not a member of the GPCC network, so the WEGN gauge data are independent of the IMERG gauge adjustment process.

WEGN network-averaged half-hourly rainfall amounts as function of the number of WEGN gauges detecting rain (gauges with $\ge$ 0.1 mm rainfall are counted). The rain amounts for half-hourly time intervals over the study period (April–October in 2014–2015) are shown, to check the reasonability of a rain/no-rain threshold for the IMERG data evaluation; 0.05 mm is used for the study. The inset shows the full range of rainfall data.

[Figure omitted. See PDF]

Approach

We assess the performance of IMERG runs using both statistical and graphical methods. After inspecting some basic time series differences, we compare probability density functions (PDFs) and cumulative distribution functions (CDFs) of half-hourly IMERG estimates and WEGN data in terms of their distribution as function of rain rate. The PDF of rain occurrence (PDF${}_{\mathrm{c}}$) describes the percentages of rain occurrence across the predefined bins. Conversely, the CDF of rain volume (CDF${}_{\mathrm{v}}$) indicates the relative contribution of rain rate in each bin to the total rain volume . The PDFs and CDFs are computed over a binning range up to 30 mm, with a 0.5 mm bin width. We also use scatter plots to visually evaluate how IMERG estimates are distributed compared to the WEGN data.

Description of the data: mean, standard deviation (SD), maximum, number of rainfall amount values ($\ge$ 0.05 mm 30 min${}^{-\mathrm{1}}$), and percentage of no-rain data ($<$ 0.05 mm 30 min${}^{-\mathrm{1}}$). Half-hourly rainfall data for the period of April to October in 2014 and 2015 are used. IMERG-L and IMERG-E data are available from April 2015, i.e., used for the second year only.

	Mean (mm)	SD (mm)	Max (mm)	Number of rain data	Percentage of no rain (%)
WEGN	0.82	1.29	15.26	3353	91.8
IMERG-F	1.23	2.02	25.61	2671	93.5
IMERG-L	1.33	2.82	23.73	1335	93.5
IMERG-E	1.36	2.89	25.67	1248	93.9

Accumulated 24 h rainfall (mm day${}^{-\mathrm{1}}$) over the study periods in 2014 (a) and 2015 (b), obtained from WEGN (top) and IMERG runs (middle), and differences between IMERG and WEGN (bottom). Regarding IMERG in the 2014 period, only IMERG-F data are available.

[Figure omitted. See PDF]

In addition, we adopt widely used statistics and contingency indices, including relative bias (RB), mean absolute error (MAE), root mean squared error (RMSE), Pearson correlation coefficient ($r$), Spearman's rank correlation coefficient ($\mathit{\rho }$), and probability of detection (POD), for quantifying differences in performance between the IMERG runs. These are used with definitions as follows. $$\begin{array}{}& {\displaystyle }& {\displaystyle }\mathrm{RB}={\displaystyle \frac{{\sum }_{i=\mathrm{1}}^n(I_i-W_i)}{{\sum }_{i=\mathrm{1}}^n{W}_i}},& {\displaystyle }& {\displaystyle }\mathrm{MAE}={\displaystyle \frac{{\sum }_{i=\mathrm{1}}^n\left|I_i,-,W_i\right|}{n}},\mathrm{and}& {\displaystyle }& {\displaystyle }\mathrm{RMSE}=\sqrt{{\displaystyle \frac{{\sum }_{i=\mathrm{1}}^n(I_i-W_i{)}^{\mathrm{2}}}{n}}},\end{array}$$ where $I_i$ and $W_i$ are respective rain rate values provided by IMERG estimates and WEGN data for a single grid box, at the $i$th time step, with $n$ pairs of data. Related to the latter two, normalized MAE and RMSE (NMAE and NRMSE) are also used, which are computed as relative values of MAE and RMSE with respect to the mean of the WEGN data. $$r={\displaystyle \frac{\mathrm{cov}(I,W)}{\sqrt{\mathrm{var}\left(I\right)}\cdot \sqrt{\mathrm{var}\left(W\right)}}},$$ where cov$(X,Y)$ is the covariance between $X$ and $Y$ values, and var$\left(X\right)$ is the variance of $X$, $$\mathit{\rho }=\mathrm{1}-{\displaystyle \frac{\mathrm{6}{\sum }_{j=\mathrm{1}}^n\left({\mathrm{rank}}_j\right(I)-{\mathrm{rank}}_j(W){)}^{\mathrm{2}}}{n(n^{\mathrm{2}}-\mathrm{1})}},$$ where rank${}_j\left(X\right)$ means the rank position of $X$, and $$\mathrm{POD}={\displaystyle \frac{\mathrm{hits}}{\mathrm{hits}+\mathrm{misses}}},$$ where “hits” means that both IMERG and WEGN data recorded rainfall ($\ge$ 0.05 mm 30 min${}^{-\mathrm{1}}$), and “misses” refers to the rainfall occurrence identified by WEGN data but missed by IMERG data. The POD ranges from 0 to 1 with a perfect score of 1 (0 in the case of misses).

Furthermore, we select two example rainfall events for case-based inspections of spatial patterns and time series, in order to visually explore some pronounced discrepancies, especially of IMERG-E and IMERG-L estimates, compared to WEGN data. The half-hourly WEGN gridded data from the whole network domain and corresponding time series of both IMERG estimates and WEGN data are used in this evaluation, including the consideration of data sources, i.e., PMW or IR observations.

Lastly, we employ the method of to provide an interpretation of IMERG rainfall estimates in terms of gauge accumulation time, $\mathrm{\Delta }$, and the offset, $\mathit{\delta }$. WEGN 5 min gridded data are used as the basis for the purpose of test-integrating these gauge data in the range between 5 and 100 min. The offset means the time from which the accumulation of gauge data is started and is considered to account for time differences between instantaneous satellite estimates and actual rainfall on the ground surface. Consequently, we can reveal the combination of $\mathrm{\Delta }$ and $\mathit{\delta }$ which leads to a minimum RMSE and interpret it as the temporal resolution of the IMERG rainfall estimates.

When only the pairs for which both IMERG and WEGN data exceeding the threshold value are investigated, we generally classify the entire data into low rain intensities ($\le$ 80th percentile) versus high rain intensities ($>$ 80th percentile), according to the percentiles of WEGN rain rates, and also into warm season (April, May, and October) versus hot season (June–September), following the approach of . According to temperature measurements collected by WEGN, the average 2 m air temperature of the study period (2014–2015) was 12.2 ${}^{\circ }$C in the warm season and 18.6 ${}^{\circ }$C in the hot season. We did not use data from the cold season, November to March, in order to guarantee the robustness of the WEGN data as ground reference, since most WEGN gauges are not heated and therefore do not capture snowfall events accurately .

Occurrence probability density function of rain rates (PDF${}_{\mathrm{c}}$, dashed) and cumulative distribution functions of rain volume (CDF${}_{\mathrm{v}}$, solid) for WEGN (gray) and IMERG (red). Comparisons are shown for IMERG-F over the full 2014–2015 period (a) and separately for IMERG-F (b), IMERG-L (c), and IMERG-E (d) over the 2015 period.

[Figure omitted. See PDF]

Same layout per panel as Fig. , but restricted to the data pairs for which IMERG and WEGN both detected rainfall ($\ge$ 0.05 mm 30 min${}^{-\mathrm{1}}$). Comparisons are shown for IMERG-F (a–e), IMERG-L (f–j), and IMERG-E (k–o), using the entire data (a, f, k), the data divided into low and high rain intensities (b–c, g–h, l–m) based on the 80th percentile of WEGN data, and the data divided into two different seasons (d–e, i–j, n–o), warm (April, May, October) and hot (June to September).

[Figure omitted. See PDF]

Scatter plots of half-hourly rainfall amounts for IMERG-F (a–c), IMERG-L (d–f), and IMERG-E (g–i) versus WEGN, for the entire dataset (a, d, g), warm season only (b, e, h), and hot season only (c, f, i). Overplotted are, for nine bins across the range with at least 30 IMERG-WEGN data pairs in each bin, the 50th percentile (black symbols and line) and the 25th and 75th percentiles (gray symbols and lines) of the IMERG estimates. Total number of data pairs ($n$) is indicated in the upper left of each panel. Note that a log-log scale is used.

[Figure omitted. See PDF]

Results

Statistical evaluation of IMERG rainfall estimates

Basic statistics of IMERG estimates and WEGN data are summarized in Table . All three IMERG estimates have a higher value for mean and maximum rain rates, and for standard deviation, compared to the rain rates shown by WEGN data. The percentage of no rain is also slightly larger in IMERG data, which is very likely related to limitations of satellite observations in detecting very low rain intensities . Figure  shows the 24 h accumulated rainfall time series comparison (0.2 mm threshold is applied for the daily amounts). IMERG estimates and WEGN data show good overall agreement on the occurrence of most daily rainfall events at IMERG grid scale, although IMERG tends to overestimate high rain rates.

Figure  shows PDF${}_{\mathrm{c}}$ and CDF${}_{\mathrm{v}}$ of IMERG estimates versus those of WEGN data. The IMERG estimates are in good agreement with the WEGN data in terms of rain occurrence except for low rain rates ($<$ 0.5 mm 30 min${}^{-\mathrm{1}}$). However, IMERG shows high rain rates exceeding the maximum value of WEGN data (15.3 mm 30 min${}^{-\mathrm{1}}$) and consequently yields relatively large differences in CDF${}_{\mathrm{v}}$ compared to WEGN data for the moderate to high rain rates. More specifically, rain rates less than 15 mm 30 min${}^{-\mathrm{1}}$ contribute to essentially 100 % of the total rain volume for WEGN data, while about 95 % for IMERG-F, and only about 75 % for IMERG NRT estimates. This shows that the satellite estimates sometimes overestimate rainfall and produce very high values, which have, in spite of their low frequency, a significant impact on the total rain volume.

Figure  is similar to Fig.  but with the data pairs restricted to those whose IMERG and WEGN values are both higher than 0.05 mm 30 min${}^{-\mathrm{1}}$; i.e., both detect rain. Here, we also divided the entire data into low and high rainfall intensities, and into warm and hot seasons, as described in Sect. . Given that the disagreement of low rain rates in PDF${}_{\mathrm{c}}$ reduces (see entire data), it is confirmed that the differences seen in Fig.  are due to the poor sensitivity (misses) of satellites to low rain rates rather than due to some general biases in the estimation. Indeed, we recomputed the POD score, only using values when WEGN is above 0.5 mm 30 min${}^{-\mathrm{1}}$ (i.e., disregarding low rain rates) and found that the POD scores are 0.70, 0.79, and 0.75 for IMERG-E, IMERG-L, and IMERG-F estimates, respectively, while compared to WEGN of the entire range of rain intensities above the 0.05 mm 30 min${}^{-\mathrm{1}}$ threshold, the POD scores are only 0.50, 0.57, and 0.53, respectively.

In the panels of low rain intensities ($<$ 1.2 mm 30 min${}^{-\mathrm{1}}$, or the 80th percentile of WEGN data), IMERG CDF${}_{\mathrm{v}}$ still gets a contribution from rain rates greater than 1.2 mm 30 min${}^{-\mathrm{1}}$, even over 10 mm 30 min${}^{-\mathrm{1}}$, while the corresponding PDF${}_{\mathrm{c}}$ has a fairly good agreement with that of WEGN. This leads us to suspect that such big differences could be associated with a time lag between rain peaks of IMERG estimates and WEGN data, rather than a tendency of the satellite to constantly overestimate rainfall; this will be further investigated and illustrated in Sect. . For the high intensities, the IMERG runs tend to underestimate the rain rates.

Validation statistics comparing the performance of IMERG runs at each grid box.

		RB	MAE (mm)	NMAE	RMSE (mm)	NRMSE	$r$	$\mathit{\rho }$
IMERG-F	Grid 15.85	0.22	1.18	0.97	2.19	1.79	0.34	0.46
	Grid 15.95	0.30	1.16	0.99	2.23	1.91	0.31	0.47
IMERG-L	Grid 15.85	0.33	1.28	1.12	2.74	2.38	0.40	0.42
	Grid 15.95	0.54	1.41	1.29	3.31	3.02	0.30	0.44
IMERG-E	Grid 15.85	0.28	1.27	1.11	2.84	2.47	0.38	0.38
	Grid 15.95	0.67	1.48	1.42	3.31	3.16	0.34	0.38

Furthermore, the CDF${}_{\mathrm{v}}$ of the IMERG NRT estimates does not show physically plausible shapes, e.g., a sudden rise between 10 and 20 mm. As seen in the hot season, the comparison reveals a clear improvement in IMERG rainfall estimates by applying more retrieval or calibration processes to the satellite observations; CDF${}_{\mathrm{v}}$ moves closer to that of WEGN data, and also, the shapes become gradually smoother from IMERG-E via IMERG-L to IMERG-F estimates. In general, it is concluded that IMERG-F estimates have the highest overall accuracy, followed by IMERG-L and IMERG-E estimates.

Figure  shows scatter plots of IMERG estimates versus WEGN data to enable a more quantitative understanding of the discrepancy between the data. Although it is a common practice to conduct regression analysis with scatter plots, we decided not to because highly skewed distributions of rain rates (outliers) seen in the CDF${}_{\mathrm{v}}$ (Fig. ) can strongly affect the results. Therefore, we chose to examine distributions of IMERG estimates over nine predefined rain rate bins (each bin containing at least 30 data pairs); 25th, 50th (median), and 75th percentiles of IMERG estimates are analyzed for each bin, and the collective results are shown in Fig. .

The IMERG runs show better performance (i.e., closer to one-to-one line) in estimating moderate rain rates within about 0.3 to 3 mm 30 min${}^{-\mathrm{1}}$ but have a tendency to overestimate low rain rates and underestimate high rain rates. It is worth noting that the slopes of IMERG-F percentile lines (Fig. , top row) are consistent, with a relatively narrow spread, across the dataset partitioning, indicating that the biases in the IMERG-F estimates are relatively small and uniformly distributed. In contrast, the 75th percentile line of the IMERG NRT estimates is slightly off the 50th and 25th percentile lines, particularly in the hot season, which indicates that the distribution of IMERG NRT rainfall estimates in the bins is skewed toward low values.

Table  provides the statistics metrics computed for each of the two IMERG grid boxes (see Fig. ). All metrics are improved in IMERG-F estimates, except the correlation coefficient ($r$) that may not be a proper metric to evaluate the accuracy of IMERG data due to some large outliers. Indeed, IMERG-F estimates show the highest Spearman's rank correlation coefficient ($\mathit{\rho }$) which is known to be much less sensitive to outliers . A somewhat better performance in Grid 15.85 compared to Grid 15.95 may be attributed to an Austrian national station, which is not part of the WEGN gauges, but located within the WEGN area (over Grid 15.85), of which measurements are integrated into the GPCC gauge product, therefore influencing IMERG over Grid 15.85.

Analysis of example rainfall events

In this subsection, we focus on diagnosing the more detailed behavior of IMERG runs by selecting example rainfall events where the IMERG estimates show distinct differences from the WEGN data. Note that the WEGN can give very accurate information about the domain in terms of spatial and temporal rainfall variability, in spite of potential overall biases of up to about 10 % in the data. Figures  and  show the spatial distribution of two such rainfall events captured by the WEGN network and the corresponding time series of IMERG estimates and WEGN data.

Example rainfall event in the warm season that occurred on 30 May 2015. (a) Spatial rainfall pattern over the WEGN network for consecutive half-hourly periods over the two selected IMERG grid boxes (red outline; see Fig. ). (b) Time series of IMERG-F, IMERG-L, IMERG-E estimates, and WEGN data in each grid box during the rainfall event, with the shaded areas highlighting the two hours illustrated by the map sequence in (a). Solid vertical lines indicate time steps where all three IMERG runs received PMW-based information for the rainfall retrieval at that time step, dotted vertical lines (not applicable for this event but for the event of Fig.  below) mean that either IMERG-E or both IMERG NRT runs did not receive PMW-based information at the time step, and no vertical line (the case for most time steps) implies that none of the IMERG runs received PMW-based information.

[Figure omitted. See PDF]

Same as Fig. , but for an example rainfall event in the hot season that occurred on 8 July 2015.

[Figure omitted. See PDF]

Figure  shows a rainfall event in the warm season, on 30 May 2015. According to the spatial WEGN maps, the rain clouds arrived at Grid 15.85 first (around 21:00 UTC), and then drifted eastwards. Among the IMERG NRT runs, the IMERG-L run is better able to describe this time lag between the two grids. This improvement can be attributed to backward morphing, which is applied in the IMERG-L, but not yet available in IMERG-E. The IMERG-L run captures rainfall withdrawal at Grid 15.95 better as well. Nevertheless, all IMERG runs tend to overestimate rainfall, with a time difference of about 2 h earlier in starting time. This false alarm suggests that the PMW observations (19:30–20:30 UTC) made by the IMERG runs were likely combined with incorrect IR cloud information. However, despite the absence of available PMW observations during the actual rainfall, the overestimation in IMERG-F is much smaller, thanks to the adjustment by the gauge analysis.

Figure  shows a rainfall event in the hot season, on 8 July 2015. Here, again, the onset of rainfall in IMERG estimates is ahead of that of the WEGN data (see the shaded area around 16:00 UTC). It is interesting that IMERG NRT runs describe the first peak (13:30–14:00 UTC) at Grid 15.85 well, albeit with overestimation, but only the IMERG-E run captures the peak at Grid 15.95 with a half-hour time shift. We suspect that the satellite-observed rain was morphed more slowly than the actual cloud movement (from west to east), for example, because the cloud motion vectors derived from IR-based data do not always accurately reflect the actual cloud advection speed . Therefore, the peak still remains at Grid 15.95 in the IMERG-E estimates.

When it comes to the IMERG-L and IMERG-F estimates, we assume that the backward morphing identified the timing of the peak correctly. However, given that the morphing weights are inversely proportional to the time difference between the target data time and the PMW observation (i.e., higher weight is assigned for the time step when IMERG-E depicted the peak), the backward morphing significantly reduced the peak in the IMERG-E run (since it has a higher weight), whereas it only slightly increased the missing peak (since it has a lower weight). This implies a possibility of conflict between the forward and backward morphing that can lead to error in the rainfall estimates.

Contour plots of RMSE (mm 30 min${}^{-\mathrm{1}}$) between IMERG estimates and WEGN data as a function of WEGN gauge rainfall accumulation time and time offset (which means the time from which the WEGN gauge rainfall accumulation is started relative to the IMERG data start time).

[Figure omitted. See PDF]

In Fig. , both IMERG-E and IMERG-L overestimate rainfall during 16:00–22:00 UTC. This can be explained by differences in the number of PMW observations conducted in each IMERG run. The IMERG NRT runs could use four or fewer PMW observations during the period, all of which overestimated the rain rates (no difference in data values between IMERG-E and IMERG-L once the data are collected from the same PMW sensor), resulting in the overestimation after the forward morphing and then even more after the backward morphing. According to , evaporation below cloud base can introduce large positive bias by the CMORPH morphing method during warm and hot seasons.

Conversely, the IMERG-F run received more PMW-based information over the same period (see 18:00–18:30 and 19:30–20:00 UTC) and the monthly gauge analysis. Thus, it shows better performance than the IMERG NRT runs. This demonstrates clearly the value of more PMW-based estimates in the morphing process as well as the ability of gauge adjustment to mitigate systematic biases . From these two case studies, it appears that the gauges provide a greater improvement to IMERG Final estimates. One reason the PMW observations overestimate rainfall is likely the subgrid-scale rainfall variability. For instance, the IMERG runs may use satellite footprints over the northwestern corner of the grid cells, where rain is stronger ($\approx$ 15 mm 30 min${}^{-\mathrm{1}}$ at 16:30–17:00 UTC), for their gridding process.

Evaluation of temporal matching of IMERG estimates

used contour diagrams of RMSE as a function of accumulation time and time offset to interpret TMPA 3-hourly rainfall estimates as a 100 min accumulation starting between 90 and 30 min before the nominal time. Here, we use the same approach to provide an evaluation and interpretation of the temporal characteristics for the IMERG estimates in terms of rain gauge accumulation on ground. The WEGN 5 min gridded data are used and integrated over accumulation times from 5 min (native sampling) to 100 min (twenty 5 min samples) for time offsets from $-$20 to $+$60 min (in 5 min steps). Figure  shows the resulting RMSE of IMERG rainfall estimates versus WEGN data as a function of the gauge accumulation time, $\mathrm{\Delta }$, and the time offset, $\mathit{\delta }$.

The minimum RMSE value for the IMERG-F estimates of the entire dataset (Fig. , top left) occurs at a $\mathrm{\Delta }$ of about 25 min and a $\mathit{\delta }$ of about $+$40 min. This offset of 40 min exceeds the 30 min time resolution of IMERG, which means that the IMERG-F estimates are, on average, displaced by more than one time step. This suggests, for example, that IMERG-F rainfall estimates during 09:00–09:30 UTC can be considered as gauge measurements during 09:40–10:05 UTC. The positive offset is consistent with the early bias in rainfall onset found in Sect. . The hot season shows a shorter offset for the minimum RMSE compared to the warm season (Fig. , top middle and right), which agrees with the results of .

Intercomparing the IMERG products for their common period in 2015 (Fig. , bottom row), it is visible that longer $\mathrm{\Delta }$ values are needed to minimize RMSE of IMERG-E and IMERG-L rainfall estimates, while optimal $\mathit{\delta }$ values are obtained at around $+$20 min for the both datasets. This analysis identifies possible sources of error that should be considered in the context of hydrological applications of IMERG data. For instance, biases (overestimation in this case) in IMERG rainfall estimates will inevitably propagate through hydrologic models, and consequently this would lead to larger errors in runoff. The magnitude of biases can be reduced when IMERG Final estimates are used. Time offset bias, however, remains relatively stable across all three IMERG runs, especially in the warm season. Therefore, comparison or adjustment of IMERG estimates using local ground reference (if available) in terms of biases, not only in amount of rainfall but also in its timing, should be considered as an approach to reach the required level of accuracy in rain data.

In general, the IMERG NRT estimates show higher RMSE values compared to the IMERG-F estimates, as expected. Also, they show relatively indistinct patterns and even multi-minimum RMSE values (in the case of IMERG-E). As such, this approach of interpreting the rainfall estimates may not be sufficiently constrained by the NRT estimates, due to the limited sample size from only 7 months of data and also due to larger errors. More years of data are needed before such an approach can provide a robust interpretation of the NRT estimates.

Conclusions

In this study, we evaluated half-hourly rainfall estimates from the IMERG-E, IMERG-L, and IMERG-F runs using gauge measurement data from the WEGN network in southeastern Austria for the period of April–October in 2014 and 2015. The dense WEGN gauge network provided a unique opportunity for a direct grid-to-grid comparison over two selected IMERG 0.1${}^{\circ }$ $\times$ 0.1${}^{\circ }$ grid boxes. This evaluation work provides valuable insights and input to improve satellite rainfall retrieval processes, to further intercompare data among satellite-based rainfall products, and to achieve a better product quality, in particular of IMERG NRT, for various data applications such as flood and landslide warning or agricultural drought forecasting and monitoring.

First, thorough statistical comparisons between IMERG estimates and WEGN data show the biases of IMERG both in rainfall occurrence and in intensity distributions. Nonetheless, we find that the IMERG-F run considerably outperforms the NRT runs. IMERG-E and IMERG-L runs overestimate low rain rates, leading to large discrepancies in accumulated rainfall, which result in a lower correlation with WEGN data in general. All three IMERG runs tend to underestimate high rain rates.

Second, the study of rainfall events selected to examine large IMERG-WEGN discrepancies reveals specific situations, e.g., a lack of PMW-based observations during short-term rainfall, when the IMERG runs can fail to describe rainfall features even qualitatively. Here, again, we find significantly smaller errors in the IMERG-F estimates, by the monthly gauge correction, compared to the IMERG NRT estimates.

Last, by calculating the RMSE of the half-hourly IMERG estimates compared to the WEGN ground-based rainfall data as a function of gauge accumulation time and time offset, the minimum RMSE found for IMERG-F estimates suggests these can be regarded as a 25 min accumulation with a $+$40 min time offset (preceding the time of the gauge data by this time span). For example, an IMERG-F estimate for 09:00–09:30 UTC can be interpreted as an accumulation over 09:40–10:05 UTC. Again, the results for the IMERG NRT estimates suggest significantly lower confidence, both due to insufficient sample size and larger estimation errors.

Consequently, our analysis across the different runs of IMERG demonstrates the effects of the additional processes on the final rainfall estimates. While the better performance of the IMERG-F run is often attributed to the gauge adjustment procedure , we also identify the advantages of a greater number of PMW-based estimates. Conversely, the inclusion of forward and backward morphing in the IMERG-L run, with sparse PMW observations, provides only marginal benefits over the forward-only morphing in the IMERG-E run. In fact, our case study of example rainfall events illustrates the interesting possibility of cancellation in the backward and forward morphing estimates for the IMERG-L run, resulting in a performance poorer than in the IMERG-E run. These results for the performance of IMERG runs could be representative of other regions under similar conditions (e.g., midlatitude land areas). The study approach is, however, not easily applicable to different precipitation regimes. This is mainly due to the limited availability of independent ground reference data like WEGN. As a result, WEGN offers valuable information about the accuracy of IMERG estimates across its three different runs.

Further studies on detailed links between the errors in the final rainfall estimates and the upstream data sources (i.e., contribution of each PMW/IR sensor to biases in IMERG estimates) or retrieval processes, to alleviate those issues, will contribute to improvements in the performance of the IMERG-L run (e.g., by accounting for time-lagged peaks or improving the cloud motion vectors) and, consequently, the IMERG-F run. Meanwhile, addressing instantaneous satellite estimates made in the IMERG runs will help us to understand overestimation in the PMW estimates themselves.

Our future work on the evaluation of IMERG products will place emphasis on the IMERG-F data, in order to better understand the behavior of rainfall estimates with various conditions, such as different temporal accumulation, varying thresholds, or the inclusion of PMW/IR sources. Using the WEGN high-resolution data, we can also explore rainfall uncertainty and variability at a IMERG subpixel-scale, another intriguing prospect. Additionally, it will be worthwhile to intercompare the version of IMERG-F data used here (V03) with the current version (V04) or the upcoming version (V05) to be released soon for evaluation of improvements of IMERG. IMERG V04 is the first version to use the GPM Core Observatory as a calibrator for the constellation satellite partners so it is expected to provide more consistent quality among the PMW/IR estimates.

WEGN data are available in the WEGN data portal, www.wegenernet.org. IMERG data are provided by the NASA/Goddard Space Flight Center's PMM and PSS teams from http://pmm.nasa.gov/data-access/.

The authors declare that they have no conflict of interest.

Acknowledgements

The study was funded by the Austrian Science Fund (FWF) under research grant W 1256-G15 (Doctoral Programme Climate Change Uncertainties, Thresholds and Coping Strategies). Walter A. Petersen and Jackson Tan acknowledge the NASA GPM/PMM Programs, and Jackson Tan acknowledges funding under the NASA Postdoctoral Program. WegenerNet funding is provided by the Austrian Ministry for Science and Research, the University of Graz, the state of Styria (which also included European Union regional development funds), and city of Graz; detailed information is found at www.wegcenter.at/wegenernet. Edited by: Wouter Buytaert Reviewed by: Remko Uijlenhoet and two anonymous referees