A comparison of the discrete cosine and wavelet transforms for hydrologic model input data reduction

Abstract

The treatment of input data uncertainty in hydrologic models is of crucial importance in the analysis, diagnosis and detection of model structural errors. Data reduction techniques decrease the dimensionality of input data, thus allowing modern parameter estimation algorithms to more efficiently estimate errors associated with input uncertainty and model structure. The discrete cosine transform (DCT) and discrete wavelet transform (DWT) are used to reduce the dimensionality of observed rainfall time series for the 438 catchments in the Model Parameter Estimation Experiment (MOPEX) data set. The rainfall time signals are then reconstructed and compared to the observed hyetographs using standard simulation performance summary metrics and descriptive statistics. The results convincingly demonstrate that the DWT is superior to the DCT in preserving and characterizing the observed rainfall data records. It is recommended that the DWT be used for model input data reduction in hydrology in preference over the DCT.

Introduction

Rainfall uncertainty is the biggest obstacle hydrologists face in their pursuit of accurate, precise and timely streamflow forecasts . Unfortunately, errors in rainfall time series data may lead to hydrological model parameter estimates that produce adequate streamflow simulations only during the calibration period . This can lead to poor-quality streamflow predictions for independent periods and low confidence in the ability of streamflow forecasts. Consequently, a precise and accurate representation of rainfall uncertainty is paramount for robust hydrological model parameter estimation, streamflow forecasting and quantitative precipitation forecasts (QPFs). and have demonstrated that skill can be added to QPFs by postprocessing with past observations. As such, skill can be added to QPFs, and consequently flood forecasts, through developing a greater understanding of rainfall uncertainty.

The propagation of input errors in rainfall runoff modeling impedes the hydrologic community's ability to validate model structural error. Despite the vast amount of literature on rainfall measurement, estimation, statistical analysis and quality control procedures , a shroud of uncertainty still surrounds how rainfall and its associated uncertainty should be addressed in rainfall runoff modeling. The implementation of uncertainty analysis in many hydrological applications is also often limited by computational power.

Recent advancements in computational power as well as remote sensing have led to considerable improvements in availability and quality of hydrological observations . These improvements can be leveraged to increase the hydrological and flood forecasting knowledge base and consequently provide water policy decision makers and emergency management services with higher-quality information.

The advancement of computational power has also aided the search for hydrological model parameters that optimally simulate hydrological observations. These approaches initially focused on finding only the global optimum values of the parameters for a given objective function . However, in the past two decades, it has been recognized that the uncertainties in model parameters and predictions need to be estimated. Methods that seek to estimate parameter and prediction uncertainty include Bayesian recursive parameter estimation , the limits of acceptability approach , the Bayesian total error analysis (BATEA) framework , the simultaneous optimization and data assimilation (SODA) , the DREAM algorithm and its variations , Bayesian model averaging , the hypothetico-inductive data-based mechanistic modeling framework of and Bayesian data assimilation . It is through the development of these parameter estimation algorithms that hydrologists are able to explore input uncertainty.

and identified the need to represent true catchment rainfall and its associated uncertainty using parameters, both applied a parametric approach to estimating true catchment rainfall and its associated uncertainty using a rainfall multiplier to storm events. The use of a parametric representation of rainfall with an effective sampling algorithm provides the ability to jointly estimate hydrologic model parameter distributions as well as input uncertainty. As in most hydrological problems, there is a lack of sufficient data to obtain a unique solution. However, and found there were sufficient data to estimate both hydrological model parameters and rainfall input. Data reduction transformations offer the potential to reduce the dimensionality of the parameter estimation problem and thus enable a more robust inference. Signal transforms, such as Fourier and wavelet transforms, are examples of data reduction transformations that have been applied in hydrology; however, they have not previously been used to reduce the dimensionality of input data.

Fourier transforms use sinusoidal functions to represent the spectral component of an input signal; thus, a periodic signal could be represented using a smaller number of Fourier coefficients than the number of input data points. A pitfall of the Fourier transform is that it represents the spectral components of a signal, without any indication of the time localization of those specific spectral components. In order to account for this, the windowed Fourier transform (WFT), sometimes referred to as the short-time Fourier transform, segments the signal into discrete time windows before performing the Fourier analysis. A major drawback to this approach is that the uncertainty principle of signal processing imposes a limitation on the time and frequency resolutions that can be obtained for a given signal. As a response to this, produced discrete basis functions with good time and frequency localization. In conjunction with the pyramid algorithm, as described by , this work formed the basis for multi-resolution analysis with the discrete wavelet transform (DWT; ). The DWT decomposes an input signal into high- and low-frequency components.

Wavelet analysis was first introduced to the geophysical sciences by and has been adopted for several different applications. Wavelet analysis has been used to assess the performance of hydrological models for parameter estimation to analyze changes over different time periods for both streamflow and precipitation data . Various spectral methods have also been applied in hydrology, including the application of discrete Fourier transforms to calibrate water and energy balance models and for the calibration of the conceptual rainfall runoff model known as the probability distributed model (PDM) . While wavelet and spectral methods have been applied in the hydrological sciences, to date there have been no instances in which the suitability of different transforms has been compared for hydrological data reduction applications. has pointed out that Fourier transforms and their derivatives are not well suited to reconstruct hydrologic data, which are generated by transient mechanisms. This is due to the Fourier transforms' poor capability to represent sporadic high-frequency events when dimensionally reduced. If model input data reduction techniques are to be accepted by the hydrologic community, it is of critical important that the transform used is able to reconstruct transient events. Through a comparative study, it will be shown that DWTs are a good multi-resolution alternative to the discrete cosine transform (DCT).

Traditionally, transform coefficients are the result of a convolution operation on an input signal. However, the aim of model input data reduction is to estimate these transform coefficients. Hence, they shall be referred to as transform parameters from herein. This paper provides novel theoretical and numerical comparisons of the DCT and DWT in a hydrological context. The ability of both transforms to reproduce key components of hydrological data sets is investigated. The extent to which each transform can reproduce hydrologic data using a decreasing number of parameters will serve as a metric upon which their ability to be used as a tool for model input data reduction for hydrological data will be evaluated. To address the requirements for hydrologic model input data reduction, this paper details (i) theoretical differences between the DCT and DWT, (ii) methodologies to reduce input rainfall to parameters and (iii) an evaluation of the proposed methodologies using several simulation performance summary metrics.

Model input data reduction theory

For this study, model input data reduction theory is introduced using a lumped conceptual watershed model. Consider a nonlinear model, $\mathcal{F}(\cdot )$, which simulates $n$ discharge values, $\widehat{\mathit{Y}}=\mathit{\{}{\widehat{y}}_{\mathrm{1}},\mathrm{\dots },{\widehat{y}}_n\mathit{\}}$, in mm day${}^{-\mathrm{1}}$ according to $$\widehat{\mathbf{Y}}=\mathcal{F}\left(\mathit{\theta },,,\stackrel{\mathrm{̃}}{{\mathit{x}}_{\mathrm{0}}},,,\widehat{\mathit{E}},,,\widehat{\mathit{R}}\right),$$ where the model input arguments are the $\mathrm{1}\times d$ vector $\mathit{\theta }$, with arbitrary model parameter values, the $\mathrm{1}\times m$ vector $\stackrel{\mathrm{̃}}{{\mathit{x}}_{\mathrm{0}}}$, with values of the initial states in millimeters and the $\mathrm{1}\times n$ vectors $\widehat{\mathit{E}}$ $=\mathit{\{}{\widehat{e}}_{\mathrm{1}},\mathrm{\dots },{\widehat{e}}_n\mathit{\}}$ and $\widehat{\mathit{R}}$ $=\mathit{\{}{\widehat{r}}_{\mathrm{1}},\mathrm{\dots },{\widehat{r}}_n\mathit{\}}$ which store the observed values of the potential evapotranspiration (PET) and rainfall in mm day${}^{-\mathrm{1}}$, respectively. Note that $\widehat{\mathit{R}}$ is used to represent rainfall and not precipitation, as snow, hail and other forms of precipitation are not considered. The $\widehat{}$ (hat) symbol is used to denote measured quantities and the $\stackrel{\mathrm{̃}}{}$ (tilde) symbol reflects variables that are either reconstructed or could, in theory, be observed in the field but due to their conceptual nature are difficult to determine accurately.

If the traditional hydrological perspective in which the inputs $\mathit{E}$ and $\mathit{R}$ are considered to be fixed and known quantities is relaxed, and rainfall is now considered unknown, then a new inference problem arises in which the input rainfall is estimated via the treatment of the input rainfall as a series of parameters. Inference problems in which the input is considered unknown can be dealt with using a Bayesian framework. Such inference problems have been considered by and but are outside the scope of this paper. Consequently, for rainfall to be inferred, a suitable parametric representation of rainfall must be determined.

Given a daily rainfall data record with $n$ observations in millimeters, $n$ rainfall parameters could be used to represent the input hyetograph. This approach would be particularly elegant and parsimonious. Yet, for a 10-year record of daily discharge data, the inference problem would grow from $d$ model parameters to roughly $\mathrm{10}\times \mathrm{365}+d=\mathrm{3650}+d$ parameters. These values would need to be estimated from the observed rainfall and discharge data record, respectively. As many hydrological models are already underdetermined, the introduction of additional parameters would make the model even less determinable. Additionally, an excessive amount of CPU time is required to solve for a $\mathrm{3600}+$ dimensional posterior parameter distribution. An alternative approach is therefore necessary.

Sparse transforms convey large amounts of data using fewer parameters than data points in the observed signal. An input rainfall signal can be reduced to sparse transform parameters. Doing so allows multiple rainfall observations to be modified using a single parameter. Some or all of these transform parameters can be altered before the transform is inverted to produce a new input signal for streamflow simulation and posterior analysis. The use of sparse transforms to represent input time series enables input uncertainty to be explored in great detail. The ability of discrete wavelet and Fourier transformations to reduce hydrological input data to a set of parameters for uncertainty estimation is compared using theoretical and analytical methods.

Overview of the DCT and DWT

Wavelet and Fourier transforms are invertible transforms in which a forward convolution operation can be used to decompose a signal into various components. Similarly, a backwards deconvolution operation can be applied to retrieve the original signal. Fourier-based transforms decompose signals into frequency components and are best used for regular time-invariant signals that do not exhibit time-specific information. Alternatively, wavelet-based transforms decompose signals into frequency and time components. The advantage of using wavelet functions to transform data is that time-specific information about when higher frequency components occur can be preserved. To obtain time-specific information, Fourier-based transforms can be applied over pre-specified temporal windows. Yet, this approach is limited by the uncertainty principle of signal processing. The uncertainty principle of signal processing imposes a lower limit on obtainable resolutions in the time–frequency domain such that $${\mathit{\sigma }}_{\mathrm{t}}{\mathit{\sigma }}_{\mathit{\omega }}\ge {\displaystyle \frac{\mathrm{1}}{\mathrm{2}}},$$ where ${\mathit{\sigma }}_{\mathrm{t}}$ (s) and ${\mathit{\sigma }}_{\mathit{\omega }}$ (s${}^{-\mathrm{1}}$) are the respective temporal and frequency widths used in the sparse transform.

Applying the uncertainty principle of signal processing (Eq. ), it is clear that any attempt to narrow the temporal period analyzed to gain increased resolution in the time domain would be met by a widening of the frequency spectrum and consequently a loss of resolution in the frequency domain.

Considering that there is no time–frequency window that is able to obtain limitless resolution in both the time and frequency domains, it is clear that an alternative solution must be found. Wavelet transforms can be used to decompose a signal into different levels that consist of different time and frequency resolution windows. Thus, the wavelet transform is able to be configured to simultaneously obtain high levels of resolution in both the time and frequency domains. For a more detailed discussion on wavelets and sparse transforms, the reader is referred to .

Discrete cosine transform

The DCT is a version of the WFT that has advantageous properties for the field of data compression. Due to the boundary conditions of the cosine function, the DCT is well suited to represent an observed input signal with a minimal number of parameters: in this case, rainfall $\widehat{\mathit{R}}\left(t\right)$. The DCT parameters $\mathit{p}\left(i\right)$ are calculated as $$\mathit{p}\left(i\right)=w\left(i\right)\sum _{t=\mathrm{1}}^n\widehat{\mathit{R}}\left(t\right)\cos \left[{\displaystyle \frac{\mathit{\pi }}{\mathrm{2}n}},(,\mathrm{2},t,-,\mathrm{1},),(,i,-,\mathrm{1},)\right],$$ where $i=\mathrm{1},\mathrm{2},\mathrm{\dots },n$ and $$w\left(i\right)=\left\{\begin{array}{ll}{\displaystyle \frac{\mathrm{1}}{\sqrt{n}}},& i=\mathrm{1}\\ \sqrt{{\displaystyle \frac{\mathrm{2}}{n}}},& \mathrm{2}\mathit{⩽}i\mathit{⩽}n.\end{array}\right.$$ The convolution process can be reversed to reconstruct the observed signal using the inverse transform: $$\stackrel{\mathrm{̃}}{\mathit{R}}\left(t\right)=\sum _{i=\mathrm{1}}^{n}w\left(i\right)\mathit{p}\left(i\right)\cos \left[{\displaystyle \frac{\mathit{\pi }(\mathrm{2}t-\mathrm{1})(i-\mathrm{1})}{\mathrm{2}n}}\right],$$ where $t=\mathrm{1},\mathrm{2},\mathrm{\dots },\mathrm{n}$.

Discrete wavelet transform

Using the pyramid algorithm, depicted in Fig. , first described the decomposition of an input signal into multi-resolution components using high- and low-pass filters. Each stage of decomposition is referred to as a level. An advantage of using wavelets is that decomposition can be performed using a variety of different wavelet families. This allows for signals with differing properties to be analyzed using the same methodology. The most commonly used wavelet family is the Daubechies wavelets . Each wavelet within each family consists of a scaling $h\left(m\right)$ and wavelet $w\left(m\right)$ function, where $m$ denotes the length along the scaling and wavelet function. The scaling and wavelet functions are used in the low- and high-pass filtering sequences, respectively. Whilst there are numerous wavelet families that can be chosen for analysis, this study applies the most commonly used Daubechies wavelets. Depending on the choice of wavelet, stepwise convolutions of the input signal are performed over the filter length $L$. $j_{\max }$ imposes an upper limit on the level of decomposition $j$ that a signal can be decomposed into, where $$j_{\max }=\lfloor {\log }_{\mathrm{2}}\left({\displaystyle \frac{n+L-\mathrm{1}}{\mathrm{2}}}\right)\rfloor ,$$ in which $\lfloor .\rfloor$ is the floor operator. The input signal is then convoluted by being passed through high- and low-pass filters, where

A schematic showing the pyramid algorithm used to decompose and downsample $(\downarrow \mathrm{2})$ an input signal ($\widehat{\mathit{R}}$) into high- and low-frequency components. The input signal is filtered using the high- and low-pass filters described in Eqs. () and () before being downsampled to produce the level 1 high- and low-pass parameters. The low pass parameters are now used as input for the high- and low-pass filters. This process of filtering and downsampling is repeated until the desired level of decomposition is met.

[Figure omitted. See PDF]

$${\mathit{p}}_j^{\mathrm{L}}\left(i\right)=\left\{\begin{array}{ll}\sum _{m=\mathrm{1}}^L\stackrel{\mathrm{̃}}{\mathit{R}}(\mathrm{2}i-m-\mathrm{1})w\left(m\right),& j=\mathrm{1}\\ \sum _{m=\mathrm{1}}^L{\mathit{p}}_{j-\mathrm{1}}^L(\mathrm{2}i-m-\mathrm{1})w\left(m\right),& j>\mathrm{1}\end{array}\right.$$ is the low pass and $${\mathit{p}}_j^{\mathrm{H}}\left(i\right)=\left\{\begin{array}{ll}\sum _{m=\mathrm{1}}^L\stackrel{\mathrm{̃}}{\mathit{R}}(\mathrm{2}i-m-\mathrm{1})h\left(m\right),& j=\mathrm{1}\\ \sum _{m=\mathrm{1}}^L{\mathit{p}}_{j-\mathrm{1}}^L(\mathrm{2}i-m-\mathrm{1})h\left(m\right),& j>\mathrm{1}\end{array}\right.$$ is the high pass, $i=\mathrm{1},\mathrm{\dots },n_{j-\mathrm{1}}+L-\mathrm{1}$ and refers to the $i$th parameter, $j=\mathrm{1},\mathrm{\dots },j_{\max }$ and refers to the $j$th level, $m$ refers to the $m$th filter coefficient. The resultant low-pass ${\mathit{p}}_j^{\mathrm{L}}\left(i\right)$ and high-pass ${\mathit{p}}_j^{\mathrm{H}}\left(i\right)$ parameters are commonly referred to as approximation and detail parameters, respectively. After the input signal is passed through the high- and low-pass filters there is an issue of redundancy that needs to be dealt with. The filters split the input signal into high- and low-frequency components that each contain roughly half the information of the input signal. As the length of each of the resultant approximation and detail parameter series is equivalent to the length of the input signal, each of the parameter series must be downsampled. The process of downsampling removes every other parameter. It is the process of high- and low-pass filtering followed by downsampling that enables the DWT to analyze multi-resolution components of a signal. After downsampling, the length of the resultant approximation and detail parameter series is $${\mathrm{n}}_j=\left\{\begin{array}{ll}\lfloor {\displaystyle \frac{n+L-\mathrm{1}}{\mathrm{2}}}\rfloor ,& j=\mathrm{1}\\ \lfloor {\displaystyle \frac{n_{j-\mathrm{1}}+L-\mathrm{1}}{\mathrm{2}}}\rfloor ,& j>\mathrm{1}\end{array}\right.,$$ where $n_j$ refers to the length of the series at the $j$th level. If further decomposition is required, the downsampled low pass may be fed back into the filters until the resultant parameters can no longer be split any further. An iteration of this process is shown in Fig. . To reverse the decomposition process and reconstruct a signal, upsampling is performed on the parameter series before the lower level parameters are obtained through $$\begin{array}{cc}{\displaystyle }& {\displaystyle }{\mathit{p}}_{j-\mathrm{1}}\left(i\right)=\sum _{m=\lceil i/\mathrm{2}\rceil }^{\lfloor (\mathrm{L}-\mathrm{1}+i)/\mathrm{2}\rfloor }\left({\mathit{p}}_j^{\mathrm{H}},\left(,i,\right),h,(,\mathrm{2},m,-,i,)\right)\left({\mathit{p}}_j,\left(,i,\right),w,(,\mathrm{2},m,-,i,)\right),\\ & {\displaystyle }& {\displaystyle }j>\mathrm{1},\end{array}$$ where $\lceil .\rceil$ is the ceiling operator and the input signal is reconstructed using $$\begin{array}{cc}{\displaystyle }& {\displaystyle }\stackrel{\mathrm{̃}}{\mathit{R}}\left(i\right)=\sum _{m=\lceil i/\mathrm{2}\rceil }^{\lfloor (\mathrm{L}-\mathrm{1}+i)/\mathrm{2}\rfloor }\left({\mathit{p}}_j^{\mathrm{H}},\left(,i,\right),h,(,\mathrm{2},m,-,i,)\right)\left({\mathit{p}}_j,\left(,i,\right),w,(,\mathrm{2},m,-,i,)\right),\\ & {\displaystyle }& {\displaystyle }j=\mathrm{1}.\end{array}$$

Data

This study utilizes data from the Model Parameter Estimation Experiment (MOPEX) data set. The 10 years of rainfall data spanning the 1990s for 438 catchments in the United States of America (USA) are used to compare the suitability of the DWT and DCT to represent rainfall time series. The catchments used in this study were chosen to ensure they had sufficient rain gauge density and represented a range of catchment sizes and climates. Rainfall for the Leaf River catchment (Collins, Mississippi), a catchment that is frequently used for hydrological studies , is used to compare the DWTs' and DCTs' ability to reconstruct high-magnitude rainfall events. A single rainfall product for each catchment is used for analysis at a daily time step. A complete description of the selection process and MOPEX data set is given by . No streamflow data are used in the experiment.

Experiment design

This experiment does not involve the use of any hydrological models. Due to this and the nature of the transforms, there are no calibration and evaluation periods. A major use of both the DWT and DCTs has been in image compression; consequently, the observed input signals were compressed and decompressed using a methodology similar to that used in image compression. In order to determine which transform's parameters are able to effectively store the most hydrological input data, both DWT and DCT parameters will be compressed to varying extents for the MOPEX rainfall time series.

The process undertaken involves a number of steps. Firstly, before any compression is applied, the original rainfall signal for a given catchment is transformed into DCT and DWT parameters using Eqs. () and () and Eqs. () to () for the DCT and DWT, respectively. Secondly, each transform is compressed by iteratively zeroing out parameters that provide a low degree of information; these parameters are those closest to zero. A threshold value $T$ (mm) applies a lower limit for which transform parameters above the threshold are retained. This threshold is iteratively increased until the compressed transform is composed of the desired number of remaining parameters $k$ and percent of original parameters (POP) is met. $$\mathrm{POP}\left(T\right)=\mathrm{100}\times \left({\displaystyle \frac{k}{n}}\right),$$ where $k$ becomes smaller as the threshold $T$ increases and ${lim}_{T\to \mathrm{\infty }}\mathrm{POP}=\mathrm{0}$. The next step is to reconstruct the observed signal from the compressed transform parameters using Eqs. () and () for the DCT and DWT, respectively. After the reconstruction has been performed, a comparison between the reconstructed and observed rainfall can be made. Lastly, this process is iterated for different POPs as well as for each catchment within the data set.

To provide a meaningful comparison between the DCTs' and DWTs' ability to reproduce different rainfall time series with an increasing POP, a number of simulation performance summary metrics are used. Following , a combination of graphical techniques and dimensionless and error index statistics that are widely accepted by the hydrological community were adopted for model evaluation. The Nash–Sutcliffe efficiency (NSE) and the root mean square error (RMSE) to standard deviation ratio (RSR) of the observed input signal $(\mathrm{RSR}=\mathrm{RMSE}/{\mathit{\sigma }}_{\mathrm{obs}})$ are used to compare the performance of the reconstructed rainfall signal with the observed rainfall signal. Once the reconstructed signals are obtained, further comparison with the observed rainfall will be made using the bias summary metric. The variance, kurtosis and skewness of the reconstructed signals will be compared with those of the observed signal. The bias is calculated as ${\sum }_{t=\mathrm{1}}^n\left[\widehat{\mathit{R}}\right(t)-\stackrel{\mathrm{̃}}{\mathit{R}}(t\left)\right]/n$, where $\widehat{\mathit{R}}\left(t\right)$ and $\stackrel{\mathrm{̃}}{\mathit{R}}\left(t\right)$ are the observed and reconstructed rainfall signals, respectively. The reconstructed variance, kurtosis and skewness are all normalized by the observed input signals variance, kurtosis and skewness, respectively. The peak error (PE) is the peak rainfall error over the 10-year period. It is used to compare the reconstructed and observed signals for seasonal and flood forecasting situations. The PE is normalized by the peak height of the observed input signal. Further, the number of rain events missed is computed for each reconstruction by flagging original or reconstructed observations that exhibit no rainfall. Either the absolute difference between the reconstructed and original observation is less than 0.01 or the ratio of the reconstructed and original observation is equal to 0 or larger than 10. Lastly, reconstructed rainfall using the DCT and DWT will be presented for the Leaf River catchment to compare each transform's ability to reconstruct high-magnitude rainfall events.

Results

Figure  shows the relationships between RSR and the number of transform parameters using the DCT and DWT for three different catchments: Arroyo Chico, Skykomish River and Ohio Brush Creek. These catchments represent the smallest, largest and mean rainfall volumes for the MOPEX data set, respectively. It is clear that for all but the highest POP the DWT is able to reconstruct the observed signal with lower RSR than the DCT and that as the rainfall volume increases the RSR decreases. For intermediate POPs, the DWT is able to reconstruct the observed signal with significantly better RSR than the DCT. As the POP approaches both 100 and 0 %, there is little discernible difference between the DCT and DWT reconstructions.

By comparing the reconstructed DWT and DCT signals, using 20 POP and the observed rainfall signal as a reference, a histogram for the NSE is shown for all catchments in Fig. . Each frequency count in the histogram represents a catchment from the MOPEX data set. The reconstructed DWT signals are clearly able to better simulate the observed rainfall signal. All DWT reconstructed rainfall signals obtained a higher NSE than the DCT reconstructed rainfall signals. Table  shows that as the transforms are compressed and fewer parameters are used in the reconstruction, the mean NSE for the DWT stays much closer to the ideal value of 1 than the DCT. Further, the standard deviation of NSE becomes much larger for the DWT.

Empirical plots showing the relationship between RSR and the POP used for reconstructing an input rainfall signal using the DWT and DCT. The three catchments, from the top to the bottom of the figure, represent the smallest, largest and mean rainfall volumes throughout the 1990s for the MOPEX data set.

[Figure omitted. See PDF]

Histogram representing the reconstructed DWT (dark bins) and DCT (clear bins) NSE when compared to the observed rainfall signal. Rainfall is reconstructed after the input signal is compressed to 20 POP. Each frequency count represents a catchment from the MOPEX data set.

[Figure omitted. See PDF]

Figure  compares the RSR for the DCT and DWT using four different POPs. A $\mathrm{1}:\mathrm{1}$ line is included in all subplots and each point represents a catchment from the data set. If the data points fall above the $\mathrm{1}:\mathrm{1}$ line, then for that catchment and POP the DWT is able to reconstruct the input rainfall signal with lower RSR. Again, it is found that DWT is always able to reconstruct the original signal with lower RSR than the DCT reconstructions for all POPs. In a similar fashion to that discussed regarding Fig. , it is observed that as the POP approaches 0 % the difference between the DWT and DCT reconstructions becomes smaller.

The mean and standard deviation (SD) of NSE for the DWT and DCT using a different POP.

	NSE DWT		NSE DCT
POP	Mean	SD	Mean	SD
40 %	0.988	0.007	0.918	0.010
30 %	0.965	0.017	0.844	0.016
20 %	0.905	0.036	0.729	0.025
10 %	0.746	0.070	0.522	0.037

Comparative plots of RSR for the DCT and DWT using a different POP. Each data point represents a catchment.

[Figure omitted. See PDF]

The bias, variance and skewness observed in the reconstructed signals for each catchment are shown in Fig.  for different POPs. The DWT reconstructions are able to maintain a smaller bias than the DCT reconstructions at different POPs for all of the catchments. As the POP decreases, the bias becomes increasingly positive and negative for the DWT and DCT, respectively. The distribution of the bias becomes more dispersed for both the DCT and DWT as the POP decreases. The bias can be seen to be dependent on the transform and POP used as well as the catchment being analyzed. Both the DWT and DCT never reconstruct the observed signal with greater variance than that of the observed rainfall signal. As the POP decreases, the normalized variance for the DCT moves further away from unity than the normalized variance for the DWT. The reduction in normalized variance means that, as the POP decreases, both the DWT and especially the DCT reconstructions will have fewer extreme values when compared to the observed rainfall. The normalized skewness is a measure of symmetry that describes whether or not the reconstructed signal is more positively skewed (more than 1) or less positively skewed (less than 1) than the observed input signal. All of the reconstructed and observed signals had a positive normalized skewness. When compared to the observed signal, the DWT becomes increasingly skewed as the POP is reduced. The opposite of this is observed for the DCT. This indicates that, when compressed, the DWT and DCT will reconstruct the observed rainfall signal with a greater and lower number of values close to zero when compared to the observed signal, respectively. This does not mean that the total volume will be any lower than the total volume of rainfall observed. This is made evident by the low bias observed in Fig. .

Bias and normalized variance and skewness of the reconstructed DWT and DCT signals for each catchment using a different POP.

[Figure omitted. See PDF]

The normalized kurtosis and PE for all catchments using different POPs are shown in Fig. . The measure of kurtosis describes how much the fraction of the distributions' variance is explained by extreme deviations. Consequently, a normalized kurtosis value larger than 1 indicates that the reconstructed signals variance is explained more by extreme deviations than the observed input signal. This is likely to be the result of more rainfall values being reconstructed at the extremities than those of the observed rainfall series. A value smaller than one indicates that the variance is described less by extreme deviations than the observed input signal. Similarly, this is likely to be the result of fewer rainfall values being reconstructed at the extremities than those of the observed rainfall series. It is worth noting that a reconstructed time series can have the same variance yet different kurtosis than the observed rainfall time series. As the POP decreases, the dispersion of normalized kurtosis and skewness increases, and the normalized kurtosis and skewness for the DWT and DCT reconstructions become larger and smaller than unity, respectively. With decreasing POP, the normalized PE for the reconstructed DWT signal remains small and relatively consistent when compared to the normalized PE for the reconstructed DCT signal.

Discussion

Figure  shows that the DWT and DCT are able to reconstruct the observed input signals with good efficiency using 20 POP. However, the DWT consistently outperforms the DCT. Fig.  shows that, as the POP decreases from 100 %, the DWT is able to reconstruct the input signal with increasingly lower RSR than the DCT; the gap in performance is largest for 40 POP. As the POP continues to decrease towards 0 %, the gap in RSR reduces to zero. It is interesting to note that the DWT perfectly reconstructs the observed rainfall signal with as many parameters as there are rainy days, whereas the DCT does not.

Normalized kurtosis of the reconstructed DWT and DCT signals and percentage PE for the reconstructed DWT and DCT signals for each catchment using a different POP.

[Figure omitted. See PDF]

As the bias for the DWT is consistently close to zero, the use of the DWT for rainfall input data reduction is likely to be beneficial for hydrologic studies that have short time steps and involve rainfall as an input. Whilst modification of the DWT parameters may slightly overestimate input rainfall, it is not as significant as the consistent underestimation of input rainfall by the DCT. The diminishing ability of both the DWT and DCT to match the input rainfall signal variance indicates that both transforms smooth out input data towards the mean. This behavior is more significant for the DCT than the DWT. Consequently, when used as a technique for input data reduction, the DWT will reconstruct temporal variances better than the DCT. The increased skewness for the reconstructed DWT signals compared to the observed input signals indicates that there is an increased reconstruction of low-magnitude rainfall events. On the contrary, the decreased normalized skewness for the reconstructed DCT signals indicates that a number of the low-magnitude rainfall events are tending to be reconstructed towards the mean. The kurtosis results shown in Fig.  demonstrate that, when compared to the observed input signal, events of extreme deviation explain more of the variance for the reconstructed DWT and less of the variance for the reconstructed DCT. Consequently, as the nature of the extreme deviations is a critical piece of information, the use of the DCT for model input data reduction for hydrologic studies that have short time steps and involving rainfall as an input is not recommended. It is also seen in Fig.  that the DCT is more likely to miss peak rainfall height information. Consequently, care needs to be taken when choosing a transform when peak height is critical. Further, the DCT should not be used for studies involving flood forecasting situations where the accuracy of peak height is critical.

Comparison of the reconstructed DCT and DWT signal for the Leaf River (Collins) catchment using 20 POP.

[Figure omitted. See PDF]

Panel (a) shows a time series comparison of the reconstructed DCT and DWT signals for the Leaf River (Collins) catchment using 20 POP for a period of 200 days. Panels (b) and (c) are smaller windows of the same time series during both low- and high-rainfall periods.

[Figure omitted. See PDF]

Whilst it is important that rain gauges measure high-magnitude rainfall events with accuracy and precision, it is also important that low-magnitude rainfall events are recorded. Consequently, when evaluating the merits of the DCT and DWT to reconstruct rainfall it would be prudent to analyze the frequency in which each transform is either unable to reconstruct a rainfall event or erroneously constructs a rainfall event. Table  illustrates that, at times, both transforms will either fail to reconstruct a low-magnitude rainfall event or will erroneously construct a rainfall event when there was none observed in the original rainfall time series. In general, the DWT outperforms the DCT. The exception to this is at 10 POP. This is a result of the discrete nature of the DWT analysis function as opposed to the continuous analysis function used in the DCT. As the POP decreases towards zero, both transforms miss more rainfall events.

Due to rapid increases in rainfall intensity, high-magnitude rainfall events tend to have high-frequency components. In Fig. , the smoothing of high-frequency, high-magnitude rainfall events by the DCT is made evident by the lower slope of the linear least squares fit for the DCT reconstruction of Leaf River observed rainfall data when compared to the DWT. This shows that the compressed DWT is able to retain more detail for high-magnitude rainfall events than the DCT. Using 20 POP, 730 DWT parameters are able to reconstruct observed rainfall with an RSR of 0.315, whereas 730 DCT parameters are able to reconstruct observed rainfall with an RSR of 0.540. Figures  and show that the DWT often misses and sometimes smooths out low-magnitude rainfall events; the DCT, however, does reconstruct inaccurate rainfall at these times. Figure  also demonstrates that, at lower POPs, the DCT will smooth out and underestimate high-magnitude events whilst the DWT will maintain accuracy and precision.

The mean and standard deviation (SD) for the number of missed rainfall events for the DWT and DCT using a different number of parameters.

	Number of missed rainfall events
	DWT		DCT
POP	Mean	SD	Mean	SD
40 %	239.004	117.317	587.934	155.375
30 %	398.005	138.793	645.495	159.378
20 %	581.591	145.769	696.288	168.524
10 %	852.340	168.590	748.075	184.910

Conclusions

Succinct descriptions of the DCT and DWT were provided to determine the suitability of each transform to be used as a tool for hydrologic model input data reduction. Due to their different construction, each transform provides different possibilities for use in model input data reduction. Since it is infeasible to estimate all transform parameters, the modeller could choose to estimate high- or low-frequency parameters of the DCT. This would result in minimal control of the temporal component being modified. Due to the multi-level decomposition of an input signal into high- and low-frequency parameters by the DWT, the modeller is able to specify the estimation of both time and frequency components. Hence, portions of the input data record can be targeted for estimation. The use of the DWT as a hydrologic model input data reduction technique allows the modeller more flexible options. A comparison of the DWTs' and DCTs' ability to reconstruct MOPEX rainfall data using standard simulation performance summary metrics, descriptive statistics and peak errors was then made, and it was found that the DWT is most efficient at preserving high-magnitude and transient rainfall events. Thus, it is recommended that the DWT be used as a model input data reduction technique for hydrologic studies that have short time steps and involve rainfall as an input. Considering that the bias for the reconstructed DWT rainfall signal is consistently lower than that of the reconstructed DCT signal and that the skewness, kurtosis and variance are also closest to the input rainfall signal, it is recommended that the DWT also be used as a model input data reduction technique for hydrologic studies that have long time steps with rainfall as an input.

All data were obtained from publicly available data sets (ftp://hydrology.nws.noaa.gov/pub/gcip/mopex/US_Data/; NWS, 2017).

AW conducted the experimental work, contributed towards the theory and wrote the paper. JW and DR assisted in the writing process. VP contributed towards the theory and assisted in the writing process.

Acknowledgements

The authors would like to extend their gratitude to Jasper Vrugt, Hamid Bazargan and the anonymous reviewers for their comments and recommendations. This work was supported by the Multi-modal Australian Sciences Imaging and Visualisation Environment (MASSIVE) (http://www.massive.org.au), a Monash University Engineering Research Living Allowance stipend and a top-up scholarship from the Bushfire & Natural Hazards Cooperative Research Centre. Valentijn Pauwels is funded by ARC grant FT130100545. Edited by: Insa Neuweiler Reviewed by: Hamid Bazargan and two anonymous referees