Content area
Full text
(ProQuest: ... denotes formulae omitted.)
1. Introduction
The ranked probability score (RPS) is the sum of the squared differences between cumulative forecast probabilities and cumulative observed probabilities, and measures both forecast reliability and resolution (Murphy 1973). The ranked probability skill score (RPSS) compares the RPS of a forecast with some reference forecast such as "climatology" (using past mean climatic values as the forecast), oriented so that RPSS < 0 (RPSS > 0) corresponds to a forecast that is less (more) skillful than climatology.
Categorical forecast probabilities are often estimated from ensembles of numerical model integrations by counting the number of ensemble members in each category. Finite ensemble size introduces sampling error into such probability estimates, and the RPSS of a reliable forecast model with finite ensemble size is an increasing function of ensemble size (Kumar et al. 2001; Tippett et al. 2007). A similar relation exists between correlation and ensemble size (Sardeshmukh et al. 2000). The dependence of RPSS on ensemble size makes it challenging to use RPSS to compare forecast models with different ensemble sizes. For instance, it may be difficult to know whether a forecast system has higher RPSS because it is based on a superior forecast model or because it uses a larger ensemble. This question often arises in the comparison of multimodel and single model forecasts (Hagedorn et al. 2005; Tippett and Barnston 2008). The dependence of RPSS on ensemble size is not a problem when comparing forecast quality. Improved RPSS is associated with improved forecast quality and is desirable whether it results from larger ensemble size or from a better forecast model.
Müller et al. (2005) recently introduced a resampling strategy to estimate the infinite-ensemble RPSS from the finite-ensemble RPSS and called this estimate the "debiased RPSS." Weigel et al. (2007) derived an analytical formula for the debiased RPSS and proved that it is an unbiased estimate of the infinite-ensemble RPSS in the case of uncorrelated ensemble members, that is, forecasts without skill. Here it is proved that the debiased RPSS is an unbiased estimate of the infiniteensemble RPSS for any reliable forecasts. It is shown that over- or underconfident forecasts introduce a dependence of the debiased RPSS on ensemble size. Simplification of the results of Weigel et al. (2007)...