Content area
Full text
We present the results of an experiment comparing two popular methods for encoding probability distributions of continuous variables in decision analysis: eliciting values of a variable, X, through comparisons with a fixed probability wheel and eUciting the percentiles of the cumulative distribution, F(X), through comparisons with fixed values of the variable. We show slight but consistent superiority for the fixed variable method along several dimensions such as monotonicity, accuracy, and precision of the estimated fractiles. The fixed variable elicitation method was also slightly faster and preferred by most participants. We discuss the reasons for its superiority and conclude with several recommendations for the practice of probability assessment.
Key words: probability elicitation; decision analysis; continuous distributions; fractile estimation
History: Received on September 23, 2007. Accepted on August 30, 2008, after 2 revisions. Published online in Articles in Advance November 5, 2008.
1. Introduction
The elicitation of a representative probability distribution for a continuous variable is a fundamental step in decision making under uncertainty and has engendered a substantial literature. Several sources focus on the steps needed to elicit a probability distribution and methods to evaluate the quality of the estimates (see, e.g., Spetzler and von Holstein 1975, Wallsten and Budescu 1983, Edwards and von Winterfeldt 1987, von Winterfeldt and Edwards 1987, Merkhofer 1987, O'Hagan et al. 2006). Other sources focus on methods to construct the distribution using the moments of the variable of interest (see, e.g., Moder and Rodgers 1968, Perry and Greig 1975, Smith 1993) or use quantiles and/or moments to construct the distribution with a maximum entropy approach (e.g., Abbas 2002, 2006).
One widely used method for constructing probability distributions assumes a functional form for the probability function and uses the judges' assessments to estimate the parameters of this functional form. For example, Lindley (1987) uses three quantile assessments of a given variable to derive the three parameters of a skew logistic distribution. Other curve-fitting approaches use the assessed data to estimate the two parameters of a Beta distribution, which has found widespread popularity among Bayesian analysts, for its ease of updating with Bernoulli likelihood functions and for the wide variety of shapes it can reproduce (Raiffa and Schlaifer 1961).
Hughes and Madden (2002) review several methods for estimating the parameters of...