How Reliable Is Haphazard Sampling?

Current audit standards sanction the use of both statistical and nonstatistical sample-selection methods, and suggest that the method chosen should depend on relative cost and effectiveness. Anecdotal evidence and results of a recent survey of practicing auditors indicate that the majority of audit samples are nonstatistical, with haphazard sampling being the method of choice in most circumstances [see "Sampling Practices of Auditors in Public Accounting, Industry, and Government," Accounting Horizons, 16 (2) 2002].

Regardless of which method is used, Statement on Auditing Standards (SAS) 39 indicates that the selection method should be expected to yield a sample that is representative of the population. Similarly, recent guidance on implementing section 404 of the Sarbanes-Oxley Act (SOA) notes that while statistical sampling is not required in these audits, samples testing internal controls should be selected in an unbiased manner (see A Framework for Evaluating Control Exceptions and Deficiencies, version 3, AICPA, December 2004).

To select a sample that satisfies SAS 39's requirement for representativeness, current standards and related guidance indicate that all population elements must have a chance of selection and that due care must be exercised to avoid selection bias. In circumstances where nonstatistical selection methods are used, auditors must select sample items without regard to their size, shape, location, or other physical features. Also, auditors are warned to avoid distorting samples by selecting only unusual or physically small items, or omitting the first or last items in a population (see AICPA Technical Practice Aids, 2002). Auditors are presumed to be exercising appropriate due care and to be capable of selecting representative samples using nonstatistical selection methods.

In the case of haphazard sampling, the selection process is intended to emulate equal probability sampling, with the effect that all population elements have the same chance of selection. More than 40 years ago, however, noted business sampling experts W. Edwards Deming and Herbert Arkin expressed concerns that nonstatistical methods, including haphazard sampling, are susceptible to unintended selection biases (differences between desired and actual selection probabilities). Two recent research studies confirmed that haphazard sampling is susceptible to selection bias and therefore may not yield representative samples.

Research

The first research study to document selection bias in haphazard samples appeared in 2000 in Behavioral Research In Accounting (vol. 12). In...