Content area
Abstract
Pitch-time displays are specialized forms of non-speech audio designed to convey spatial data. By plotting the y-coordinates as pitch and the x-coordinates along the axis of time, scatterplot data can be presented in the auditory modality. A series of three experiments was performed to explore the perceptual and performance equivalence of pitch-time displays with traditional visual presentations of the border contours of shapes. The purpose of Experiment 1 was to explore the perceptual equivalence of pitch-time, visually presented, and imaged presentations of the border contours of U.S. states. The purpose of Experiment 2 was to demonstrate performance equivalence by testing the ability of participants to correctly match cross-modal presentations of stimuli. The purpose of Experiment 3 was to test the effect of practice on performance with pitch-time displays. The results from the first experiment suggest that there is only partial perceptual equivalence between visual and auditory displays. Specifically, similarity judgements for pitch-time displays are driven by the relative steadiness of the upper auditory stream, while visual processing seems to be driven by the overall symmetry and direction of elongation (vertically or horizontally) of the stimuli. The results from Experiment 2 clearly demonstrate that pitch-time displays can and do convey shape information. The results of the third experiment were somewhat surprising. Practice does indeed improve accuracy in cross-modal identification for at least a subset of participants, both for practiced and for newly presented novel stimuli. What is surprising is that practice also increases the dissimilarity between the perceptual strategy employed and those exhibited in the initial auditory and visual conditions from Experiment 1.
