Aurora: Embodied Sound and Multimodal Composition

Abstract

This dissertation explores vibrotactile and vibroacoustic sensation as a mode of communication due to my own experience as a hard-of-hearing individual and musician. As a practicing audio engineer, I recognized a need in the industry for a more direct method of sound reproduction. Studios using near and mid-field audio monitors heavily rely on pure acoustics, and as a result incorporate the room as a function of frequency response which can vary wildly—even from studios within the same facility. Piecing together my experiences as a DJ, audio engineer, and music producer, I began to understand the relationship to music and vibrations, namely how certain synthesis and processing activities within the digital realm can have dramatic effect on the loudspeaker in the physical realm.

Manufacturers of transducers, and audio reproduction equipment have also recognized this relationship, and have created tools and instruments to help composers and end-users feel the unique relationship between sound and vibrations; however, composers are ultimately limited by which reproduction format the end-user chooses to experience the work and the limited adoption of physical sound reproduction tools to allow end-users to experience the strange sensation of feeling sound.

This dissertation details five separate experiments to understand how specialized actuators (device that help perform mechanical action) called tactile transducers function. The experiments explore refining elements a of vibrotactile system to provide a more reliable acoustic and vibrational representation of a sound source. The experiments realized an enclosed purely-vibrotactile device with specifications that are comparable to a traditional loudspeaker design. This realization triggered an entrepreneurial journey that culminates in the creation of startup EDGE Sound Research.

Through this applied experimentation, I create three scales of embodied audio technology, the largest of which is the presentation of Aurora—a three-movement multimodal installation that demonstrates several compositional techniques to include embodied audio. These experiments are to help illustrate a scalable method of sound reproduction that engages the spectrum of hearing and hard-of-hearing participants by enhancing micro-vibrations that naturally exist when reproducing higher-frequency sounds via vibrotactile and vibroacoustic methods.