Beyond the Bleeps & Bloops: Narrative, Multi-Sensory, and Social Approaches to Communicating Data with Sound

Abstract

In today's technological landscape, we are inundated by data about ourselves and our world. Yet, we often simply glance at that data on a screen---like the myriad line and bar graphs on fitness tracker dashboards---and move on without reflection. How might we develop more meaningful interactions with data by building a deeper capacity to think and engage with data using sound and other sensory modalities? In this dissertation, I share several research-through-design projects that develop tools and techniques for engaging with data and sound as creative materials. First, I examine several sonic artifacts---radio pieces that use sound to convey data; a live event where musicologists dissect pop music---to understand practices that can support people as they learn close listening skills. Then, I present sensory translation as a critical technical practice for accessing bodily ways of knowing data. Finally, I build participatory communities around data and sound through personalizable, reconfigurable data interactions I call ``data recipes'' and through design workshops focusing on multi-sensory sketching. Through these projects, I demonstrate how treating data and sound as creative materials and site for exploration can expand our concepts about who can engage with data and how.