Listening loops and the adapting auditory brain

Abstract

Analysing complex auditory scenes depends partly on learning the long-term statistical structure of sounds comprising those scenes. One way the listening brain achieves this is by analysing the statistical structure of acoustic environments over multiple time courses. A critical component of this statistical learning in the auditory brain is the interplay between feedforward and feedback pathways—' listening loops'—connecting the inner ear to higher cortical regions and back. These loops are likely important in setting and adjusting the different cadences over which learned listening occurs through adaptive processes that tailor neural responses to sound environments that unfold over seconds, days, development, and the life course. Here, we posit that exploring listening loops at different scales of investigation—from in vivo recording to human assessment—will reveal the fundamental processes that transform hearing into the essential task of listening.

The act of listening

Our brain is continuously interpreting the soundscape, it is listening even when we are not. Listening is essential to understanding. Without listening, sound is meaningless to us—a wash of noise, reflections, and competing sources vying for our attention. Many of our listening environments are challenging—from restaurants to railway stations, we listen in complex, multi-sensory and multi-dimensional spaces. Compared to even the most advanced listening technologies, however, we navigate these spaces with relative ease, and it is not obvious how we do so. We evolved to deal with listening in an embodied manner but our experimental approaches, and often our listening technologies, pay little regard to the immersive and embodied qualities of listening. A reductionist approach to our exploration of the listening brain will limit the development of algorithms, devices, and therapies that seek to establish or re-establish listening—in humans and machines—as an immersive experience.

Here, we posit that advancing our understanding of the listening brain requires a reframing of our investigative neuroscience to include both the multi-layered soundscape with its noisy background as well as its complex foreground. In doing so, we will have to contend with the complexities of an extensive neural circuit and the specific features of the auditory pathway—evident from cochlea to cortex and back—the “listening loops” responsible for setting the cadences of our listening lives (Winer, 2005; Asilador and Llano, 2020). Sensitivity to salient foreground acoustic cues is important for processing speech information, for example, but background features such as multi-talker babble or the flurry of late-arriving reflections from walls and other surfaces in a room also need to be integrated into our listening experience. Exploring how the listening brain parses background features of the soundscape is critical to survival—fight or flight—since this sensitivity to the statistical structure of background sounds may also enhance our capacity to attend to foreground sounds. Here we posit that studying the mechanisms underlying the detection and coding of the background is essential to understand listening (McWalter and McDermott, 2018). What are the statistics of the background that facilitate its detection? How is it coded? What is the role of feedback? And on which time scale? How and when does its coding depend on contextual information (spatial context, movement, visual stimuli)? Learning the longer-term statistical structure of acoustic environments involves an interplay between feedforward and feedback pathways—the listening loops—including to the level of the inner ear, which takes us directly to the issue of how to explore listening through, and in the context of the complex neural circuits that constitutes the auditory brain. Though afferent, or feedforward, pathways in the auditory brain are rightly considered vital at the juncture between hearing and cognition, feedback (efferent) fibres outnumber feedforward in the auditory brain to influence every station in the pathway, including mechanical and neural structures within the middle and inner ear (Saldaña et al., 1996; Terreros and Delano, 2015). The functional understanding of these cortico-subcortical loops lags well behind our knowledge of their anatomy. Overall, it seems reasonable to assume that the act of listening arises from activity generated in a rich subcortical network replete with bilateral and feedback connectivity, and that this activity operates over progressively wider time windows along the ascending pathway (Ding et al., 2016; Kell and McDermott, 2019; Asokan et al., 2021; Henin et al., 2021), with feedback from relatively higher centres in the auditory pathway modulating neural activity at lower centres over potentially progressively longer epochs (Robinson et al., 2016; Figure 1). Understanding the functional role of cortico-subcortical listening loops in the human brain could support the many autonomous listening devices—from hearing aids and cochlear implants to Amazon’s “Alexa”—that currently provide little of the capacity of human listening abilities. Striving for signal fidelity on millisecond, and even sub-millisecond, timescales, they often struggle to perform in even moderately noisy environments, and fail to operate over the multiple, and much slower, cadences of listening that make effective communication possible. The dominance of rapid signal-processing techniques in the development of hearing technologies and therapies, also surfaces in machine-learning and artificial intelligence approaches to listening. Performance remains distinctly subpar but progress on this front will be critical if autonomous listening devices.

FIGURE 1

Figure 1. Schematic representation of major feedforward (black) and feedback (green) pathways between subcortical and cortical auditory structures, from cochlea to primary cortex, projected on a mouse brain (BioRender).

Tools for exploring listening loops

If we are to take advantage of listening loops to explore the timescales over which sensory information is integrated in the auditory brain (Ding et al., 2016) we may need to implement some new tools to do so. One difficulty in studying cortico-subcortical loops has been the sampling and targeting of, not only deep-sitting neurons, but also those specifically involved in the loop. Thanks to the development of genetic tools, combined with the creation of manipulation tools, we can now opto- and chemo-genetically target deep and superficial cells specifically involved in cortico-subcortical interactions in awake rodents (Clayton et al., 2021; Souffi et al., 2021). The use of the newly developed large-scale high-density recording probes (Jun et al., 2017) allows to record activity from deep and superficial neurons simultaneously across structures (Kleinfeld et al., 2019). The advent of brain-imaging techniques, with the potential to sample from wide populations of neurons within and across brain structures (Bathellier et al., 2012; Silva, 2017), has put hearing on a more equal footing to other sensory systems, particularly vision, for which an understanding of cortical structure and function was well advanced through in vivo experimentation (Hübener and Bonhoeffer, 2005). Imaging of the auditory brain has rapidly advanced from employing simple sounds that build on our understanding of sensory reception and the importance of spectral analysis—tonotopy is widely accepted as the primary representation of the cochlea (Marin et al., 2022)—to more-naturalistic listening assessments permitted by advances in audio technologies (Filipchuk et al., 2022). The downside of current brain-imaging techniques, however, is that they still favour a cortico-centric perspective, with some exceptions (Barnstedt et al., 2015), at a time when subcortical structures and efferent pathways are increasingly understood to be critical to the act of listening (Cruces-Solís et al., 2018). In in vivo experimental settings, two-photon imaging is generally confined to the exploration of cortical structures—though this is changing with the implementation of mesoscale imaging techniques. However, access to subcortical structures—some deep within the brainstem—as well investigations of the efferent pathways, remain limited, especially in humans. Further, many practical limitations of imaging arise beyond the inability to access subcortical structures. The (dangerously loud) sounds generated by magnetic resonance imaging (MRI) scanners pose a specific challenge to structural and functional investigations of the listening brain per se, but MRI as well as magnetoencephalography (MEG) are contra-indicated for the use of the very listening devices that might provide powerful insights to hearing and listening in health and disease.

Listening loops and the adapting brain

Exploring the auditory brain in terms of listening loops conditioned for effective sensing and communication with the outside world is, in fact, how the research field is starting to align (Bajo et al., 2010; Robinson et al., 2016; Weible et al., 2020; Yudintsev et al., 2021; Wang et al., 2022), powered by a combination of new technologies applied generally across sensory neuroscience (e.g. Zingg et al., 2017; Williamson and Polley, 2019), and a specific re-imagining of the structure and function of subcortical auditory structures (Xiong et al., 2015; Bidelman et al., 2018; Lohse et al., 2021). Freed from a cortico-centric approach, the concept of listening loops provides the time-dimensional perspective to understanding, or at least exploring, the different cadences of listening (Antunes and Malmierca, 2021; Homma and Bajo, 2021), and connects with the well-developed concept of the predictive brain. Indeed, despite the technical challenges of accessing sub-cortical structures, the concept of listening loops that operate over distinct feedforward and feedback pathways provides an excellent framework in which to investigate fundamental principles of brain processing such as predictive coding that might be applied to other sensory systems, not generally a role the auditory system has performed.

One means by which the temporal dynamics of the listening brain might be investigated, including its capacity for prediction, is by assessing how it adapts over time to enhance the flow of information (Latimer et al., 2019). We can define adaptation to mean changes (usually a reduction) in neural firing in response to sustained stimulation, though definitions of the term are plentiful. From a functional perspective, firing-rate adaptation seems important in the listening brain’s ability to adjust dynamically to the listening environments in response to changes in that environment, or in response to internal changes that alter its overall sensitivity or dynamics. Adaptive coding is a common phenomenon throughout the brain, and a recent review article provides an excellent primer for understanding the different cadences over which adaptation in the auditory brain unfolds, from the range of milliseconds to over the life-course, as well as potential mechanisms by which these cadences are set or arise (Willmore and King, 2022).

Continuous adaptation within listening loops likely sets and adjusts the cadences over which learned listening occurs, tailoring neural responses to sound environments that unfold over seconds, days, development, and the life-course. Exploring these loops in the context of the adapting auditory brain—from single neurons in animal models to human behavioural assessments—will help us understand the immersive quality of listening, as well as advance the many technologies currently available or under development that purport to listen to us.

Data availability statement

The original contributions presented in this study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

Both authors conceptualised, wrote, and edited the manuscript.

Funding

Einstein Foundation (Grant number: EVF-2021-618) funded the research for which LH was the local applicant and DM was the visiting fellow.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Antunes, F. M., and Malmierca, M. S. (2021). Corticothalamic pathways in auditory processing: Recent advances and insights from other sensory systems. Front. Neural Circuits 15:721186. doi: 10.3389/fncir.2021.721186

Asilador, A., and Llano, D. A. (2020). Top-down inference in the auditory system: Potential roles for corticofugal projections. Front. Neural Circuits 14:615259. doi: 10.3389/fncir.2020.615259

Asokan, M. M., Williamson, R. S., Hancock, K. E., and Polley, D. B. (2021). Inverted central auditory hierarchies for encoding local intervals and global temporal patterns. Curr. Biol. 31, 1762–1770.e4. doi: 10.1016/j.cub.2021.01.076

Bajo, V. M., Nodal, F. R., Moore, D. R., and King, A. J. (2010). The descending corticocollicular pathway mediates learning-induced auditory plasticity. Nat. Neurosci. 13, 253–260. doi: 10.1038/nn.2466

Barnstedt, O., Keating, P., Weissenberger, Y., King, A. J., and Dahmen, J. C. (2015). Functional microarchitecture of the mouse dorsal inferior colliculus revealed through in vivo two-photon calcium imaging. J. Neurosci. 35, 10927–10939. doi: 10.1523/JNEUROSCI.0103-15.2015

Bathellier, B., Ushakova, L., and Rumpel, S. (2012). Discrete neocortical dynamics predict behavioral categorization of sounds. Neuron 76, 435–449. doi: 10.1016/j.neuron.2012.07.008

Bidelman, G. M., Davis, M. K., and Pridgen, M. H. (2018). Brainstem-cortical functional connectivity for speech is differentially challenged by noise and reverberation. Hear. Res. 367, 149–160. doi: 10.1016/j.heares.2018.05.018

Clayton, K. K., Williamson, R., Hancock, K., Tasaka, G., Mizrahi, A., Hackett, T., et al. (2021). Auditory corticothalamic neurons are recruited by motor preparatory inputs. Curr. Biol. 31, 310–321.e5. doi: 10.1016/j.cub.2020.10.027

Cruces-Solís, H., Jing, Z., Babaev, O., Rubin, J., Gür, B., Krueger-Burg, D., et al. (2018). Auditory midbrain coding of statistical learning that results from discontinuous sensory stimulation. PLoS Biol. 16:e2005114. doi: 10.1371/journal.pbio.2005114

Ding, N., Melloni, L., Zhang, H., Tian, X., and Poeppel, D. (2016). Cortical tracking of hierarchical linguistic structures in connected speech. Nat. Neurosci. 19, 158–164.

Filipchuk, A., Schwenkgrub, J., Destexhe, A., and Bathellier, B. (2022). Awake perception is associated with dedicated neuronal assemblies in the cerebral cortex. Nat. Neurosci. 25, 1327–1338. doi: 10.1038/s41593-022-01168-5

Henin, S., Turk-Browne, N. B., Friedman, D., Liu, A., Dugan, P., Flinker, A., et al. (2021). Learning hierarchical sequence representations across human cortex and hippocampus. Sci. Adv. 7:eabc4530. doi: 10.1126/sciadv.abc4530

Homma, N. Y., and Bajo, V. M. (2021). Lemniscal corticothalamic feedback in auditory scene analysis. Front. Neurosci. 15:723893. doi: 10.3389/fnins.2021.723893

Hübener, M., and Bonhoeffer, T. (2005). Visual cortex: Two-photon excitement. Curr. Biol. 15, R205–R208.

Jun, J. J., Steinmetz, N. A., Siegle, J. H., Denman, D. J., Bauza, M., Barbarits, B., et al. (2017). Fully integrated silicon probes for high-density recording of neural activity. Nature 551, 232–236.

Kell, A. J. E., and McDermott, J. H. (2019). Invariance to background noise as a signature of non-primary auditory cortex. Nat. Commun. 10:3958. doi: 10.1038/s41467-019-11710-y

Kleinfeld, D., Luan, L., Mitra, P. P., Robinson, J. T., Sarpeshkar, R., Shepard, K., et al. (2019). Can one concurrently record electrical spikes from every neuron in a mammalian brain? Neuron 103, 1005–1015. doi: 10.1016/j.neuron.2019.08.011

Latimer, K. W., Barbera, D., Sokoletsky, M., Awwad, B., Katz, Y., Nelken, I., et al. (2019). Multiple timescales account for adaptive responses across sensory cortices. J. Neurosci. 39, 10019–10033. doi: 10.1523/JNEUROSCI.1642-19.2019

Lohse, M., Dahmen, J. C., Bajo, V. M., and King, A. J. (2021). Subcortical circuits mediate communication between primary sensory cortical areas in mice. Nat. Commun. 12:3916. doi: 10.1038/s41467-021-24200-x

Marin, N., Lobo Cerna, F., and Barral, J. (2022). Signatures of cochlear processing in neuronal coding of auditory information. Mol. Cell. Neurosci. 120:103732. doi: 10.1016/j.mcn.2022.103732

McWalter, R., and McDermott, J. H. (2018). Adaptive and selective time averaging of auditory scenes. Curr. Biol. 28, 1405–1418.e10. doi: 10.1016/j.cub.2018.03.049

Robinson, B. L., Harper, N. S., and McAlpine, D. (2016). Meta-adaptation in the auditory midbrain under cortical influence. Nat. Commun. 7:13442. doi: 10.1038/ncomms13442

Saldaña, E., Feliciano, M., and Mugnaini, E. (1996). Distribution of descending projections from primary auditory neocortex to inferior colliculus mimics the topography of intracollicular projections. J. Comp. Neurol. 371, 15–40. doi: 10.1002/(SICI)1096-9861(19960715)371:1<15::AID-CNE2>3.0.CO;2-O

Silva, A. C. (2017). Anatomical and functional neuroimaging in awake, behaving marmosets. Dev. Neurobiol. 77, 373–389.

Souffi, S., Nodal, F. R., Bajo, V. M., and Edeline, J.-M. (2021). When and how does the auditory cortex influence subcortical auditory structures? New insights about the roles of descending cortical projections. Front. Neurosci. 15:690223. doi: 10.3389/fnins.2021.690223

Terreros, G., and Delano, P. H. (2015). Corticofugal modulation of peripheral auditory responses. Front. Syst. Neurosci. 9:134. doi: 10.3389/fnsys.2015.00134

Wang, X., Zhang, Y., Zhu, L., Bai, S., Li, R., Sun, H., et al. (2022). Selective corticofugal modulation on sound processing in auditory thalamus of awake marmosets. Cereb. Cortex. bhac278. doi: 10.1093/cercor/bhac278

Weible, A. P., Yavorska, I., and Wehr, M. (2020). A cortico-collicular amplification mechanism for gap detection. Cereb. Cortex 30, 3590–3607. doi: 10.1093/cercor/bhz328

Williamson, R. S., and Polley, D. B. (2019). Parallel pathways for sound processing and functional connectivity among layer 5 and 6 auditory corticofugal neurons. Elife 8:e42974. doi: 10.7554/eLife.42974

Willmore, B. D. B., and King, A. J. (2022). Adaptation in auditory processing. Physiol. Rev. 103, 1025–1058. doi: 10.1152/physrev.00011.2022

Winer, J. A. (2005). Decoding the auditory corticofugal systems. Hear. Res. 207, 1–9.

Xiong, X. R., Liang, F., Zingg, B., Ji, X., Ibrahim, L. A., Tao, H. W., et al. (2015). Auditory cortex controls sound-driven innate defense behaviour through corticofugal projections to inferior colliculus. Nat. Commun. 6:7224. doi: 10.1038/ncomms8224

Yudintsev, G., Asilador, A. R., Sons, S., Sekaran, N. V., Coppinger, M., Nair, K., et al. (2021). Evidence for layer-specific connectional heterogeneity in the mouse auditory corticocollicular system. J. Neurosci. 41, 9906–9918. doi: 10.1523/JNEUROSCI.2624-20.2021

Zingg, B., Chou, X., Zhang, Z., Mesik, L., Liang, F., Tao, H. W., et al. (2017). AAV-mediated anterograde transsynaptic tagging: Mapping corticocollicular input-defined neural pathways for defense behaviors. Neuron 93, 33–47. doi: 10.1016/j.neuron.2016.11.045