Decoding words during sentence production with ECoG reveals syntactic role encoding and structure-dependent temporal dynamics

Abstract

Sentence production is the uniquely human ability to transform complex thoughts into strings of words. Despite the importance of this process, language production research has primarily focused on single words. It remains a largely untested assumption that the principles of word production generalize to more naturalistic utterances like sentences. Here, we investigate this using high-resolution neurosurgical recordings (ECoG) and an overt production experiment where ten patients produced six words in isolation (picture naming) and in sentences (scene description). We trained machine learning classifiers to identify the unique brain activity patterns for each word during picture naming, and used these patterns to decode which words patients were processing while they produced sentences. Our findings confirm that words share cortical representations across tasks, but reveal a division of labor within the language network. In sensorimotor cortex, words were consistently activated in the order in which they were said in the sentence. However, in prefrontal cortex, the order in which words were processed depended on the syntactic structure of the sentence. In non-canonical sentences (passives), we further observed a spatial code for syntactic roles, with subjects selectively encoded in inferior frontal gyrus (IFG) and objects selectively encoded in middle frontal gyrus (MFG). We suggest that these complex dynamics of prefrontal cortex may impose a subtle pressure on language evolution, potentially explaining why nearly all the world’s languages position subjects before objects.

Using electrical recordings taken from the surface of the brain, researchers decode what words neurosurgical patients are saying and show that the brain plans words in a different order than they are ultimately spoken.