Abstract
Despite long knowing what brain areas support language comprehension, our knowledge of the neural computations that these frontal and temporal regions implement remains limited. One important unresolved question concerns functional differences among the neural populations that comprise the language network. Here we leveraged the high spatiotemporal resolution of human intracranial recordings (n = 22) to examine responses to sentences and linguistically degraded conditions. We discovered three response profiles that differ in their temporal dynamics. These profiles appear to reflect different temporal receptive windows, with average windows of about 1, 4 and 6 words, respectively. Neural populations exhibiting these profiles are interleaved across the language network, which suggests that all language regions have direct access to distinct, multiscale representations of linguistic input—a property that may be critical for the efficiency and robustness of language processing.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
Preprocessed data, all stimuli and statistical results, as well as selected additional analyses are available on OSF at https://osf.io/xfbr8/ (ref. 37). Raw data may be provided upon request to the corresponding authors and institutional approval of a data-sharing agreement.
Code availability
Code used to conduct analyses and generate figures from the preprocessed data is available publicly on GitHub at https://github.com/coltoncasto/ecog_clustering_PUBLIC (ref. 93). The VERA software suite used to perform electrode localization can also be found on GitHub at https://github.com/neurotechcenter/VERA (ref. 82).
References
Fedorenko, E., Hsieh, P. J., Nieto-Castañón, A., Whitfield-Gabrieli, S. & Kanwisher, N. New method for fMRI investigations of language: defining ROIs functionally in individual subjects. J. Neurophysiol. 104, 1177–1194 (2010).
Pallier, C., Devauchelle, A. D. & Dehaene, S. Cortical representation of the constituent structure of sentences. Proc. Natl Acad. Sci. USA 108, 2522–2527 (2011).
Regev, M., Honey, C. J., Simony, E. & Hasson, U. Selective and invariant neural responses to spoken and written narratives. J. Neurosci. 33, 15978–15988 (2013).
Scott, T. L., Gallée, J. & Fedorenko, E. A new fun and robust version of an fMRI localizer for the frontotemporal language system. Cogn. Neurosci. 8, 167–176 (2017).
Diachek, E., Blank, I., Siegelman, M., Affourtit, J. & Fedorenko, E. The domain-general multiple demand (MD) network does not support core aspects of language comprehension: a large-scale fMRI investigation. J. Neurosci. 40, 4536–4550 (2020).
Malik-Moraleda, S. et al. An investigation across 45 languages and 12 language families reveals a universal language network. Nat. Neurosci. 25, 1014–1019 (2022).
Fedorenko, E., Behr, M. K. & Kanwisher, N. Functional specificity for high-level linguistic processing in the human brain. Proc. Natl Acad. Sci. USA 108, 16428–16433 (2011).
Monti, M. M., Parsons, L. M. & Osherson, D. N. Thought beyond language: neural dissociation of algebra and natural language. Psychol. Sci. 23, 914–922 (2012).
Deen, B., Koldewyn, K., Kanwisher, N. & Saxe, R. Functional organization of social perception and cognition in the superior temporal sulcus. Cereb. Cortex 25, 4596–4609 (2015).
Ivanova, A. A. et al. The language network is recruited but not required for nonverbal event semantics. Neurobiol. Lang. 2, 176–201 (2021).
Chen, X. et al. The human language system, including its inferior frontal component in “Broca’s area,” does not support music perception. Cereb. Cortex 33, 7904–7929 (2023).
Fedorenko, E., Ivanova, A. A. & Regev, T. I. The language network as a natural kind within the broader landscape of the human brain. Nat. Rev. Neurosci. 25, 289–312 (2024).
Okada, K. & Hickok, G. Identification of lexical-phonological networks in the superior temporal sulcus using functional magnetic resonance imaging. Neuroreport 17, 1293–1296 (2006).
Graves, W. W., Grabowski, T. J., Mehta, S. & Gupta, P. The left posterior superior temporal gyrus participates specifically in accessing lexical phonology. J. Cogn. Neurosci. 20, 1698–1710 (2008).
DeWitt, I. & Rauschecker, J. P. Phoneme and word recognition in the auditory ventral stream. Proc. Natl Acad. Sci. USA 109, E505–E514 (2012).
Price, C. J., Moore, C. J., Humphreys, G. W. & Wise, R. J. S. Segregating semantic from phonological processes during reading. J. Cogn. Neurosci. 9, 727–733 (1997).
Mesulam, M. M. et al. Words and objects at the tip of the left temporal lobe in primary progressive aphasia. Brain 136, 601–618 (2013).
Friederici, A. D. The brain basis of language processing: from structure to function. Physiol. Rev. 91, 1357–1392 (2011).
Hagoort, P. On Broca, brain, and binding: a new framework. Trends Cogn. Sci. 9, 416–423 (2005).
Grodzinsky, Y. & Santi, A. The battle for Broca’s region. Trends Cogn. Sci. 12, 474–480 (2008).
Matchin, W. & Hickok, G. The cortical organization of syntax. Cereb. Cortex 30, 1481–1498 (2020).
Fedorenko, E., Blank, I. A., Siegelman, M. & Mineroff, Z. Lack of selectivity for syntax relative to word meanings throughout the language network. Cognition 203, 104348 (2020).
Bautista, A. & Wilson, S. M. Neural responses to grammatically and lexically degraded speech. Lang. Cogn. Neurosci. 31, 567–574 (2016).
Anderson, A. J. et al. Deep artificial neural networks reveal a distributed cortical network encoding propositional sentence-level meaning. J. Neurosci. 41, 4100–4119 (2021).
Regev, T. I. et al. High-level language brain regions process sublexical regularities. Cereb. Cortex 34, bhae077 (2024).
Mukamel, R. & Fried, I. Human intracranial recordings and cognitive neuroscience. Annu. Rev. Psychol. 63, 511–537 (2011).
Fedorenko, E. et al. Neural correlate of the construction of sentence meaning. Proc. Natl Acad. Sci. USA 113, E6256–E6262 (2016).
Nelson, M. J. et al. Neurophysiological dynamics of phrase-structure building during sentence processing. Proc. Natl Acad. Sci. USA 114, E3669–E3678 (2017).
Woolnough, O. et al. Spatiotemporally distributed frontotemporal networks for sentence reading. Proc. Natl Acad. Sci. USA 120, e2300252120 (2023).
Desbordes, T. et al. Dimensionality and ramping: signatures of sentence integration in the dynamics of brains and deep language models. J. Neurosci. 43, 5350–5364 (2023).
Goldstein, A. et al. Shared computational principles for language processing in humans and deep language models. Nat. Neurosci. 25, 369–380 (2022).
Lerner, Y., Honey, C. J., Silbert, L. J. & Hasson, U. Topographic mapping of a hierarchy of temporal receptive windows using a narrated story. J. Neurosci. 31, 2906–2915 (2011).
Blank, I. A. & Fedorenko, E. No evidence for differences among language regions in their temporal receptive windows. Neuroimage 219, 116925 (2020).
Jain, S. et al. Interpretable multi-timescale models for predicting fMRI responses to continuous natural speech. In NeurIPS Proc. Advances in Neural Information Processing Systems 33 (NeurIPS 2020) (eds Larochelle, H. et al.) 1–12 (NeurIPS, 2020).
Fedorenko, E., Nieto-Castañon, A. & Kanwisher, N. Lexical and syntactic representations in the brain: an fMRI investigation with multi-voxel pattern analyses. Neuropsychologia 50, 499–513 (2012).
Shain, C. et al. Distributed sensitivity to syntax and semantics throughout the human language network. J. Cogn. Neurosci. 36, 1427–1471 (2024).
Regev, T. I., Casto, C. & Fedorenko, E. Neural populations in the language network differ in the size of their temporal receptive windows. OSF osf.io/xfbr8 (2024).
Stelzer, J., Chen, Y. & Turner, R. Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): random permutations and cluster size control. Neuroimage 65, 69–82 (2013).
Maris, E. & Oostenveld, R. Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 164, 177–190 (2007).
Hasson, U., Yang, E., Vallines, I., Heeger, D. J. & Rubin, N. A hierarchy of temporal receptive windows in human cortex. J. Neurosci. 28, 2539–2550 (2008).
Norman-Haignere, S. V. et al. Multiscale temporal integration organizes hierarchical computation in human auditory cortex. Nat. Hum. Behav. 6, 455–469 (2022).
Overath, T., McDermott, J. H., Zarate, J. M. & Poeppel, D. The cortical analysis of speech-specific temporal structure revealed by responses to sound quilts. Nat. Neurosci. 18, 903–911 (2015).
Keshishian, M. et al. Joint, distributed and hierarchically organized encoding of linguistic features in the human auditory cortex. Nat. Hum. Behav. 7, 740–753 (2023).
Braga, R. M., DiNicola, L. M., Becker, H. C. & Buckner, R. L. Situating the left-lateralized language network in the broader organization of multiple specialized large-scale distributed networks. J. Neurophysiol. 124, 1415–1448 (2020).
Fedorenko, E. & Blank, I. A. Broca’s area is not a natural kind. Trends Cogn. Sci. 24, 270–284 (2020).
Dick, F. et al. Language deficits, localization, and grammar: evidence for a distributive model of language breakdown in aphasic patients and neurologically intact individuals. Psychol. Rev. 108, 759–788 (2001).
Runyan, C. A., Piasini, E., Panzeri, S. & Harvey, C. D. Distinct timescales of population coding across cortex. Nature 548, 92–96 (2017).
Murray, J. D. et al. A hierarchy of intrinsic timescales across primate cortex. Nat. Neurosci. 17, 1661–1663 (2014).
Chien, H. S. & Honey, C. J. Constructing and forgetting temporal context in the human cerebral cortex. Neuron 106, 675–686 (2020).
Jacoby, N. & Fedorenko, E. Discourse-level comprehension engages medial frontal Theory of Mind brain regions even for expository texts. Lang. Cogn. Neurosci. 35, 780–796 (2018).
Caucheteux, C., Gramfort, A. & King, J. R. Evidence of a predictive coding hierarchy in the human brain listening to speech. Nat. Hum. Behav. 7, 430–441 (2023).
Chang, C. H. C., Nastase, S. A. & Hasson, U. Information flow across the cortical timescale hierarchy during narrative construction. Proc. Natl Acad. Sci. USA 119, e2209307119 (2022).
Bozic, M., Tyler, L. K., Ives, D. T., Randall, B. & Marslen-Wilson, W. D. Bihemispheric foundations for human speech comprehension. Proc. Natl Acad. Sci. USA 107, 17439–17444 (2010).
Paulk, A. C. et al. Large-scale neural recordings with single neuron resolution using Neuropixels probes in human cortex. Nat. Neurosci. 25, 252–263 (2022).
Leonard, M. K. et al. Large-scale single-neuron speech sound encoding across the depth of human cortex. Nature 626, 593–602 (2024).
Evans, N. & Levinson, S. C. The myth of language universals: language diversity and its importance for cognitive science. Behav. Brain Sci. 32, 429–448 (2009).
Shannon, C. E. Communication in the presence of noise. Proc. IRE 37, 10–21 (1949).
Levy, R. Expectation-based syntactic comprehension. Cognition 106, 1126–1177 (2008).
Levy, R. A noisy-channel model of human sentence comprehension under uncertain input. In Proc. 2008 Conference on Empirical Methods in Natural Language Processing (eds Lapata, M. & Ng, H. T.) 234–243 (Association for Computational Linguistics, 2008).
Gibson, E., Bergen, L. & Piantadosi, S. T. Rational integration of noisy evidence and prior semantic expectations in sentence interpretation. Proc. Natl Acad. Sci. USA 110, 8051–8056 (2013).
Keshev, M. & Meltzer-Asscher, A. Noisy is better than rare: comprehenders compromise subject–verb agreement to form more probable linguistic structures. Cogn. Psychol. 124, 101359 (2021).
Gibson, E. et al. How efficiency shapes human language. Trends Cogn. Sci. 23, 389–407 (2019).
Tuckute, G., Kanwisher, N. & Fedorenko, E. Language in brains, minds, and machines. Annu. Rev. Neurosci. https://doi.org/10.1146/annurev-neuro-120623-101142 (2024).
Norman-Haignere, S., Kanwisher, N. G. & McDermott, J. H. Distinct cortical pathways for music and speech revealed by hypothesis-free voxel decomposition. Neuron 88, 1281–1296 (2015).
Baker, C. I. et al. Visual word processing and experiential origins of functional selectivity in human extrastriate cortex. Proc. Natl Acad. Sci. USA 104, 9087–9092 (2007).
Buckner, R. L. & DiNicola, L. M. The brain’s default network: updated anatomy, physiology and evolving insights. Nat. Rev. Neurosci. 20, 593–608 (2019).
Saxe, R., Brett, M. & Kanwisher, N. Divide and conquer: a defense of functional localizers. Neuroimage 30, 1088–1096 (2006).
Baldassano, C. et al. Discovering event structure in continuous narrative perception and memory. Neuron 95, 709–721 (2017).
Wilson, S. M. et al. Recovery from aphasia in the first year after stroke. Brain 146, 1021–1039 (2023).
Piantadosi, S. T., Tily, H. & Gibson, E. Word lengths are optimized for efficient communication. Proc. Natl Acad. Sci. USA 108, 3526–3529 (2011).
Shain, C., Blank, I. A., Fedorenko, E., Gibson, E. & Schuler, W. Robust effects of working memory demand during naturalistic language comprehension in language-selective cortex. J. Neurosci. 42, 7412–7430 (2022).
Schrimpf, M. et al. The neural architecture of language: integrative modeling converges on predictive processing. Proc. Natl Acad. Sci. USA 118, e2105646118 (2021).
Tuckute, G. et al. Driving and suppressing the human language network using large language models. Nat. Hum. Behav. 8, 544–561 (2024).
Mollica, F. & Piantadosi, S. T. Humans store about 1.5 megabytes of information during language acquisition. R. Soc. Open Sci. 6, 181393 (2019).
Skrill, D. & Norman-Haignere, S. V. Large language models transition from integrating across position-yoked, exponential windows to structure-yoked, power-law windows. Adv. Neural Inf. Process. Syst. 36, 638–654 (2023).
Giglio, L., Ostarek, M., Weber, K. & Hagoort, P. Commonalities and asymmetries in the neurobiological infrastructure for language production and comprehension. Cereb. Cortex 32, 1405–1418 (2022).
Hu, J. et al. Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production. Cereb. Cortex 33, 4384–4404 (2023).
Lee, E. K., Brown-Schmidt, S. & Watson, D. G. Ways of looking ahead: hierarchical planning in language production. Cognition 129, 544–562 (2013).
Wechsler, D. Wechsler abbreviated scale of intelligence (WASI) [Database record]. APA PsycTests https://psycnet.apa.org/doi/10.1037/t15170-000 (APA PsycNet, 1999).
Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N. & Wolpaw, J. R. BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51, 1034–1043 (2004).
Adamek, M., Swift, J. R. & Brunner, P. VERA - Versatile Electrode Localization Framework. Zenodo https://doi.org/10.5281/zenodo.7486842 (2022).
Adamek, M., Swift, J. R. & Brunner, P. VERA - A Versatile Electrode Localization Framework (Version 1.0.0). GitHub https://github.com/neurotechcenter/VERA (2022).
Avants, B. B., Epstein, C. L., Grossman, M. & Gee, J. C. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12, 26–41 (2008).
Janca, R. et al. Detection of interictal epileptiform discharges using signal envelope distribution modelling: application to epileptic and non-epileptic intracranial recordings. Brain Topogr. 28, 172–183 (2015).
Dichter, B. K., Breshears, J. D., Leonard, M. K. & Chang, E. F. The control of vocal pitch in human laryngeal motor cortex. Cell 174, 21–31 (2018).
Ray, S., Crone, N. E., Niebur, E., Franaszczuk, P. J. & Hsiao, S. S. Neural correlates of high-gamma oscillations (60–200 Hz) in macaque local field potentials and their potential implications in electrocorticography. J. Neurosci. 28, 11526–11536 (2008).
Lipkin, B. et al. Probabilistic atlas for the language network based on precision fMRI data from >800 individuals. Sci. Data 9, 529 (2022).
Kučera, H. Computational Analysis of Present-day American English (Univ. Pr. of New England, 1967).
Kaufman, L. & Rousseeuw, P. J. in Finding Groups in Data: An Introduction to Cluster Analysis (eds L. Kaufman, L. & Rousseeuw, P. J.) Ch. 2 (Wiley, 1990).
Rokach, L. & Maimon, O. in The Data Mining and Knowledge Discovery Handbook (eds Maimon, O. & Rokach, L.) 321–352 (Springer, 2005).
Wilkinson, G.N. & Rogers, C.E. Symbolic description of factorial models for analysis of variance. J. R. Stat. Soc., C: Appl.Stat. 22, 392–399 (1973).
Luke, S. G. Evaluating significance in linear mixed-effects models in R. Behav. Res. Methods 49, 1494–1502 (2017).
Regev, T. I. et al. Neural populations in the language network differ in the size of their temporal receptive windows. GitHub https://github.com/coltoncasto/ecog_clustering_PUBLIC (2024).
Acknowledgements
We thank the participants for agreeing to take part in our study, as well as N. Kanwisher, former and current EvLab members, especially C. Shain and A. Ivanova, and the audience at the Neurobiology of Language conference (2022, Philadelphia) for helpful discussions and comments on the analyses and manuscript. T.I.R. was supported by the Zuckerman-CHE STEM Leadership Program and by the Poitras Center for Psychiatric Disorders Research. C.C. was supported by the Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University. A.L.R. was supported by NIH award U01-NS108916. J.T.W. was supported by NIH awards R01-MH120194 and P41-EB018783, and the American Epilepsy Society Research and Training Fellowship for clinicians. P.B. was supported by NIH awards R01-EB026439, U24-NS109103, U01-NS108916, U01-NS128612 and P41-EB018783, the McDonnell Center for Systems Neuroscience, and Fondazione Neurone. E.F. was supported by NIH awards R01-DC016607, R01-DC016950 and U01-NS121471, and research funds from the McGovern Institute for Brain Research, Brain and Cognitive Sciences Department, and the Simons Center for the Social Brain. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Author information
Authors and Affiliations
Contributions
T.I.R. and C.C. equally contributed to study conception and design, data analysis and interpretation of results, and manuscript writing. E.A.H. contributed to data analysis and manuscript editing; M.A. to data collection and analysis; A.L.R., J.T.W. and P.B. to data collection and manuscript editing. E.F. contributed to study conception and design, supervision, interpretation of results and manuscript writing.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Human Behaviour thanks Nima Mesgarani, Jonathan Venezia and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Dataset 1 k-medoids (k = 3) cluster assignments by participant.
Average cluster responses as in Fig. 2e grouped by participant. Shaded areas around the signal reflect a 99% confidence interval over electrodes. The number of electrodes constructing the average (n) is denoted above each signal in parenthesis. Prototypical responses for each of the three clusters were found in nearly all participants individually. However, for participants with only a few electrodes assigned to a given cluster (for example, P5 Cluster 3), the responses were more variable.
Extended Data Fig. 2 Dataset 1 k-medoids clustering with k = 10.
a) Clustering mean electrode responses (S + W + J + N) using k-medoids with k = 10 and a correlation-based distance. Shading of the data matrix reflects normalized high-gamma power (70–150 Hz). b) Electrode responses visualized on their first two principal components, colored by cluster. c) Timecourses of best representative electrodes (‘medoids’) selected by the algorithm from each of the ten clusters. d) Timecourses averaged across all electrodes in each cluster. Shaded areas around the signal reflect a 99% confidence interval over electrodes. Correlation with the k = 3 cluster averages are shown to the right of the timecourses. Many clusters exhibited high correlations with the k = 3 response profiles from Fig. 2.
Extended Data Fig. 3 All Dataset 1 responses.
a-c) All Dataset 1 electrode responses. The timecourses (concatenated across the four conditions, ordered: sentences, word lists, Jabberwocky sentences, non-word lists) of all electrodes in Dataset 1 sorted by their correlation to the cluster medoid (medoid shown at the bottom of each cluster). Colors reflect the reliability of the measured neural signal, computed by correlating responses to odd and even trials (Fig. 1d). The estimated temporal receptive window (TRW) using the toy model from Fig. 4 is displayed to the left, and the participant who contributed the electrode is displayed to the right. There was strong consistency in the responses from individual electrodes within a cluster (with more variability in the less reliable electrodes), and electrodes with responses that were more similar to the cluster medoid tended to be more reliable (more pink). Note that there were two reliable response profiles (relatively pink) that showed a pattern that was distinct from the three prototypical response profiles: One electrode in Cluster 2 (the 10th electrode from the top in panel B) responded only to the onset of the first word/nonword in each trial; and one electrode in Cluster 3 (the 4th electrode from the top in panel C) was highly locked to all onsets except the first word/nonword. These profiles indicate that although the prototypical clusters explain a substantial amount of the functional heterogeneity of responses in the language network, they were not the only observed responses.
Extended Data Fig. 4 Partial correlations of individual response profiles with each of the cluster medoids.
a) Pearson correlations of all response profiles with each of the cluster medoids, grouped by cluster assignment. b) Partial correlations (Methods) of all response profiles with each of the cluster medoids, controlling for the other two cluster medoids, grouped by cluster assignment. c) Response profiles from electrodes assigned to Cluster 1 that had a high partial correlation ( > 0.2, arbitrarily defined threshold) with the Cluster 2 medoid (and split-half reliability>0.3). Top: Average over all electrodes that met these criteria (n = 18, black). The Cluster 1 medoid is shown in red, and the Cluster 2 medoid is shown in green. Bottom: Four sample electrodes (black). d) Response profiles assigned to Cluster 2 that had a high partial correlation ( > 0.2, arbitrarily defined threshold) with the Cluster 1 medoid (and split-half reliability>0.3). Top: Average over all electrodes that meet these criteria (n = 12, black). The Cluster 1 medoid is shown in red, and the Cluster 2 medoid is shown in green. Bottom: Four sample electrodes (black; see osf.io/xfbr8/ for all mixed response profiles with split-half reliability>0.3). e) Anatomical distribution of electrodes in Dataset 1 colored by their partial correlation with a given cluster medoid (controlling for the other two medoids). Cluster-1- and Cluster-2-like responses were present throughout frontal and temporal areas (with Cluster 1 responses having a slightly higher concentration in the temporal pole and Cluster 2 responses having a slightly higher concentration in the superior temporal gyrus (STG)), whereas Cluster-3-like responses were localized to the posterior STG.
Extended Data Fig. 5 N-gram frequencies of sentences and word lists diverge with n-gram length.
N-gram frequencies were extracted from the Google n-gram online platform (https://books.google.com/ngrams/), averaging across Google books corpora between the years 2010 and 2020. For each individual word, the n-gram frequency for n = 1 was the frequency of that word in the corpus; for n = 2 it was the frequency of the bigram (sequence of 2 words) ending in that word; for n = 3 it was the frequency of the trigram (sequence of 3 words) ending in that word; and so on. Sequences that were not found in the corpus were assigned a value of 0. Results are only presented until n = 4 because for n > 4 most of the string sequences, both from the Sentence and Word-list conditions, were not found in the corpora. The plot shows that the difference between the log n-gram values for the sentences and word lists in our stimulus set grows as a function of N. Error bars represent the standard error of the mean across all n-grams extracted from the stimuli used (640, 560, 480, 399 n-grams for n-gram length = 1, 2, 3, and 4, respectively).
Extended Data Fig. 6 Temporal receptive window (TRW) estimates with kernels of different shapes.
The toy TRW model from Fig. 4 was applied using five different kernel shapes: cosine (a), ‘wide’ Gaussian (Gaussian curves with a standard deviation of σ/2 that were truncated at +/− 1 standard deviation, as used in Fig. 4; b), ‘narrow’ Gaussian (Gaussian curves with a standard deviation of σ/16 that were truncated at +/− 8 standard deviations; c), a square (that is, boxcar) function (1 for the entire window; d) and a linear asymmetric function (linear function with a value of 0 initially and a value of 1 at the end of the window; e). For each kernel (a-e), the plots represent (left to right, all details are identical to Fig. 4 in the manuscript): 1) The kernel shapes for TRW = 1, 2, 3, 4, 6 and 8 words, superimposed on the simplified stimulus train; 2) The simulated neural signals for each of those TRWs; 3) violin plots of best fitted TRW values across electrodes (each dot represents an electrode, horizontal black lines are means across the electrodes, white dots are medians, vertical thin box represents lower and upper quartiles and ‘x’ marks indicate outliers; more than 1.5 interquartile ranges above the upper quartile or less than 1.5 interquartile ranges below the lower quartile) for all electrodes (black), or electrodes from only Clusters 1 (red) 2 (green) or 3 (blue); and 4) Estimated TRW as a function of goodness of fit. Each dot is an electrode, its size represents the reliability of its neural response, computed via correlation between the mean signals when using only odd vs. only even trials, x-axis is the electrode’s best fitted TRW, y-axis is the goodness of fit, computed via correlation between the neural signal and the closest simulated signal. For all kernels the TRWs showed a decreasing trend from Cluster 1 to 3.
Extended Data Fig. 7 Dataset 1 k-medoids clustering results with only S and N conditions.
a) Search for optimal k using the ‘elbow method’. Top: variance (sum of the distances of all electrodes to their assigned cluster centre) normalized by the variance when k = 1 as a function of k (normalized variance (NV)). Bottom: change in NV as a function of k (NV(k + 1) – NV(k)). After k = 3 the change in variance became more moderate, suggesting that 3 clusters appropriately described Dataset 1 when using only the responses to sentences and non-words (as was the case when all four conditions were used). b) Clustering mean electrode responses (only S and N, importantly) using k-medoids (k = 3) with a correlation-based distance. Shading of the data matrix reflects normalized high-gamma power (70–150 Hz). c) Average timecourse by cluster. Shaded areas around the signal reflect a 99% confidence interval over electrodes (n = 99, n = 61, and n = 17 electrodes for Cluster 1, 2, and 3, respectively). Clusters 1-3 showed a strong similarity to the clusters reported in Fig. 2. d) Mean condition responses by cluster. Error bars reflect standard error of the mean over electrodes. e) Electrode responses visualized on their first two principal components, colored by cluster. f) Anatomical distribution of clusters across all participants (n = 6). g) Robustness of clusters to electrode omission (random subsets of electrodes were removed in increments of 5). Stars reflect significant similarity with the full dataset (with a p threshold of 0.05; evaluated with a one-sided permutation test, n = 1000 permutations; Methods). Shaded regions reflect standard error of the mean over randomly sampled subsets of electrodes. Relative to when all conditions were used, Cluster 2 was less robust to electrode omission (although still more robust than Cluster 3), suggesting that responses to word lists and Jabberwocky sentences (both not present here) are particularly important for distinguishing Cluster 2 electrodes from Cluster 1 and 3 electrodes.
Extended Data Fig. 8 Dataset 2 electrode assignment to most correlated Dataset 1 cluster under ‘winner-take-all’ (WTA) approach.
a) Assigning electrodes from Dataset 2 to the most correlated cluster from Dataset 1. Assignment was performed using the correlation with the Dataset 1 cluster average, not the cluster medoid. Shading of the data matrix reflects normalized high-gamma power (70–150 Hz). b) Average timecourse by group. Shaded areas around the signal reflect a 99% confidence interval over electrodes (n = 142, n = 95, and n = 125 electrodes for groups 1, 2, and 3, respectively). c) Mean condition responses by group. Error bars reflect standard error of the mean over electrodes (n = 142, n = 95, and n = 125 electrodes for groups 1, 2, and 3, respectively, as in b). d) Electrode responses visualized on their first two principal components, colored by group. e) Anatomical distribution of groups across all participants (n = 16). f-g) Comparison of cluster assignment of electrodes from Dataset 2 using clustering vs. winner-take-all (WTA) approach. f) The numbers in the matrix correspond to the number of electrodes assigned to cluster y during clustering (y-axis) versus the number electrodes assigned to group x during the WTA approach (x-axis). For instance, there were 44 electrodes that were assigned to Cluster 1 during clustering but were ‘pulled out’ to Group 2 (the analog of Cluster 2) during the WTA approach. The total number of electrodes assigned to each cluster during the clustering approach are shown to the right of each row. The total number of electrodes assigned to each group during the WTA approach are shown at the top of each column. N = 362 is the total number of electrodes in Dataset 2. g) Similar to F, but here the average timecourse across all electrodes assigned to the corresponding cluster/group during both procedures is presented. Shaded areas around the signals reflect a 99% confidence interval over electrodes.
Extended Data Fig. 9 Anatomical distribution of the clusters in Dataset 2.
a) Anatomical distribution of language-responsive electrodes in Dataset 2 across all subjects in MNI space, colored by cluster. Only Clusters 1 and 3 (those from Dataset 1 that replicate to Dataset 2) are shown. b) Anatomical distribution of language-responsive electrodes in subject-specific space for eight sample participants. c-h) Violin plots of MNI coordinate values for Clusters 1 and 3 in the left and right hemisphere (c-e and f-h, respectively), where plotted points (n = 16 participants) represent the mean of all coordinate values for a given participant and cluster. The mean across participants is plotted with a black horizontal line, and the median is shown with a white circle. Vertical thin black boxes within violins plots represent the upper and lower quartiles. Significance is evaluated with a LME model (Methods, Supplementary Tables 3 and 4). The Cluster 3 posterior bias from Dataset 1 was weakly present but not statistically reliable.
Extended Data Fig. 10 Estimation of temporal receptive window (TRW) sizes for electrodes in Dataset 2.
As in Fig. 4 but for electrodes in Dataset 2. a) Best TRW fit (using the toy model from Fig. 4) for all electrodes, colored by cluster (when k-medoids clustering with k = 3 was applied, Fig. 6) and sized by the reliability of the neural signal as estimated by correlating responses to odd and even trials (Fig. 6c). The ‘goodness of fit’, or correlation between the simulated and observed neural signal (Sentence condition only), is shown on the y-axis. b) Estimated TRW sizes across all electrodes (grey) and per cluster (red, green, and blue). Black vertical lines correspond to the mean window size and the white dots correspond to the median. ‘x’ marks indicate outliers (more than 1.5 interquartile ranges above the upper quartile or less than 1.5 interquartile ranges below the lower quartile). Significance values were calculated using a linear mixed-effects model (comparing estimate values, two-sided ANOVA for LME, Methods, see Supplementary Table 8 for exact p-values). c-d) Same as A and B, respectively, except that clusters were assigned by highest correlation with Dataset 1 clusters (Extended Data Fig. 8). Under this procedure, Cluster 2 reliably separated from Cluster 3 in terms of its TRW (all ps<0.001, evaluated with a LME model, Methods, see Supplementary Table 9 for exact p-values).
Supplementary information
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Regev, T.I., Casto, C., Hosseini, E.A. et al. Neural populations in the language network differ in the size of their temporal receptive windows. Nat Hum Behav 8, 1924–1942 (2024). https://doi.org/10.1038/s41562-024-01944-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41562-024-01944-2
