{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T04:20:48Z","timestamp":1787026848886,"version":"3.56.0"},"reference-count":71,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,6,16]],"date-time":"2025-06-16T00:00:00Z","timestamp":1750032000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["KR 5148\/3-1"],"award-info":[{"award-number":["KR 5148\/3-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["KR 5148\/5-1"],"award-info":[{"award-number":["KR 5148\/5-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["GRK 2839"],"award-info":[{"award-number":["GRK 2839"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["SCHI 1482\/3-1"],"award-info":[{"award-number":["SCHI 1482\/3-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:p>Understanding how language and linguistic constructions are processed in the brain is a fundamental question in cognitive computational neuroscience. This study builds directly on our previous work analyzing Argument Structure Constructions (ASCs) in the BERT language model, extending the investigation to a simpler, brain-constrained architecture: a recurrent neural language model. Specifically, we explore the representation and processing of four ASCs\u2013transitive, ditransitive, caused-motion, and resultative\u2013in a Long Short-Term Memory (LSTM) network. We trained the LSTM on a custom GPT-4-generated dataset of 2,000 syntactically balanced sentences. We then analyzed the internal hidden layer activations using Multidimensional Scaling (MDS) and t-Distributed Stochastic Neighbor Embedding (t-SNE) to visualize sentence representations. The Generalized Discrimination Value (GDV) was calculated to quantify cluster separation. Our results show distinct clusters for the four ASCs across all hidden layers, with the strongest separation observed in the final layer. These findings are consistent with our earlier study based on a large language model and demonstrate that even relatively simple RNNs can form abstract, construction-level representations. This supports the hypothesis that hierarchical linguistic structure can emerge through prediction-based learning. In future work, we plan to compare these model-derived representations with neuroimaging data from continuous speech perception, further bridging computational and biological perspectives on language processing.<\/jats:p>","DOI":"10.3389\/fncom.2025.1474860","type":"journal-article","created":{"date-parts":[[2025,6,16]],"date-time":"2025-06-16T02:16:21Z","timestamp":1750040181000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Analysis of argument structure constructions in a deep recurrent language model"],"prefix":"10.3389","volume":"19","author":[{"given":"Pegah","family":"Ramezani","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Achim","family":"Schilling","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Patrick","family":"Krauss","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,6,16]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","author":"Almeida","year":"2019","DOI":"10.48550\/arXiv.1901.09069"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1319","DOI":"10.3758\/s13415-019-00721-3","article-title":"The hierarchically mechanistic mind: an evolutionary systems theory of the human brain, cognition, and behavior","volume":"19","author":"Badcock","year":"2019","journal-title":"Cognit. Affect. Behav. Neurosci"},{"key":"B3","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B4","unstructured":"Chollet\n              F.\n            \n          \n          Keras\n          \n          2015"},{"key":"B5","doi-asserted-by":"publisher","first-page":"8514","DOI":"10.1523\/JNEUROSCI.1503-22.2022","article-title":"Recent advances at the interface of neuroscience and artificial neural networks","volume":"42","author":"Cohen","year":"2022","journal-title":"J. Neurosci"},{"key":"B6","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1007\/978-3-540-33037-0_14","article-title":"\u201cMultidimensional scaling,\u201d","volume-title":"Handbook of Data Visualization","author":"Cox","year":"2008"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1810.04805","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018","journal-title":"arXiv"},{"key":"B8","unstructured":"Explosion\n              A.\n            \n          \n          Spacy-Industrial-Strength Natural Language Processing in Python\n          \n          2017"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1076339","DOI":"10.3389\/fpsyg.2022.1076339","article-title":"Neural correlates of linguistic collocations during continuous speech perception","volume":"13","author":"Garibyan","year":"2022","journal-title":"Front. Psychol"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.2307\/j.ctvc772nn","author":"Goldberg","year":"2019","journal-title":"Explain Me This"},{"key":"B11","volume-title":"Constructions: A Construction Grammar Approach to Argument Structure","author":"Goldberg","year":"1995"},{"key":"B12","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1016\/S1364-6613(03)00080-9","article-title":"Constructions: a new theoretical approach to language","volume":"7","author":"Goldberg","year":"2003","journal-title":"Trends Cogn. Sci"},{"key":"B13","volume-title":"Constructions at Work: The Nature of Generalization in Language","author":"Goldberg","year":"2006"},{"key":"B14","author":"Goldberg","year":"2009","journal-title":"The Nature of Generalization in Language"},{"key":"B15","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1177\/1367006912456585","article-title":"Brain structural correlates of individual differences at low-to high-levels of the language processing hierarchy: A review of new approaches to imaging research","volume":"18","author":"Golestani","year":"2014","journal-title":"Int. J. Bilingual"},{"key":"B16","doi-asserted-by":"crossref","DOI":"10.25080\/TCWV9851","article-title":"\u201cExploring network structure, dynamics, and function using networkx,\u201d","author":"Hagberg","year":"2008","journal-title":"Proceedings of the 7th Python in Science Conference"},{"key":"B17","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","article-title":"Array programming with NumPy","volume":"585","author":"Harris","year":"2020","journal-title":"Nature"},{"key":"B18","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1016\/j.neuron.2017.06.011","article-title":"Neuroscience-inspired artificial intelligence","volume":"95","author":"Hassabis","year":"2017","journal-title":"Neuron"},{"key":"B19","doi-asserted-by":"publisher","first-page":"2533","DOI":"10.1007\/s00426-021-01591-6","article-title":"Modelling concrete and abstract concepts using brain-constrained deep neural networks","volume":"86","author":"Henningsen-Schomers","year":"2022","journal-title":"Psychol. Res"},{"key":"B20","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"B21","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1109\/MCSE.2007.55","article-title":"Matplotlib: A 2d graphics environment","volume":"9","author":"Hunter","year":"2007","journal-title":"Comp. Sci. Eng"},{"key":"B22","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-662-67179-5","volume-title":"K\u00fcnstliche Intelligenz und Hirnforschung: Neuronale Netze, Deep Learning und die Zukunft der Kognition","author":"Krauss","year":"2023"},{"key":"B23","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-662-68980-6","volume-title":"Artificial Intelligence and Brain Research: Neural Networks, Deep Learning and the Future of Cognition","author":"Krauss","year":"2024"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2405.02024","article-title":"Analyzing narrative processing in large language models (LLMs): Using GPT4 to test bert","author":"Krauss","year":"2024","journal-title":"arXiv"},{"key":"B25","doi-asserted-by":"publisher","first-page":"100064","DOI":"10.1016\/j.nbscr.2021.100064","article-title":"Analysis and visualization of sleep stages based on deep neural networks","volume":"10","author":"Krauss","year":"2021","journal-title":"Neurobiol. Sleep Circad. Rhythms"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-018-23765-w","article-title":"A statistical method for analyzing and comparing spatiotemporal cortical activation patterns","volume":"8","author":"Krauss","year":"","journal-title":"Sci. Rep"},{"key":"B27","doi-asserted-by":"publisher","first-page":"64","DOI":"10.3389\/fncom.2019.00064","article-title":"\u201cRecurrence resonance\u201d in three-neuron motifs","volume":"2019","author":"Krauss","year":"","journal-title":"Front. Comput. Neurosci"},{"key":"B28","doi-asserted-by":"publisher","first-page":"121","DOI":"10.3389\/fnhum.2018.00121","article-title":"Analysis of multichannel eeg patterns during human sleep: a novel approach","volume":"12","author":"Krauss","year":"","journal-title":"Front. Hum. Neurosci"},{"key":"B29","doi-asserted-by":"publisher","first-page":"e0214541","DOI":"10.1371\/journal.pone.0214541","article-title":"Weight statistics controls dynamics in recurrent neural networks","volume":"14","author":"Krauss","year":"","journal-title":"PLoS ONE"},{"key":"B30","doi-asserted-by":"publisher","first-page":"5","DOI":"10.3389\/fncom.2019.00005","article-title":"Analysis of structure and dynamics in three-neuron motifs","volume":"13","author":"Krauss","year":"","journal-title":"Front. Comput. Neurosci"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1148","DOI":"10.1038\/s41593-018-0210-5","article-title":"Cognitive computational neuroscience","volume":"21","author":"Kriegeskorte","year":"2018","journal-title":"Nat. Neurosci"},{"key":"B32","doi-asserted-by":"publisher","first-page":"249","DOI":"10.3389\/neuro.06.004.2008","article-title":"Representational similarity analysis-connecting the branches of systems neuroscience","volume":"2","author":"Kriegeskorte","year":"2008","journal-title":"Front. Syst. Neurosci"},{"key":"B33","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/BF02289694","article-title":"Nonmetric multidimensional scaling: a numerical method","volume":"29","author":"Kruskal","year":"1964","journal-title":"Psychometrika"},{"key":"B34","doi-asserted-by":"crossref","DOI":"10.4135\/9781412985130","volume-title":"Multidimensional Scaling","author":"Kruskal","year":"1978"},{"key":"B35","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res"},{"key":"B36","doi-asserted-by":"publisher","first-page":"749","DOI":"10.1080\/09515080701694128","article-title":"Neural models that convince: model hierarchies and other strategies to bridge the gap between behavior and the brain","volume":"20","author":"Meeter","year":"2007","journal-title":"Philos. Psychol"},{"key":"B37","doi-asserted-by":"publisher","first-page":"876315","DOI":"10.3389\/fncom.2022.876315","article-title":"Dynamics and information import in recurrent neural networks","volume":"16","author":"Metzner","year":"2022","journal-title":"Front. Comput. Neurosci"},{"key":"B38","doi-asserted-by":"publisher","first-page":"1385","DOI":"10.1038\/s42003-021-02912-6","article-title":"Sleep as a random walk: a super-statistical analysis of eeg data across sleep stages","volume":"4","author":"Metzner","year":"2021","journal-title":"Commun. Biol"},{"key":"B39","doi-asserted-by":"publisher","first-page":"100097","DOI":"10.1016\/j.nbscr.2023.100097","article-title":"Extracting continuous sleep depth from eeg data without machine learning","volume":"14","author":"Metzner","year":"","journal-title":"Neurobiol. Sleep Circad. Rhythms"},{"key":"B40","doi-asserted-by":"publisher","first-page":"22121","DOI":"10.1038\/s41598-022-26498-z","article-title":"Classification at the accuracy limit: facing the problem of data ambiguity","volume":"12","author":"Metzner","year":"2022","journal-title":"Sci. Rep"},{"key":"B41","doi-asserted-by":"publisher","author":"Metzner","year":"","DOI":"10.48550\/arXiv.2301.12892"},{"key":"B42","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1162\/neco_a_01651","article-title":"Quantifying and maximizing the information flux in recurrent neural networks","volume":"36","author":"Metzner","year":"2024","journal-title":"Neural Comput"},{"key":"B43","doi-asserted-by":"publisher","first-page":"1482","DOI":"10.1038\/s41587-019-0336-3","article-title":"Visualizing structure and transitions in high-dimensional biological data","volume":"37","author":"Moon","year":"2019","journal-title":"Nat. Biotechnol"},{"key":"B44","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1109\/MCSE.2007.58","article-title":"Python for scientific computing","volume":"9","author":"Oliphant","year":"2007","journal-title":"Comp. Sci. Eng"},{"key":"B45","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res"},{"key":"B46","unstructured":"Plappert\n              M.\n            \n          \n          keras-RL\n          \n          2016"},{"key":"B47","volume-title":"The Neuroscience of Language: On Brain Circuits of Words and Serial Order","author":"Pulverm\u00fcller","year":"2002"},{"key":"B48","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1016\/j.jneuroling.2011.03.004","article-title":"Meaning and the brain: the neurosemantics of referential, interactive, and combinatorial knowledge","volume":"25","author":"Pulverm\u00fcller","year":"2012","journal-title":"J. Neurolinguistics"},{"key":"B49","doi-asserted-by":"publisher","first-page":"102511","DOI":"10.1016\/j.pneurobio.2023.102511","article-title":"Neurobiological mechanisms for language, symbols and concepts: clues from brain-constrained deep neural networks","volume":"230","author":"Pulverm\u00fcller","year":"2023","journal-title":"Prog. Neurobiol"},{"key":"B50","doi-asserted-by":"publisher","first-page":"488","DOI":"10.1038\/s41583-021-00473-5","article-title":"Biological constraints on neural network models of cognitive function","volume":"22","author":"Pulverm\u00fcller","year":"2021","journal-title":"Nat. Rev. Neurosci"},{"key":"B51","author":"Radford","year":"2018","journal-title":"Improving Language Understanding with Unsupervised Learning"},{"key":"B52","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI blog"},{"key":"B53","doi-asserted-by":"publisher","first-page":"1477246","DOI":"10.3389\/frai.2025.1477246","article-title":"Analysis of argument structure constructions in the large language model bert","volume":"8","author":"Ramezani","year":"2025","journal-title":"Front. Artif. Intellig"},{"key":"B54","doi-asserted-by":"publisher","first-page":"20890","DOI":"10.1073\/pnas.2003383117","article-title":"Hierarchical dynamics as a macroscopic organizing principle of the human brain","volume":"117","author":"Raut","year":"2020","journal-title":"Proc. Nat. Acad. Sci"},{"key":"B55","doi-asserted-by":"publisher","first-page":"S15","DOI":"10.1038\/d41586-019-02212-4","article-title":"How ai and neuroscience drive each other forwards","volume":"571","author":"Savage","year":"2019","journal-title":"Nature"},{"key":"B56","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1016\/j.neunet.2021.03.035","article-title":"Quantifying the separability of data classes in neural networks","volume":"139","author":"Schilling","year":"","journal-title":"Neural Netw"},{"key":"B57","doi-asserted-by":"publisher","first-page":"1293552","DOI":"10.3389\/978-2-8325-3756-5","article-title":"Auditory perception and phantom perception in brains, minds and machines","volume":"17","author":"Schilling","year":"2023","journal-title":"Front. Neurosci"},{"key":"B58","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1080\/23273798.2020.1803375","article-title":"Analysis of continuous neuronal activity evoked by natural speech with computational corpus linguistics methods","volume":"36","author":"Schilling","year":"","journal-title":"Lang. Cognit. Neurosci"},{"key":"B59","doi-asserted-by":"publisher","first-page":"7429","DOI":"10.1523\/JNEUROSCI.1247-23.2023","article-title":"Attentional modulation of the cortical contribution to the frequency-following response evoked by continuous speech","volume":"43","author":"Sch\u00fcller","year":"2023","journal-title":"J. Neurosci"},{"key":"B60","doi-asserted-by":"publisher","first-page":"475","DOI":"10.1162\/jocn_a_02103","article-title":"The early subcortical response at the fundamental frequency of speech is temporally separated from later cortical contributions","volume":"36","author":"Sch\u00fcller","year":"2024","journal-title":"J. Cogn. Neurosci"},{"key":"B61","doi-asserted-by":"crossref","first-page":"1481","DOI":"10.1109\/ICMLA58977.2023.00223","article-title":"\u201cWord class representations spontaneously emerge in a deep neural network trained on next word prediction,\u201d","volume-title":"2023 International Conference on Machine Learning and Applications (ICMLA)","author":"Surendra","year":"2023"},{"key":"B62","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1007\/BF02288916","article-title":"Multidimensional scaling: I. theory and method","volume":"17","author":"Torgerson","year":"1952","journal-title":"Psychometrika"},{"key":"B63","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1007\/s11818-019-0201-0","article-title":"Microstructure of cortical activity during sleep reflects respiratory events and state of daytime vigilance","volume":"23","author":"Traxdorf","year":"2019","journal-title":"Somnologie"},{"key":"B64","doi-asserted-by":"publisher","first-page":"101142","DOI":"10.1146\/annurev-neuro-120623-101142","article-title":"Language in brains, minds, and machines","volume":"47","author":"Tuckute","year":"2024","journal-title":"Annu. Rev. Neurosci"},{"key":"B65","doi-asserted-by":"publisher","first-page":"1423","DOI":"10.1038\/s41587-019-0330-9","article-title":"Exploring a world of a thousand dimensions","volume":"37","author":"Vallejos","year":"2019","journal-title":"Nat. Biotechnol"},{"key":"B66","first-page":"11","article-title":"Visualizing data using t-SNE","volume":"9","author":"Van der Maaten","year":"2008","journal-title":"J. Mach. Learn. Res"},{"key":"B67","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1109\/MCSE.2011.37","article-title":"The numpy array: a structure for efficient numerical computation","volume":"13","author":"Van Der Walt","year":"2011","journal-title":"Comp. Sci. Eng"},{"key":"B68","article-title":"Attention is all you need","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B69","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1512.08849","article-title":"Learning natural language inference with LSTM","author":"Wang","year":"2015","journal-title":"arXiv"},{"key":"B70","doi-asserted-by":"publisher","DOI":"10.21105\/joss.03021","article-title":"Seaborn: statistical data visualization","author":"Waskom","year":"2021","journal-title":"J. Open Source Softw"},{"key":"B71","doi-asserted-by":"publisher","first-page":"e2","DOI":"10.23915\/distill.00002","article-title":"How to use t-SNE effectively","volume":"1","author":"Wattenberg","year":"2016","journal-title":"Distill"}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2025.1474860\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,16]],"date-time":"2025-06-16T02:16:28Z","timestamp":1750040188000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2025.1474860\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,16]]},"references-count":71,"alternative-id":["10.3389\/fncom.2025.1474860"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2025.1474860","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,16]]},"article-number":"1474860"}}