{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T13:49:51Z","timestamp":1781704191817,"version":"3.54.5"},"reference-count":68,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2023,6,23]],"date-time":"2023-06-23T00:00:00Z","timestamp":1687478400000},"content-version":"vor","delay-in-days":173,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,6,20]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Many current NLP systems are built from language models trained to optimize unsupervised objectives on large amounts of raw text. Under what conditions might such a procedure acquire meaning? Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations (i.e., languages with strong transparency), both autoregressive and masked language models successfully learn to emulate semantic relations between expressions. However, when denotations are changed to be context-dependent with the language otherwise unmodified, this ability degrades. Turning to natural language, our experiments with a specific phenomenon\u2014referential opacity\u2014add to the growing body of evidence that current language models do not represent natural language semantics well. We show this failure relates to the context-dependent nature of natural language form-meaning mappings.<\/jats:p>","DOI":"10.1162\/tacl_a_00565","type":"journal-article","created":{"date-parts":[[2023,6,23]],"date-time":"2023-06-23T17:35:39Z","timestamp":1687541739000},"page":"617-634","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":7,"title":["Transparency Helps Reveal When Language Models Learn Meaning"],"prefix":"10.1162","volume":"11","author":[{"given":"Zhaofeng","family":"Wu","sequence":"first","affiliation":[{"name":"MIT. zfw@csail.mit.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William","family":"Merrill","sequence":"additional","affiliation":[{"name":"New York University. willm@nyu.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Peng","sequence":"additional","affiliation":[{"name":"Allen Institute for Artificial Intelligence. haop@allenai.org"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Iz","family":"Beltagy","sequence":"additional","affiliation":[{"name":"Allen Institute for Artificial Intelligence. beltagy@allenai.org"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Noah A.","family":"Smith","sequence":"additional","affiliation":[{"name":"Allen Institute for Artificial Intelligence. noah@allenai.org"},{"name":"Paul G. Allen School of Computer Science & Engineering, University of Washington. noah@allenai.org"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2023,6,20]]},"reference":[{"key":"2023062317264083700_bib1","article-title":"Understanding intermediate layers using linear classifier probes","volume-title":"5th International Conference on Learning Representations, Workshop Track Proceedings","author":"Alain","year":"2017"},{"key":"2023062317264083700_bib2","volume-title":"Propositional Attitudes: The Role of Content in Logic, Language, and Mind. CSLI Lecture Notes; No. 20","author":"Anthony Anderson","year":"1990"},{"key":"2023062317264083700_bib3","article-title":"Measuring compositionality in representation learning","volume-title":"Proceedings of International Conference on Learning Representations","author":"Andreas","year":"2019"},{"key":"2023062317264083700_bib4","doi-asserted-by":"publisher","first-page":"5185","DOI":"10.18653\/v1\/2020.acl-main.463","article-title":"Climbing towards NLU: On meaning, form, and understanding in the age of data","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Bender","year":"2020"},{"key":"2023062317264083700_bib5","doi-asserted-by":"publisher","first-page":"632","DOI":"10.18653\/v1\/D15-1075","article-title":"A large annotated corpus for learning natural language inference","volume-title":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","author":"Bowman","year":"2015"},{"key":"2023062317264083700_bib6","first-page":"1877","article-title":"Language models are few-shot learners","volume-title":"Proceedings of Advances in Neural Information Processing Systems","author":"Brown","year":"2020"},{"key":"2023062317264083700_bib7","doi-asserted-by":"publisher","first-page":"1860","DOI":"10.18653\/v1\/2021.acl-long.146","article-title":"Knowledgeable or educated guess? Revisiting language models as knowledge bases","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Cao","year":"2021"},{"key":"2023062317264083700_bib8","volume-title":"Meaning and Necessity: A Study in Semantics and Modal Logic","author":"Carnap","year":"1947"},{"key":"2023062317264083700_bib9","volume-title":"Lectures on Government and Binding","author":"Chomsky","year":"1981"},{"key":"2023062317264083700_bib10","volume-title":"Some Concepts and Consequences of the Theory of Government and Binding","author":"Chomsky","year":"1983"},{"key":"2023062317264083700_bib11","doi-asserted-by":"publisher","first-page":"2126","DOI":"10.18653\/v1\/P18-1198","article-title":"What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Conneau","year":"2018"},{"key":"2023062317264083700_bib12","doi-asserted-by":"publisher","first-page":"4171","DOI":"10.18653\/v1\/N19-1423","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2023062317264083700_bib13","doi-asserted-by":"publisher","first-page":"2353","DOI":"10.18653\/v1\/2021.naacl-main.186","article-title":"Static embeddings as efficient knowledge bases?","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Dufter","year":"2021"},{"key":"2023062317264083700_bib14","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4899-4541-9","volume-title":"An Introduction to the Bootstrap","author":"Efron","year":"1993"},{"key":"2023062317264083700_bib15","article-title":"T-REx: A large scale alignment of natural language with knowledge base triples","volume-title":"Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)","author":"Elsahar","year":"2018"},{"key":"2023062317264083700_bib16","doi-asserted-by":"publisher","first-page":"55","DOI":"10.18653\/v1\/D19-1006","article-title":"How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Ethayarajh","year":"2019"},{"key":"2023062317264083700_bib17","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1162\/tacl_a_00298","article-title":"What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models","volume":"8","author":"Ettinger","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023062317264083700_bib18","first-page":"1","article-title":"Quine on quantifying in","volume-title":"Propositional attitudes: The role of content in logic, language, and mind","author":"Fine","year":"1990"},{"key":"2023062317264083700_bib19","volume-title":"The Design of Experiments","author":"Fisher","year":"1935"},{"issue":"1","key":"2023062317264083700_bib20","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1016\/0010-0277(88)90031-5","article-title":"Connectionism and cognitive architecture: A critical analysis","volume":"28","author":"Fodor","year":"1988","journal-title":"Cognition"},{"key":"2023062317264083700_bib21","article-title":"\u00dcber sinn und bedeutung","author":"Frege","year":"1892","journal-title":"Zeitschrift f\u00fcr Philosophie Und Philosophische Kritik"},{"key":"2023062317264083700_bib22","doi-asserted-by":"publisher","first-page":"825","DOI":"10.1162\/tacl_a_00400","article-title":"Let\u2019s play mono-poly: BERT can reveal words\u2019 polysemy level and partitionability into senses","volume":"9","author":"Soler","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023062317264083700_bib23","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1163\/9789004368811_003","article-title":"Logic and conversation","volume-title":"Syntax and Semantics: Vol. 3: Speech Acts","author":"Grice","year":"1975"},{"issue":"1","key":"2023062317264083700_bib24","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1007\/BF00628304","article-title":"Dynamic predicate logic","volume":"14","author":"Groenendijk","year":"1991","journal-title":"Linguistics and Philosophy"},{"key":"2023062317264083700_bib25","doi-asserted-by":"publisher","first-page":"1195","DOI":"10.18653\/v1\/N18-1108","article-title":"Colorless green recurrent networks dream hierarchically","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Gulordava","year":"2018"},{"key":"2023062317264083700_bib26","doi-asserted-by":"crossref","unstructured":"Adi Haviv , OriRam, OfirPress, PeterIzsak, and OmerLevy. 2022. Transformer language models without positional encodings still learn positional information. 10.48550\/arXiv.2203.16634","DOI":"10.18653\/v1\/2022.findings-emnlp.99"},{"key":"2023062317264083700_bib27","unstructured":"Irene Heim . 1982. The Semantics of Definite and Indefinite Noun Phrases. Ph.D. thesis, University of Massachusetts Amherst."},{"key":"2023062317264083700_bib28","volume-title":"Semantics in Generative Grammar","author":"Heim","year":"1998"},{"key":"2023062317264083700_bib29","doi-asserted-by":"publisher","first-page":"2733","DOI":"10.18653\/v1\/D19-1275","article-title":"Designing and interpreting probes with control tasks","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Hewitt","year":"2019"},{"key":"2023062317264083700_bib30","doi-asserted-by":"publisher","first-page":"4129","DOI":"10.18653\/v1\/N19-1419","article-title":"A structural probe for finding syntax in word representations","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Hewitt","year":"2019"},{"key":"2023062317264083700_bib31","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1162\/tacl_a_00324","article-title":"How can we know what language models know?","volume":"8","author":"Jiang","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023062317264083700_bib32","first-page":"277","article-title":"A theory of truth and semantic representation","volume-title":"Formal Methods in the Study of Language","author":"Kamp","year":"1981"},{"key":"2023062317264083700_bib33","doi-asserted-by":"publisher","DOI":"10.1515\/9783110219258.43","volume-title":"Syntactic Recursion and Iteration","author":"Karlsson","year":"2010"},{"key":"2023062317264083700_bib34","doi-asserted-by":"publisher","first-page":"6769","DOI":"10.18653\/v1\/2020.emnlp-main.550","article-title":"Dense passage retrieval for open-domain question answering","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Karpukhin","year":"2020"},{"key":"2023062317264083700_bib35","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-230-35609-2","volume-title":"Semantics","author":"Kearns","year":"2011"},{"key":"2023062317264083700_bib36","doi-asserted-by":"publisher","first-page":"4365","DOI":"10.18653\/v1\/D19-1445","article-title":"Revealing the dark secrets of BERT","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Kovaleva","year":"2019"},{"key":"2023062317264083700_bib37","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1007\/978-94-010-2557-7_9","article-title":"Naming and necessity","volume-title":"Semantics of Natural Language","author":"Kripke","year":"1972"},{"key":"2023062317264083700_bib38","doi-asserted-by":"publisher","first-page":"1813","DOI":"10.18653\/v1\/2021.acl-long.143","article-title":"Implicit representations of meaning in neural language models","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Li","year":"2021"},{"key":"2023062317264083700_bib39","doi-asserted-by":"crossref","first-page":"9119","DOI":"10.18653\/v1\/2020.emnlp-main.733","article-title":"On the sentence embeddings from pre-trained language models","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Li","year":"2020"},{"key":"2023062317264083700_bib40","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1162\/tacl_a_00115","article-title":"Assessing the ability of LSTMs to learn syntax-sensitive dependencies","volume":"4","author":"Linzen","year":"2016","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023062317264083700_bib41","doi-asserted-by":"publisher","first-page":"1073","DOI":"10.18653\/v1\/N19-1112","article-title":"Linguistic knowledge and transferability of contextual representations","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Liu","year":"2019"},{"key":"2023062317264083700_bib42","doi-asserted-by":"publisher","first-page":"2171","DOI":"10.18653\/v1\/N19-1225","article-title":"Inoculation by fine-tuning: A method for analyzing challenge datasets","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Liu","year":"2019"},{"key":"2023062317264083700_bib43","unstructured":"Yinhan Liu , MyleOtt, NamanGoyal, JingfeiDu, MandarJoshi, DanqiChen, OmerLevy, MikeLewis, LukeZettlemoyer, and VeselinStoyanov. 2019c. RoBERTa: A robustly optimized BERT pretraining approach. 10.48550\/arXiv.1907.11692"},{"key":"2023062317264083700_bib44","article-title":"Decoupled weight decay regularization","volume-title":"Proceedings of International Conference on Learning Representations","author":"Loshchilov","year":"2019"},{"key":"2023062317264083700_bib45","doi-asserted-by":"publisher","first-page":"5286","DOI":"10.18653\/v1\/2022.naacl-main.388","article-title":"Is \u201cmy favorite new movie\u201d my favorite movie? Probing the understanding of recursive noun phrases","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Lyu","year":"2022"},{"key":"2023062317264083700_bib46","doi-asserted-by":"publisher","first-page":"1047","DOI":"10.1162\/tacl_a_00412","article-title":"Provable limitations of acquiring meaning from ungrounded form: What will future language models understand?","volume":"9","author":"Merrill","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023062317264083700_bib47","doi-asserted-by":"publisher","first-page":"1583","DOI":"10.18653\/v1\/2021.emnlp-main.119","article-title":"Sorting through the noise: Testing robustness of information processing in pre-trained language models","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Pandia","year":"2021"},{"key":"2023062317264083700_bib48","article-title":"Mapping language models to grounded conceptual spaces","volume-title":"International Conference on Learning Representations","author":"Patel","year":"2022"},{"key":"2023062317264083700_bib49","doi-asserted-by":"publisher","first-page":"2463","DOI":"10.18653\/v1\/D19-1250","article-title":"Language models as knowledge bases?","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Petroni","year":"2019"},{"key":"2023062317264083700_bib50","doi-asserted-by":"publisher","first-page":"803","DOI":"10.18653\/v1\/2020.findings-emnlp.71","article-title":"E-BERT: Efficient-yet-effective entity embeddings for BERT","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Poerner","year":"2020"},{"issue":"5","key":"2023062317264083700_bib51","doi-asserted-by":"publisher","first-page":"177","DOI":"10.2307\/2022451","article-title":"Quantifiers and propositional attitudes","volume":"53","author":"Quine","year":"1956","journal-title":"The Journal of Philosophy"},{"key":"2023062317264083700_bib52","unstructured":"Alec Radford , JeffWu, RewonChild, DavidLuan, DarioAmodei, and IlyaSutskever. 2019. Language models are unsupervised multitask learners. https:\/\/d4mucfpksywv.cloudfront.net\/better-language-models\/language_models_are_unsupervised_multitask_learners.pdf."},{"key":"2023062317264083700_bib53","first-page":"88","article-title":"On the systematicity of probing contextualized word representations: The case of hypernymy in BERT","volume-title":"Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics","author":"Ravichander","year":"2020"},{"key":"2023062317264083700_bib54","doi-asserted-by":"publisher","first-page":"3982","DOI":"10.18653\/v1\/D19-1410","article-title":"Sentence-BERT: Sentence embeddings using Siamese BERT-networks","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Reimers","year":"2019"},{"key":"2023062317264083700_bib55","unstructured":"Tanya Reinhart . 1976. The Syntactic Domain of Anaphora. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge."},{"key":"2023062317264083700_bib56","unstructured":"Daniel S. Shabasson . 2018. The Two Indexical Uses Theory of Proper Names and Frege\u2019s Puzzle. Ph.D. thesis, City University of New York."},{"key":"2023062317264083700_bib57","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1007\/BF00630917","article-title":"Evidence against the context-freeness of natural language","volume":"8","author":"Shieber","year":"1985","journal-title":"Linguistics and Philosophy"},{"key":"2023062317264083700_bib58","doi-asserted-by":"publisher","first-page":"4222","DOI":"10.18653\/v1\/2020.emnlp-main.346","article-title":"AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Shin","year":"2020"},{"key":"2023062317264083700_bib59","article-title":"Theories of meaning","volume-title":"The Stanford Encyclopedia of Philosophy","author":"Speaks","year":"2021"},{"key":"2023062317264083700_bib60","article-title":"What do you learn from context? Probing for sentence structure in contextualized word representations","volume-title":"Proceedings of International Conference on Learning Representations","author":"Tenney","year":"2019"},{"key":"2023062317264083700_bib61","doi-asserted-by":"publisher","first-page":"158","DOI":"10.18653\/v1\/2021.acl-short.21","article-title":"AND does not mean OR: Using formal languages to study language models\u2019 representations","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)","author":"Traylor","year":"2021"},{"key":"2023062317264083700_bib62","doi-asserted-by":"publisher","first-page":"4396","DOI":"10.18653\/v1\/D19-1448","article-title":"The bottom-up evolution of representations in the transformer: A study with machine translation and language modeling objectives","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Voita","year":"2019"},{"key":"2023062317264083700_bib63","article-title":"Intensional semantics","author":"Von Fintel","year":"2011","journal-title":"Unpublished Lecture Notes"},{"key":"2023062317264083700_bib64","doi-asserted-by":"publisher","first-page":"7222","DOI":"10.18653\/v1\/2020.emnlp-main.586","article-title":"Probing pretrained language models for lexical semantics","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Vuli\u0107","year":"2020"},{"key":"2023062317264083700_bib65","doi-asserted-by":"publisher","first-page":"226","DOI":"10.1162\/tacl_a_00363","article-title":"Infusing finetuning with semantic dependencies","volume":"9","author":"Zhaofeng","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023062317264083700_bib66","doi-asserted-by":"publisher","first-page":"4896","DOI":"10.18653\/v1\/2020.emnlp-main.397","article-title":"Assessing phrasal representation and composition in transformers","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Lang","year":"2020"},{"key":"2023062317264083700_bib67","first-page":"2279","article-title":"On the interplay between fine-tuning and composition in transformers","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Lang","year":"2021"},{"key":"2023062317264083700_bib68","doi-asserted-by":"publisher","first-page":"5017","DOI":"10.18653\/v1\/2021.naacl-main.398","article-title":"Factual probing is [MASK]: Learning vs. learning to recall","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Zhong","year":"2021"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00565\/2138350\/tacl_a_00565.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00565\/2138350\/tacl_a_00565.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,15]],"date-time":"2023-12-15T15:33:10Z","timestamp":1702654390000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00565\/116467\/Transparency-Helps-Reveal-When-Language-Models"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":68,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00565","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}