{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T14:01:00Z","timestamp":1774965660896,"version":"3.50.1"},"reference-count":70,"publisher":"MIT Press","issue":"00","license":[{"start":{"date-parts":[[2024,4,17]],"date-time":"2024-04-17T00:00:00Z","timestamp":1713312000000},"content-version":"vor","delay-in-days":107,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,4,16]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Automatic text simplification (TS) aims to automate the process of rewriting text to make it easier for people to read. A pre-requisite for TS to be useful is that it should convey information that is consistent with the meaning of the original text. However, current TS evaluation protocols assess system outputs for simplicity and meaning preservation without regard for the document context in which output sentences occur and for how people understand them. In this work, we introduce a human evaluation framework to assess whether simplified texts preserve meaning using reading comprehension questions. With this framework, we conduct a thorough human evaluation of texts by humans and by nine automatic systems. Supervised systems that leverage pre-training knowledge achieve the highest scores on the reading comprehension tasks among the automatic controllable TS systems. However, even the best-performing supervised system struggles with at least 14% of the questions, marking them as \u201cunanswerable\u201d based on simplified content. We further investigate how existing TS evaluation metrics and automatic question-answering systems approximate the human judgments we obtained.<\/jats:p>","DOI":"10.1162\/tacl_a_00653","type":"journal-article","created":{"date-parts":[[2024,4,17]],"date-time":"2024-04-17T20:20:50Z","timestamp":1713385250000},"page":"432-448","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":5,"title":["Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehension"],"prefix":"10.1162","volume":"12","author":[{"given":"Sweta","family":"Agrawal","sequence":"first","affiliation":[{"name":"Instituto de Telecomunica\u00e7\u00f5es. swetaagrawal20@gmail.com"}]},{"given":"Marine","family":"Carpuat","sequence":"additional","affiliation":[{"name":"University of Maryland, USA. marine@umd.edu"}]}],"member":"281","published-online":{"date-parts":[[2024,4,16]]},"reference":[{"key":"2024041720204272700_bib1","doi-asserted-by":"publisher","first-page":"7550","DOI":"10.18653\/v1\/2022.acl-long.520","article-title":"An imitation learning curriculum for text editing with non-autoregressive models","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Agrawal","year":"2022"},{"issue":"2","key":"2024041720204272700_bib2","doi-asserted-by":"publisher","DOI":"10.1145\/3442695","article-title":"Automated text simplification: A survey","volume":"54","author":"Al-Thanyyan","year":"2021","journal-title":"ACM Computing Surveys"},{"issue":"4","key":"2024041720204272700_bib3","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1016\/j.system.2009.09.004","article-title":"A study of the role of relative clauses in the simplification of news texts for learners of English","volume":"37","author":"Allen","year":"2009","journal-title":"System"},{"key":"2024041720204272700_bib4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3313831.3376563","article-title":"Automatic text simplification tools for deaf and hard of hearing adults: Benefits of lexical simplification and providing users with autonomy","volume-title":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems","author":"Alonzo","year":"2020"},{"issue":"4","key":"2024041720204272700_bib5","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1162\/coli_a_00418","article-title":"The (un)suitability of automatic evaluation metrics for text simplification","volume":"47","author":"Alva-Manchego","year":"2021","journal-title":"Computational Linguistics"},{"key":"2024041720204272700_bib6","first-page":"1996","article-title":"Lexico-syntactic text simplification and compression with typed dependencies","volume-title":"Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers","author":"Angrosh","year":"2014"},{"key":"2024041720204272700_bib7","doi-asserted-by":"publisher","first-page":"5726","DOI":"10.18653\/v1\/2020.acl-main.507","article-title":"STARC: Structured annotations for reading comprehension","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Berzak","year":"2020"},{"key":"2024041720204272700_bib8","doi-asserted-by":"publisher","first-page":"315","DOI":"10.3115\/1220355.1220401","article-title":"Confidence estimation for machine translation","volume-title":"COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics","author":"Blatz","year":"2004"},{"key":"2024041720204272700_bib9","doi-asserted-by":"publisher","DOI":"10.3115\/993268.993361","article-title":"Motivations and methods for text simplification","volume-title":"COLING 1996 Volume 2: The 16th International Conference on Computational Linguistics","author":"Chandrasekar","year":"1996"},{"issue":"3","key":"2024041720204272700_bib10","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1016\/S0950-7051(97)00029-4","article-title":"Automatic induction of rules for text simplification","volume":"10","author":"Chandrasekar","year":"1997","journal-title":"Knowledge-Based Systems"},{"key":"2024041720204272700_bib11","article-title":"Cochrane \u2013 trusted evidence. informed decisions. better health","author":"The Cochrane Collaboration","year":"2023"},{"issue":"1","key":"2024041720204272700_bib12","first-page":"92","article-title":"What\u2019s so simple about simplified texts? A computational and psycholinguistic investigation of text comprehension and text processing.","volume":"26","author":"Crossley","year":"2014","journal-title":"Reading in a Foreign Language"},{"key":"2024041720204272700_bib13","doi-asserted-by":"publisher","first-page":"4972","DOI":"10.18653\/v1\/2021.naacl-main.395","article-title":"Paragraph-level simplification of medical texts","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devaraj","year":"2021"},{"key":"2024041720204272700_bib14","doi-asserted-by":"publisher","first-page":"7331","DOI":"10.18653\/v1\/2022.acl-long.506","article-title":"Evaluating factuality in text simplification","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Devaraj","year":"2022"},{"key":"2024041720204272700_bib15","doi-asserted-by":"publisher","first-page":"5055","DOI":"10.18653\/v1\/2020.acl-main.454","article-title":"FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Durmus","year":"2020"},{"key":"2024041720204272700_bib16","article-title":"Onestopenglish \u2013 A teacher resource site","author":"Education","year":"2023"},{"key":"2024041720204272700_bib17","doi-asserted-by":"publisher","first-page":"2587","DOI":"10.18653\/v1\/2022.naacl-main.187","article-title":"QAFactEval: Improved QA-based factual consistency evaluation for summarization","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Fabbri","year":"2022"},{"key":"2024041720204272700_bib18","article-title":"Sentence simplification via large language models","author":"Feng","year":"2023","journal-title":"arXiv preprint arXiv:2302.11957"},{"issue":"3","key":"2024041720204272700_bib19","first-page":"2007","article-title":"Flesch-Kincaid readability test","volume":"26","author":"Flesch","year":"2007","journal-title":"Retrieved October"},{"key":"2024041720204272700_bib20","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-49478-2_40","article-title":"Improving translation quality by manipulating sentence length","volume-title":"Conference of the Association for Machine Translation in the Americas (AMTA)","author":"Gerber","year":"1998"},{"key":"2024041720204272700_bib21","doi-asserted-by":"publisher","first-page":"380","DOI":"10.18653\/v1\/2021.conll-1.30","article-title":"Predicting text readability from scrolling interactions","volume-title":"Proceedings of the 25th Conference on Computational Natural Language Learning","author":"Gooding","year":"2021"},{"key":"2024041720204272700_bib22","first-page":"453","article-title":"Evaluation of automatic text simplification: Where are we now, where should we go from here","volume-title":"Actes de la 29e Conf\u00e9rence sur le Traitement Automatique des Langues Naturelles. Volume 1: Conf\u00e9rence principale","author":"Grabar","year":"2022"},{"key":"2024041720204272700_bib23","doi-asserted-by":"publisher","first-page":"5598","DOI":"10.18653\/v1\/2022.emnlp-main.378","article-title":"SimQA: Detecting simultaneous MT errors through word-by-word question answering","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Han","year":"2022"},{"issue":"Supplement C","key":"2024041720204272700_bib24","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1016\/j.csl.2016.12.001","article-title":"Source sentence simplification for statistical machine translation","volume":"45","author":"Hasler","year":"2017","journal-title":"Computer Speech & Language"},{"issue":"2","key":"2024041720204272700_bib25","first-page":"176","article-title":"On the effectiveness of strategy-based instruction of textual simplification on efl learners\u2019 reading comprehension ability","volume":"3","author":"Heydari","year":"2013","journal-title":"Indian Journal of Fundamental and Applied Life Sciences"},{"key":"2024041720204272700_bib26","article-title":"Newsela - online education platform for content","author":"Inc","year":"2023"},{"key":"2024041720204272700_bib27","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.709","article-title":"Neural crf model for sentence alignment in text simplification","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Jiang","year":"2020"},{"key":"2024041720204272700_bib28","doi-asserted-by":"publisher","first-page":"132","DOI":"10.18653\/v1\/2023.eacl-main.11","article-title":"Shortcomings of question answering based factuality frameworks for error localization","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Kamoi","year":"2023"},{"key":"2024041720204272700_bib29","doi-asserted-by":"publisher","first-page":"13291","DOI":"10.18653\/v1\/2023.emnlp-main.821","article-title":"BLESS: Benchmarking large language models on sentence simplification","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Kew","year":"2023"},{"key":"2024041720204272700_bib30","first-page":"495","article-title":"Just ask! Evaluating machine translation by asking and answering questions","volume-title":"Proceedings of the Sixth Conference on Machine Translation","author":"Krubi\u0144ski","year":"2021"},{"key":"2024041720204272700_bib31","doi-asserted-by":"publisher","first-page":"6365","DOI":"10.18653\/v1\/2021.acl-long.498","article-title":"Keep it simple: Unsupervised simplification of multi-paragraph text","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Laban","year":"2021"},{"issue":"2","key":"2024041720204272700_bib32","doi-asserted-by":"publisher","first-page":"ooac044","DOI":"10.1093\/jamiaopen\/ooac044","article-title":"Evaluation of an online text simplification editor using manual and automated metrics for perceived and actual text difficulty","volume":"5","author":"Leroy","year":"2022","journal-title":"JAMIA Open"},{"key":"2024041720204272700_bib33","doi-asserted-by":"publisher","first-page":"7871","DOI":"10.18653\/v1\/2020.acl-main.703","article-title":"BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Lewis","year":"2020"},{"key":"2024041720204272700_bib34","article-title":"Modifications that preserve language and content","author":"Long","year":"1993"},{"key":"2024041720204272700_bib35","doi-asserted-by":"publisher","first-page":"16383","DOI":"10.18653\/v1\/2023.acl-long.905","article-title":"LENS: A learnable evaluation metric for text simplification","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Maddela","year":"2023"},{"key":"2024041720204272700_bib36","first-page":"4689","article-title":"Controllable sentence simplification","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference","author":"Martin","year":"2020"},{"key":"2024041720204272700_bib37","first-page":"1651","article-title":"MUSS: Multilingual unsupervised sentence simplification by mining paraphrases","volume-title":"Proceedings of the Thirteenth Language Resources and Evaluation Conference","author":"Martin","year":"2022"},{"issue":"3","key":"2024041720204272700_bib38","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: The kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochemia Medica"},{"key":"2024041720204272700_bib39","doi-asserted-by":"publisher","first-page":"8488","DOI":"10.1609\/aaai.v34i05.6369","article-title":"Simplify-then- translate: Automatic preprocessing for black-box translation","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Mehta","year":"2020"},{"key":"2024041720204272700_bib40","first-page":"788","article-title":"Entity-focused sentence simplification for relation extraction","volume-title":"Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)","author":"Miwa","year":"2010"},{"key":"2024041720204272700_bib41","article-title":"Evaluating the suitability of human-oriented text simplification for machine translation","volume-title":"Proceedings of the 33rd Pacific Asia Conference on Language, Information and Computation","author":"Miyata","year":"2019"},{"issue":"1","key":"2024041720204272700_bib42","doi-asserted-by":"publisher","first-page":"69","DOI":"10.2307\/3587860","article-title":"Two types of input modification and efl reading comprehension: Simplification versus elaboration","volume":"35","author":"OH","year":"2001","journal-title":"TESOL Quarterly"},{"key":"2024041720204272700_bib43","doi-asserted-by":"publisher","first-page":"311","DOI":"10.3115\/1073083.1073135","article-title":"Bleu: A method for automatic evaluation of machine translation","volume-title":"Proceedings of the 40th annual meeting of the Association for Computational Linguistics","author":"Papineni","year":"2002"},{"key":"2024041720204272700_bib44","doi-asserted-by":"publisher","DOI":"10.21437\/SLaTE.2007-20","article-title":"Text simplification for language learners: A corpus analysis","volume-title":"Workshop on Speech and Language Technology in Education","author":"Petersen","year":"2007"},{"issue":"3","key":"2024041720204272700_bib45","doi-asserted-by":"publisher","first-page":"705","DOI":"10.1111\/jcal.12517","article-title":"To simplify or not? Facilitating English l2 users\u2019 comprehension and processing of open educational resources in english using text simplification","volume":"37","author":"Rets","year":"2021","journal-title":"Journal of Computer Assisted Learning"},{"key":"2024041720204272700_bib46","doi-asserted-by":"publisher","first-page":"712","DOI":"10.18653\/v1\/P18-2113","article-title":"Learning simplifications for specific target audiences","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Scarton","year":"2018"},{"key":"2024041720204272700_bib47","first-page":"3720","article-title":"Improving open relation extraction via sentence re-structuring","volume-title":"Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC\u201914)","author":"Schmidek","year":"2014"},{"key":"2024041720204272700_bib48","article-title":"Human evaluation for text simplification: The simplicity-adequacy tradeoff","volume-title":"SoCal NLP Symposium","author":"Schwarzer","year":"2018"},{"key":"2024041720204272700_bib49","doi-asserted-by":"publisher","first-page":"35","DOI":"10.18653\/v1\/N16-1005","article-title":"Controlling politeness in neural machine translation via side constraints","author":"Sennrich","year":"2016"},{"key":"2024041720204272700_bib50","doi-asserted-by":"publisher","first-page":"341","DOI":"10.18653\/v1\/2021.inlg-1.38","article-title":"Controllable sentence simplification with a unified text-to-text transfer transformer","volume-title":"Proceedings of the 14th International Conference on Natural Language Generation","author":"Sheang","year":"2021"},{"key":"2024041720204272700_bib51","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1007\/978-3-642-12320-7_5","article-title":"Translating from complex to simplified sentences","volume-title":"International Conference on Computational Processing of the Portuguese Language","author":"Specia","year":"2010"},{"key":"2024041720204272700_bib52","doi-asserted-by":"publisher","first-page":"2637","DOI":"10.18653\/v1\/2021.findings-acl.233","article-title":"Automatic text simplification for social good: Progress and challenges","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Stajner","year":"2021"},{"key":"2024041720204272700_bib53","first-page":"230","article-title":"Can text simplification help machine translation?","volume-title":"Proceedings of the 19th Annual Conference of the European Association for Machine Translation","author":"\u0160tajner","year":"2016"},{"key":"2024041720204272700_bib54","doi-asserted-by":"publisher","first-page":"39","DOI":"10.18653\/v1\/W18-7006","article-title":"Improving machine translation of English relative clauses with automatic text simplification","volume-title":"Proceedings of the 1st Workshop on Automatic Text Adaptation (ATA)","author":"\u0160tajner","year":"2018"},{"key":"2024041720204272700_bib55","doi-asserted-by":"publisher","first-page":"4208","DOI":"10.18653\/v1\/D18-1453","article-title":"What makes reading comprehension questions easier?","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Sugawara","year":"2018"},{"key":"2024041720204272700_bib56","doi-asserted-by":"publisher","first-page":"738","DOI":"10.18653\/v1\/D18-1081","article-title":"BLEU is not suitable for the evaluation of text simplification","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Sulem","year":"2018"},{"key":"2024041720204272700_bib57","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2021.gem-1.1","article-title":"Flesch-kincaid is not a text simplification evaluation metric","volume-title":"Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)","author":"Tanprasert","year":"2021"},{"key":"2024041720204272700_bib58","unstructured":"Irina\n              Temnikova\n             and GalinaManeva. 2013. The C-score \u2013 proposing a reading comprehension metrics as a common evaluation measure for text simplification. In Proceedings of the Second Workshop on Predicting and Improving Text Readability for Target Reader Populations, pages 20\u201329, Sofia, Bulgaria. Association for Computational Linguistics."},{"key":"2024041720204272700_bib59","first-page":"3007","article-title":"CLCM - a linguistic resource for effective simplification of instructions in the crisis management domain and its evaluations","volume-title":"Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC\u201912)","author":"Temnikova","year":"2012"},{"key":"2024041720204272700_bib60","article-title":"The effects of the amount and type of simplification on foreign language reading comprehension","author":"Tweissi","year":"1998"},{"key":"2024041720204272700_bib61","doi-asserted-by":"publisher","first-page":"297","DOI":"10.18653\/v1\/W18-0535","article-title":"OneStopEnglish corpus: A new corpus for automatic readability assessment and text simplification","volume-title":"Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications","author":"Vajjala","year":"2018"},{"key":"2024041720204272700_bib62","doi-asserted-by":"publisher","first-page":"349","DOI":"10.18653\/v1\/W19-4437","article-title":"On understanding the relation between expert annotations of text readability and target reader comprehension","volume-title":"Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications","author":"Vajjala","year":"2019"},{"key":"2024041720204272700_bib63","doi-asserted-by":"publisher","first-page":"4074","DOI":"10.18653\/v1\/2021.findings-emnlp.343","article-title":"How may I help you? Using neural text simplification to improve downstream NLP tasks","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Van","year":"2021"},{"key":"2024041720204272700_bib64","first-page":"344","article-title":"Sentence simplification for semantic role labeling","volume-title":"Proceedings of ACL-08: HLT","author":"Vickrey","year":"2008"},{"key":"2024041720204272700_bib65","doi-asserted-by":"publisher","first-page":"5008","DOI":"10.18653\/v1\/2020.acl-main.450","article-title":"Asking and answering questions to evaluate the factual consistency of summaries","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Wang","year":"2020"},{"key":"2024041720204272700_bib66","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1145\/1621995.1622002","article-title":"Facilita: Reading assistance for low-literacy readers","volume-title":"Proceedings of the 27th ACM international conference on Design of communication","author":"Watanabe","year":"2009"},{"key":"2024041720204272700_bib67","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1162\/tacl_a_00107","article-title":"Optimizing statistical machine translation for text simplification","volume":"4","author":"Wei","year":"2016","journal-title":"Transactions of the Association for Computational Linguistics"},{"issue":"3","key":"2024041720204272700_bib68","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1111\/0026-7902.00027","article-title":"Linguistic simplification of sl reading material: Effective instructional practice?","volume":"83","author":"Young","year":"1999","journal-title":"The Modern Language Journal"},{"key":"2024041720204272700_bib69","article-title":"Bertscore: Evaluating text generation with bert","volume-title":"International Conference on Learning Representations","author":"Zhang","year":"2020"},{"key":"2024041720204272700_bib70","doi-asserted-by":"publisher","first-page":"584","DOI":"10.18653\/v1\/D17-1062","article-title":"Sentence simplification with deep reinforcement learning","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Zhang","year":"2017"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/12\/00\/432\/2364120\/tacl_a_00653.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/12\/00\/432\/2364120\/tacl_a_00653.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,17]],"date-time":"2024-04-17T20:21:04Z","timestamp":1713385264000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/12\/00\/432\/120649\/Do-Text-Simplification-Systems-Preserve-Meaning-A"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":70,"journal-issue":{"issue":"00","published-print":{"date-parts":[[2024,4,16]]}},"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00653","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}