{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T04:27:08Z","timestamp":1789705628833,"version":"4.0.1"},"reference-count":56,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Operative notes are frequently mined for surgical concepts in clinical care, research, quality improvement, and billing, often requiring hours of manual extraction. These notes are typically analyzed at the document level to determine the presence or absence of specific procedures or findings (eg, whether a hand-sewn anastomosis was performed or contamination occurred). Extracting several binary classification labels simultaneously is a multilabel classification problem. Traditional natural language processing approaches\u2014bag-of-words (BoW) and term frequency-inverse document frequency (tf-idf) with linear classifiers\u2014have been used previously for this task but are now being augmented or replaced by large language models (LLMs). However, few studies have examined their utility in surgery.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n We developed and evaluated LLMs for the purpose of expediting data extraction from surgical notes.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n A total of 388 exploratory laparotomy notes from a single institution were annotated for 21 concepts related to intraoperative findings, intraoperative techniques, and closure techniques. Annotation consistency was measured using the Cohen \u03ba statistic. Data were preprocessed to include only the description of the procedure. We compared the evolution of document classification technologies from BoW and tf-idf to encoder-only (Clinical-Longformer) and decoder-only (Llama 3) transformer models. Multilabel classification performance was evaluated with 5-fold cross-validation with\n F<\/jats:italic>\n 1<\/jats:sub>\n -score and hamming loss (HL). We experimented with and without context. Errors were assessed by manual review. Code and implementation instructions may be found on GitHub.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n The prevalence of labels ranged from 0.05 (colostomy, ileostomy, active bleed from named vessel) to 0.50 (running fascial closure). Llama 3.3 was the overall best-performing model (micro\n F<\/jats:italic>\n 1<\/jats:sub>\n -score 0.88, 5-fold range: 0.88-0.89; HL 0.11, 5-fold range: 0.11-0.12). The BoW model (micro\n F<\/jats:italic>\n 1<\/jats:sub>\n -score 0.68, 5-fold range: 0.64-0.71; HL 0.14, 5-fold range: 0.13-0.16) and Clinical-Longformer (micro\n F<\/jats:italic>\n 1<\/jats:sub>\n -score 0.73, 5-fold range: 0.70-0.74; HL 0.11, 5-fold range: 0.10-0.12) had overall similar performance, with tf-idf models trailing (micro\n F<\/jats:italic>\n 1<\/jats:sub>\n -score 0.57, 5-fold range: 0.55-0.59; HL 0.27, 5-fold range: 0.25-0.29).\n F<\/jats:italic>\n 1<\/jats:sub>\n -scores varied across concepts in the Llama model, ranging from 0.30 (5-fold range: 0.23-0.39) for class III contamination to 0.92 (5-fold range: 0.98-0.84) for bowel resection. Context enhanced Llama\u2019s performance, adding an average of 0.16 improvement to the\n F<\/jats:italic>\n 1<\/jats:sub>\n -scores. Error analysis demonstrated semantic nuances and edge cases within operative notes, particularly when patients had references to prior operations in their operative notes or simultaneous operations with other surgical services.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Off-the-shelf autoregressive LLMs outperformed fined-tuned, encoder-only transformers and traditional natural language processing techniques in classifying operative notes. Multilabel classification with LLMs may streamline retrospective reviews in surgery, though further refinements are required prior to reliable use in research and quality improvement.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/71176","type":"journal-article","created":{"date-parts":[[2025,7,9]],"date-time":"2025-07-09T17:52:24Z","timestamp":1752083544000},"page":"e71176-e71176","source":"Crossref","is-referenced-by-count":7,"title":["Language Models for Multilabel Document Classification of Surgical Concepts in Exploratory Laparotomy Operative Notes: Algorithm Development Study"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1826-7884","authenticated-orcid":false,"given":"Jeremy A","family":"Balch","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0880-5122","authenticated-orcid":false,"given":"Sasank S","family":"Desaraju","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6069-963X","authenticated-orcid":false,"given":"Victoria J","family":"Nolan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3151-9804","authenticated-orcid":false,"given":"Divya","family":"Vellanki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-7815-7967","authenticated-orcid":false,"given":"Timothy R","family":"Buchanan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0390-5310","authenticated-orcid":false,"given":"Lindsey M","family":"Brinkley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8520-9417","authenticated-orcid":false,"given":"Yordan","family":"Penev","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5449-2114","authenticated-orcid":false,"given":"Ahmet","family":"Bilgili","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3301-8066","authenticated-orcid":false,"given":"Aashay","family":"Patel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3184-2278","authenticated-orcid":false,"given":"Corinne E","family":"Chatham","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8468-5833","authenticated-orcid":false,"given":"David M","family":"Vanderbilt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-3010-6048","authenticated-orcid":false,"given":"Rayon","family":"Uddin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5745-2863","authenticated-orcid":false,"given":"Azra","family":"Bihorac","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3931-650X","authenticated-orcid":false,"given":"Philip","family":"Efron","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5354-443X","authenticated-orcid":false,"given":"Tyler J","family":"Loftus","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0000-8558","authenticated-orcid":false,"given":"Protiva","family":"Rahman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5304-7027","authenticated-orcid":false,"given":"Benjamin","family":"Shickel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,7,9]]},"reference":[{"issue":"821-6","key":"R1","first-page":"821","volume":"216","author":"Melton","journal-title":"Stud Health Technol Inform"},{"key":"R2","doi-asserted-by":"crossref","unstructured":"Maldonado R Goodwin T Harabagiu SM Skinner MA . The role of semantic and discourse information in learning the structure of surgical procedures. Presented at: 2015 International Conference on Healthcare Informatics; Oct 21-23, 2015; Dallas, TX, USA. [doi: 10.1109\/ICHI.2015.34]","DOI":"10.1109\/ICHI.2015.34"},{"issue":"2","key":"R3","doi-asserted-by":"publisher","first-page":"1015","DOI":"10.1007\/s13042-024-02318-w","article-title":"Large language models for medicine: a survey","volume":"16","author":"Zheng","journal-title":"Int J Mach Learn Cyber"},{"key":"R4","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.n71","article-title":"The PRISMA 2020 statement: an updated guideline for reporting systematic reviews","volume":"372","author":"Page","journal-title":"BMJ"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1542\/hpeds.2019-0225","article-title":"Structured chart review: assessment of a structured chart review methodology","volume":"10","author":"Siems","journal-title":"Hosp Pediatr"},{"issue":"5","key":"R6","doi-asserted-by":"publisher","first-page":"236","DOI":"10.3390\/a16050236","article-title":"Twenty years of machine-learning-based text classification: a systematic review","volume":"16","author":"Palanivinayagam","journal-title":"Algorithms"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2024.3349952","article-title":"A survey of text classification with transformers: how wide? How large? How long? How accurate? How expensive? How safe?","volume":"12","author":"Fields","journal-title":"IEEE Access"},{"key":"R8","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2022.104050","article-title":"Implementation of specialised attention mechanisms: ICD-10 classification of gastrointestinal discharge summaries in English, Spanish and Swedish","volume":"130","author":"Blanco","journal-title":"J Biomed Inform"},{"issue":"8","key":"R9","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0270595","article-title":"Multi-label classification of symptom terms from free-text bilingual adverse drug reaction reports using natural language processing","volume":"17","author":"Chaichulee","journal-title":"PLoS One"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1097\/SLA.0000000000006107","article-title":"TraumaICD bidirectional encoder representation from transformers: a natural language processing algorithm to extract injury International Classification of Diseases, 10th edition diagnosis code from free text","volume":"280","author":"Choi","journal-title":"Ann Surg"},{"key":"R11","doi-asserted-by":"crossref","unstructured":"Pedersen JS Laursen MS Soguero-Ruiz C Savarimuthu TR Hansen RS Vinholt PJ . Domain over size: clinical ELECTRA surpasses general BERT for bleeding site classification in the free text of electronic health records. Presented at: 2022 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI); Sep 27-30, 2022; Ioannina, Greece. [doi: 10.1109\/BHI56158.2022.9926955]","DOI":"10.1109\/BHI56158.2022.9926955"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","first-page":"1063","DOI":"10.1109\/TETCI.2024.3423444","article-title":"From bag-of-words to transformers: a comparative study for text classification in healthcare discussions in social media","volume":"9","author":"De Santis","journal-title":"IEEE Trans Emerg Top Comput Intell"},{"issue":"4","key":"R13","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1001\/jama.2024.21700","article-title":"Testing and evaluation of health care applications of large language models: a systematic review","volume":"333","author":"Bedi","journal-title":"JAMA"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1038\/s41746-023-00879-8","article-title":"The shaky foundations of large language models and foundation models for electronic health records","volume":"6","author":"Wornow","journal-title":"NPJ Digital Med"},{"issue":"1","key":"R15","doi-asserted-by":"publisher","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","journal-title":"Sci Data"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Agrawal M Hegselmann S Lang H Kim Y Sontag D . Large language models are few-shot clinical information extractors. arXiv. Preprint posted online on May 25, 2022. [doi: 10.48550\/arXiv.2205.12689]","DOI":"10.18653\/v1\/2022.emnlp-main.130"},{"issue":"7","key":"R17","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2020-048008","article-title":"Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence","volume":"11","author":"Collins","journal-title":"BMJ Open"},{"issue":"3","key":"R18","first-page":"276","volume":"22","author":"McHugh","journal-title":"Biochem Med (Zagreb)"},{"key":"R19","unstructured":"Dimick JB Upchurch GR Alam HB et al. Greenfield\u2019s Surgery: Scientific Principles and Practice. Lippincott Williams & Wilkins; 2021. ISBN: 1975143183"},{"key":"R20","doi-asserted-by":"crossref","unstructured":"Sechidis K Tsoumakas G Vlahavas I . On the stratification of multi-label data. Presented at: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2011; Sep 5-9, 2011; Athens, Greece.","DOI":"10.1007\/978-3-642-23808-6_10"},{"key":"R21","unstructured":"Szyma\u0144ski P Kajdanowicz T . A network perspective on stratification of multi-label data. Presented at: First International Workshop on Learning with Imbalanced Domains: Theory and Applications; Sep 22, 2017; Skopje, Macedonia."},{"key":"R22","unstructured":"Szyma\u0144ski P Kajdanowicz T . A scikit-based python environment for performing multi-label classification. arXiv. Preprint posted online on Feb 5, 2017. [doi: 10.48550\/arXiv.1702.01460]"},{"issue":"2","key":"R23","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1093\/jamia\/ocac225","article-title":"A comparative study of pretrained language models for long clinical text","volume":"30","author":"Li","journal-title":"J Am Med Inform Assoc"},{"key":"R24","unstructured":"Dubey A Jauhri A Pandey A et al. The llama 3 herd of models. arXiv. Preprint posted online on Jul 31, 2024. [doi: 10.48550\/arXiv.2407.21783]"},{"issue":"7972","key":"R25","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","journal-title":"Nat New Biol"},{"issue":"9","key":"R26","doi-asserted-by":"publisher","first-page":"3596","DOI":"10.1109\/JBHI.2021.3062322","article-title":"Limitations of transformers on clinical text classification","volume":"25","author":"Gao","journal-title":"IEEE J Biomed Health Inform"},{"key":"R27","unstructured":"Beltagy I Peters ME Cohan A . Longformer: the long-document transformer. arXiv. Preprint posted online on Apr 10, 2020. [doi: 10.48550\/arXiv.2004.05150]"},{"key":"R28","unstructured":"Devlin J Chang MW Lee K Toutanova K . BERT: pre-training of deep bidirectional transformers for language understanding. Presented at: 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; Jun 2-7, 2019; Minneapolis, Minnesota, United States."},{"key":"R29","unstructured":"Huang K Altosaar J Ranganath R . ClinicalBERT: modeling clinical notes and predicting hospital readmission. arXiv. Preprint posted online on Apr 10, 2019. [doi: 10.48550\/arXiv.1904.05342]"},{"issue":"4","key":"R30","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","journal-title":"Bioinformatics"},{"key":"R31","doi-asserted-by":"crossref","unstructured":"Wolf T Debut L Sanh V Chaumond J Delangue C Moi A et al. HuggingFace\u2019s transformers: state-of-the-art natural language processing. arXiv. Preprint posted online on Oct 9, 2019. [doi: 10.48550\/arXiv.1910.03771]","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"R32","unstructured":"Liu J . LlamaIndex. Github. 2022. URL: https:\/\/github.com\/jerryjliu\/llama_index [Accessed 15-09-2024]"},{"key":"R33","unstructured":"Balch JA Desaraju S . Multi-label classification for operative notes. GitHub. 2024. URL: https:\/\/github.com\/balchja\/op_note_multi_label_classification [Accessed 15-09-2024]"},{"key":"R34","unstructured":"Pal A . Performance comparison: llama-3 models in medical and healthcare AI domains. Hugg Face Repository. 2024. URL: https:\/\/huggingface.co\/blog\/aaditya\/llama3-in-medical-domain [Accessed 15-09-2024]"},{"issue":"8","key":"R35","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2016-011078","article-title":"Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review","volume":"6","author":"Hanskamp-Sebregts","journal-title":"BMJ Open"},{"issue":"3","key":"R36","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1258\/135581907781543012","article-title":"Inter-rater reliability of case-note audit: a systematic review","volume":"12","author":"Lilford","journal-title":"J Health Serv Res Policy"},{"issue":"1","key":"R37","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1038\/s43856-023-00370-1","article-title":"The future landscape of large language models in medicine","volume":"3","author":"Clusmann","journal-title":"Commun Med (Lond)"},{"issue":"8","key":"R38","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","journal-title":"Nat Med"},{"key":"R39","unstructured":"Soni S Roberts K , editors. Evaluation of dataset selection for pre-training and fine-tuning transformer language models for clinical question answering. Presented at: Proceedings of the Twelfth Language Resources and Evaluation Conference; May 11-16, 2020; Marseille, France."},{"issue":"1","key":"R40","doi-asserted-by":"publisher","DOI":"10.1056\/AIra2400380","article-title":"RAG in health care: a novel framework for improving communication and decision-making by addressing LLM limitations","volume":"2","author":"Ng","journal-title":"NEJM AI"},{"issue":"9","key":"R41","doi-asserted-by":"publisher","first-page":"1964","DOI":"10.1093\/jamia\/ocae131","article-title":"Reasoning with large language models for medical question answering","volume":"31","author":"Lucas","journal-title":"J Am Med Inform Assoc"},{"key":"R42","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-3483777\/v1","article-title":"Clinical text summarization: adapting large language models can outperform human experts","author":"Van Veen","journal-title":"Res Sq"},{"issue":"18","key":"R43","doi-asserted-by":"publisher","first-page":"8319","DOI":"10.3390\/app11188319","article-title":"A survey on recent named entity recognition and relationship extraction techniques on clinical texts","volume":"11","author":"Bose","journal-title":"Appl Sci (Basel)"},{"key":"R44","doi-asserted-by":"crossref","unstructured":"Pagad NS Pradeep N , editors. Clinical named entity recognition methods: an overview. In: International Conference on Innovative Computing and Communications: Proceedings of ICICC. Vol 2. Springer; 2022. [doi: 10.1007\/978-981-16-2597-8_13]","DOI":"10.1007\/978-981-16-2597-8_13"},{"key":"R45","volume":"2017","author":"Wu","journal-title":"AMIA Annu Symp Proc"},{"issue":"9","key":"R46","doi-asserted-by":"publisher","first-page":"1929","DOI":"10.1093\/jamia\/ocae095","article-title":"RT: a retrieving and chain-of-thought framework for few-shot medical named entity recognition","volume":"31","author":"Li","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"R47","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1038\/s41597-019-0322-0","article-title":"MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports","volume":"6","author":"Johnson","journal-title":"Sci Data"},{"key":"R48","doi-asserted-by":"publisher","DOI":"10.1109\/BIBM.2018.8621574","article-title":"Early prediction of acute kidney injury in critical care setting using clinical notes","author":"Li","journal-title":"Proc IEEE Int Conf Bioinformatics Biomed"},{"issue":"10","key":"R49","doi-asserted-by":"publisher","first-page":"2181","DOI":"10.1093\/jamia\/ocae210","article-title":"Evaluating large language models for health-related text classification tasks with public social media data","volume":"31","author":"Guo","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"R50","doi-asserted-by":"publisher","first-page":"900","DOI":"10.1097\/SLA.0000000000004419","article-title":"Natural language processing in surgery: a systematic review and meta-analysis","volume":"273","author":"Mellia","journal-title":"Ann Surg"},{"issue":"6","key":"R51","doi-asserted-by":"publisher","first-page":"1300","DOI":"10.1016\/j.surg.2018.05.008","article-title":"Autonomous detection, grading, and reporting of postoperative complications using natural language processing","volume":"164","author":"Selby","journal-title":"Surgery"},{"issue":"10","key":"R52","doi-asserted-by":"publisher","first-page":"2216","DOI":"10.1016\/j.arth.2019.07.025","article-title":"Use of natural language processing tools to identify and classify periprosthetic femur fractures","volume":"34","author":"Tibbo","journal-title":"J Arthroplasty"},{"issue":"1","key":"R53","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/s41060-023-00433-5","article-title":"Surgicberta: a pre-trained language model for procedural surgical language","volume":"18","author":"Bombieri","journal-title":"Int J Data Sci Anal"},{"key":"R54","unstructured":"Jurafsky D Martin JH . Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition with language models. Stanford University. 2024. URL: https:\/\/web.stanford.edu\/~jurafsky\/slp3\/ [Accessed 06-06-2025]"},{"key":"R55","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2024.104662","article-title":"Applying generative AI with retrieval augmented generation to summarize and extract key clinical information from electronic health records","volume":"156","author":"Alkhalaf","journal-title":"J Biomed Inform"},{"key":"R56","volume":"33","author":"Lewis","journal-title":"Adv Neural Inf Process Syst"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e71176","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:00:22Z","timestamp":1782842422000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e71176"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,9]]},"references-count":56,"URL":"https:\/\/doi.org\/10.2196\/71176","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,9]]},"article-number":"v13i821-6e71176"}}