{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T23:08:35Z","timestamp":1784243315615,"version":"3.55.0"},"reference-count":150,"publisher":"Association for Computing Machinery (ACM)","issue":"4","funder":[{"name":"Component 5\u2014Capitalization and Business Innovation"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Healthcare"],"published-print":{"date-parts":[[2025,10,31]]},"abstract":"<jats:p>Electronic Health Records store extensive patient health data, playing a crucial role in healthcare management. Extracting information from these text-heavy records is difficult due to their domain-specific vocabulary, which challenges applying general-domain techniques. Recent advancements in Large Language Models (LLMs) and an increasing interest in the field have sparked considerable progress in solving Clinical Information Extraction (IE) tasks. We review these applications in Clinical IE, highlighting the most common tasks, most successful methods, and most used datasets and evaluation criteria. Examining 85 studies, we synthesize and organize the current research trends, highlighting common points between papers. The presence of LLMs can be felt in the most common tasks, with novel approaches being attempted and showing promising results. However, breakthroughs are still necessary in designing reliable end-to-end systems that can perform all the Clinical IE tasks within a single system.<\/jats:p>","DOI":"10.1145\/3744660","type":"journal-article","created":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T08:12:41Z","timestamp":1750666361000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Harnessing Large Language Models for Clinical Information Extraction: A Systematic Literature Review"],"prefix":"10.1145","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-0178-3901","authenticated-orcid":false,"given":"Tiago","family":"Rodrigues","sequence":"first","affiliation":[{"name":"Faculty of Engineering, University of Porto, Porto, Portugal and INESC TEC, Porto, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4202-791X","authenticated-orcid":false,"given":"Carla","family":"Teixeira Lopes","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Porto, Porto, Portugal and INESC TEC, Porto, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,10,13]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-021-01633-4"},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","unstructured":"Monica Agrawal Stefan Hegselmann Hunter Lang Yoon Kim and David Sontag. 2022. Large language models are few-shot clinical information extractors. arXiv:2205.12689. Retrieved from https:\/\/arxiv.org\/abs\/2205.12689","DOI":"10.18653\/v1\/2022.emnlp-main.130"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2023.41.16_suppl.6607"},{"key":"e_1_3_2_5_2","unstructured":"Haifa Alrdahi Lifeng Han Hendrik \u0160uvalov and Goran Nenadic. 2023. MedMine: Examining pre-trained language models on medication mining. arXiv:2308.03629. Retrieved from https:\/\/arxiv.org\/abs\/2308.03629"},{"key":"e_1_3_2_6_2","unstructured":"Paul Barry Sam Henry Meliha Yetisgen Bridget McInnes and Ozlem Uzuner. 2021. Jointly learning clinical entities and relations with contextual language models and explicit context. arXiv:2102.11031. Retrieved from https:\/\/arxiv.org\/abs\/2102.11031"},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","unstructured":"Samuel Belkadi Lifeng Han Yuping Wu and Goran Nenadic. 2023. Exploring the value of pre-trained language models for clinical named entity recognition. arXiv:2210.12770. Retrieved from https:\/\/arxiv.org\/abs\/2210.12770","DOI":"10.1109\/BigData59044.2023.10386154"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S16-1165"},{"key":"e_1_3_2_9_2","unstructured":"Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et\u00a0al. 2020. Language models are few-shot learners. arXiv:2005.14165. Retrieved from https:\/\/arxiv.org\/abs\/2005.14165"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocab084"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-022-02059-2"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.3857\/roj.2023.00633"},{"key":"e_1_3_2_13_2","first-page":"73","volume-title":"International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications (NLPBA\/BioNLP)","author":"Collier Nigel","year":"2004","unstructured":"Nigel Collier, Tomoko Ohta, Yoshimasa Tsuruoka, Yuka Tateisi, and Jin-Dong Kim. 2004. Introduction to the bio-entity recognition task at JNLPBA. In International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications (NLPBA\/BioNLP). Nigel Collier, Patrick Ruch, and Adeline Nazarenko (Eds.), COLING, 73\u201378."},{"key":"e_1_3_2_14_2","first-page":"36","volume-title":"Actes de La 6e Conf\u00e9rence Conjointe Journ\u00e9es d\u2019\u00c9tudes Sur La Parole (JEP, 33e \u00c9dition), Traitement Automatique Des Langues Naturelles (TALN, 27e \u00c9dition), Rencontre Des \u00c9tudiants Chercheurs En Informatique Pour Le Traitement Automatique Des Langues (R\u00c9CITAL, 22e \u00c9dition)","author":"Copara Jenny","year":"2020","unstructured":"Jenny Copara, Julien Knafou, Nona Naderi, Claudia Moro, Patrick Ruch, and Douglas Teodoro. 2020. Contextualized French language models for biomedical named entity recognition. In Actes de La 6e Conf\u00e9rence Conjointe Journ\u00e9es d\u2019\u00c9tudes Sur La Parole (JEP, 33e \u00c9dition), Traitement Automatique Des Langues Naturelles (TALN, 27e \u00c9dition), Rencontre Des \u00c9tudiants Chercheurs En Informatique Pour Le Traitement Automatique Des Langues (R\u00c9CITAL, 22e \u00c9dition). Atelier D\u00c9fi Fouille de Textes. R\u00e9mi Cardon, Natalia Grabar, Cyril Grouin, and Thierry Hamon (Eds.), ATALA et AFCP, 36\u201348."},{"key":"e_1_3_2_15_2","volume-title":"CEUR Workshop Proceedings","author":"Costa Joao","year":"2020","unstructured":"Joao Costa, Ines Lopes, Andre Carreiro, and David Ribeiro. 2020. Fraunhofer AICOS at CLEF eHealth 2020 Task 1: Clinical code extraction from textual data using fine-tuned BERT models. In CEUR Workshop Proceedings. CEUR, 14 pages."},{"key":"e_1_3_2_16_2","first-page":"19","volume-title":"Conference and Labs of the Evaluation Forum","author":"Cotik Viviana","year":"2021","unstructured":"Viviana Cotik, L. A. Alemany, D. Filippo, Franco Luque, Roland Roller, J. Vivaldi, Ammer Ayach, Fernando Carranza, Lucas Defrancesca, Antonella Dellanzo, and Macarena Fern\u00e1ndez Urquiza. 2021. Overview of CLEF eHealth Task 1\u2014SpRadIE: A challenge on information extraction from Spanish radiology reports. In Conference and Labs of the Evaluation Forum. CEUR, 19 pages."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-srw.27"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.imu.2023.101381"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CISP-BMEI48845.2019.8965823"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324920000352"},{"key":"e_1_3_2_21_2","first-page":"366","article-title":"A multilabel approach to Portuguese clinical named entity recognition","volume":"12","author":"de Souza Jo\u00e3o Vitor Andrioli","year":"2020","unstructured":"Jo\u00e3o Vitor Andrioli de Souza, Elisa Terumi Rubel Schneider, Josilaine Oliveira Cezar, Lucas Emanuel Silva e Oliveira, Yohan Bonescki Gumiel, Emerson Cabrera Paraiso, Douglas Teodoro, and Claudia Maria Cabral Moro Barra. 2020. A multilabel approach to Portuguese clinical named entity recognition. Journal of Health Informatics 12 (2020), 366\u2013372.","journal-title":"Journal of Health Informatics"},{"key":"e_1_3_2_22_2","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. Retrieved from https:\/\/arxiv.org\/abs\/1810.04805"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.clinicalnlp-1.12"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2013.12.006"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.clinicalnlp-1.14"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocaa218"},{"key":"e_1_3_2_27_2","first-page":"6","volume-title":"Proceedings of DEIM","author":"Fang Xintao","year":"2021","unstructured":"Xintao Fang, Yuting Song, and Akira Maeda. 2021. Joint entity and relation extraction from clinical records using pre-trained language model. In Proceedings of DEIM. DEIM, 6 pages."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404555.3404635"},{"key":"e_1_3_2_29_2","first-page":"2979","volume-title":"International Conference on Computational Linguistics 2022 (COLING)","author":"Gao Yanjun","year":"2022","unstructured":"Yanjun Gao, Timothy Miller, Dongfang Xu, Dmitriy Dligach, Matthew M. Churpek, and Majid Afshar. 2022. Summarizing patients\u2019 problems from hospital progress notes using pre-trained sequence-to-sequence models. In International Conference on Computational Linguistics 2022 (COLING), 2979\u20132991."},{"key":"e_1_3_2_30_2","unstructured":"John Giorgi Xindi Wang Nicola Sahar Won Young Shin Gary D. Bader and Bo Wang. 2019. End-to-end named entity recognition and relation extraction using pre-trained language models. arXiv:1912.13415. Retrieved from https:\/\/arxiv.org\/abs\/1912.13415"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-5701"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-5614"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.10103750"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICHI52183.2021.00090"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2023.1266771"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2012.04.008"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1055\/s-0040-1702001"},{"key":"e_1_3_2_38_2","first-page":"6","volume-title":"CEUR Workshop Proceedings","author":"Haq H.","year":"2021","unstructured":"H. Haq, Veysel Kocaman, and D. Talby. 2021. Deeper clinical document understanding using relation extraction. In CEUR Workshop Proceedings. CEUR, 6 pages."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpbup.2021.100042"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI50040.2020.00072"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz166"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2013.07.011"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.12688\/f1000research.27337.3"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-emnlp.204"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","unstructured":"Mark Iscoe Vimig Socrates Aidan Gilson Ling Chi Huan Li Thomas Huang Thomas Kearns Rachelle Perkins Laura Khandjian and R. Andrew Taylor. 2023. Identifying signs and symptoms of urinary tract infection from emergency department clinical notes using large language models. DOI: 10.1101\/2023.10.20.23297156","DOI":"10.1101\/2023.10.20.23297156"},{"key":"e_1_3_2_46_2","first-page":"30","volume-title":"TERM21 Workshop","author":"Jerdhaf Oskar","year":"2022","unstructured":"Oskar Jerdhaf, Marina Santini, Peter Lundberg, Tomas Bjerner, Yosef Al-Abasse, Arne Jonsson, and Thomas Vakili. 2022. Evaluating pre-trained language models for focused terminology extraction from Swedish medical records. In TERM21 Workshop. LREC, 30\u201332."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","unstructured":"Alistair E. W. Johnson Tom J. Pollard Lu Shen Li-wei H. Lehman Mengling Feng Mohammad Ghassemi Benjamin Moody Peter Szolovits Leo Anthony Celi and Roger G. Mark. 2016. MIMIC-III a freely accessible critical care database. Scientific Data 3 1 (May 2016) 160035. DOI: 10.1038\/sdata.2016.35","DOI":"10.1038\/sdata.2016.35"},{"key":"e_1_3_2_48_2","volume-title":"Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition","author":"Jurafsky Dan","year":"2009","unstructured":"Dan Jurafsky and James H. Martin. 2009. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (2nd ed.). Pearson Prentice Hall, Upper Saddle River, NJ.","edition":"2"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12967-023-04011-y"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3584741"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-020-01241-8"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1002\/joe.21554"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-33080-3_30"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICHI52183.2021.00032"},{"key":"e_1_3_2_55_2","first-page":"141","volume-title":"BioCreative VI Challenge Evaluation Workshop","author":"Krallinger Martin","year":"2017","unstructured":"Martin Krallinger, Obdulia Rabal, S. Akhondi, Mart\u00edn P\u00e9rez P\u00e9rez, J. Santamar\u00eda, Gael P\u00e9rez Rodr\u00edguez, G. Tsatsaronis, Ander Intxaurrondo, Jos\u00e9 Antonio Baso L\u00f3pez, U. Nandal, et\u00a0al. 2017. Overview of the BioCreative VI chemical-protein interaction track. In BioCreative VI Challenge Evaluation Workshop. BioCreative, 141\u2013146."},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.09.018"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.805"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/InC457730.2023.10263169"},{"key":"e_1_3_2_59_2","unstructured":"Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma and Radu Soricut. 2020. ALBERT: A Lite BERT for self-supervised learning of language representations. arXiv:1909.11942. Retrieved from https:\/\/arxiv.org\/abs\/1909.11942"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-022-01779-1"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btac297"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10278-022-00717-5"},{"key":"e_1_3_2_63_2","doi-asserted-by":"crossref","unstructured":"Brian Lester Rami Al-Rfou and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv:2104.08691. Retrieved from https:\/\/arxiv.org\/abs\/2104.08691","DOI":"10.18653\/v1\/2021.emnlp-main.243"},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","unstructured":"Mike Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdelrahman Mohamed Omer Levy Ves Stoyanov and Luke Zettlemoyer. 2019. BART: Denoising sequence-to-sequence pre-training for natural language generation translation and comprehension. arXiv:1910.13461. Retrieved from https:\/\/arxiv.org\/abs\/1910.13461","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.clinicalnlp-1.17"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1093\/database\/baw068"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.21037\/qims-21-90"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2020.103422"},{"key":"e_1_3_2_69_2","unstructured":"Yikuan Li Ramsey M. Wehbe Faraz S. Ahmad Hanyin Wang and Yuan Luo. 2022. Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequences. arXiv:2201.11838. Retrieved from https:\/\/arxiv.org\/abs\/2201.11838"},{"key":"e_1_3_2_70_2","first-page":"243","article-title":"NLP applied to occupational health: MEDDOPROF shared task at IberLEF 2021 on automatic recognition, classification and normalization of professions and occupations from medical texts","volume":"67","author":"Lima-L\u00f3pez Salvador","year":"2021","unstructured":"Salvador Lima-L\u00f3pez, Eul\u00e0lia Farr\u00e9-Maduell, Antonio Miranda-Escalada, Vicent Briv\u00e1-Iglesias, and Martin Krallinger. 2021. NLP applied to occupational health: MEDDOPROF shared task at IberLEF 2021 on automatic recognition, classification and normalization of professions and occupations from medical texts. Procesamiento del Lenguaje Natural 67 (Sept. 2021), 243\u2013256.","journal-title":"Procesamiento del Lenguaje Natural"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-1908"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.bionlp-1.7"},{"key":"e_1_3_2_73_2","first-page":"74","volume-title":"Text Summarization Branches Out","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out. Association for Computational Linguistics, 74\u201381."},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2021.3131180"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbac282"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.4000\/books.aaccademia.8663"},{"key":"e_1_3_2_77_2","first-page":"833","volume-title":"AMIA Annual Symposium Proceedings","author":"Mahajan Diwakar","year":"2022","unstructured":"Diwakar Mahajan, Jennifer J. Liang, and Ching-Huei Tsou. 2022. Toward understanding clinical context of medication change events in clinical narratives. In AMIA Annual Symposium Proceedings 2021 (Feb. 2022), 833\u2013842."},{"key":"e_1_3_2_78_2","first-page":"420","article-title":"Extracting adverse drug events from clinical notes","volume":"2021","author":"Mahendran Darshini","year":"2021","unstructured":"Darshini Mahendran and Bridget T. McInnes. 2021. Extracting adverse drug events from clinical notes. In AMIA Summits on Translational Science Proceedings 2021 (May 2021), 420\u2013429.","journal-title":"AMIA Summits on Translational Science Proceedings"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511809071"},{"key":"e_1_3_2_80_2","doi-asserted-by":"crossref","unstructured":"Aidan Mannion Thierry Chevalier Didier Schwab and Lorraine Geouriot. 2023. UMLS-KGI-BERT: Data-centric knowledge integration in transformers for biomedical entity recognition. arXiv:2307.11170. Retrieved from https:\/\/arxiv.org\/abs\/2307.11170","DOI":"10.18653\/v1\/2023.clinicalnlp-1.35"},{"key":"e_1_3_2_81_2","first-page":"618","volume-title":"IberLEF 2019 Proceedings","author":"Marimon Montserrat","year":"2019","unstructured":"Montserrat Marimon, Aitor Gonzalez-Agirre, Ander Intxaurrondo, Jose Antonio Lopez Martin, and Marta Villegas. 2019. Automatic de-identification of medical texts in Spanish: The MEDDOCAN track, corpus, guidelines, methods and evaluation of results. In IberLEF 2019 Proceedings. CEUR, 618\u2013638."},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/CIBCB49929.2021.9562815"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.bionlp-1.15"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.139"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","unstructured":"Antonio Miranda-Escalada Eul\u00e0lia Farr\u00e9 and Martin Krallinger. 2020. Cantemist corpus: Gold standard of oncology clinical cases annotated with CIE-O 3 terminology. DOI: 10.5281\/zenodo.3978041","DOI":"10.5281\/zenodo.3978041"},{"key":"e_1_3_2_86_2","first-page":"29","volume-title":"CLEF 2020 Working Notes","author":"Miranda-Escalada Antonio","year":"2020","unstructured":"Antonio Miranda-Escalada, Aitor Gonzalez-Agirre, Jordi Armengol-Estap\u00e9, and Martin Krallinger. 2020. Overview of automatic clinical coding: Annotations, guidelines, and solutions for non-English clinical cases at CodiEsp track of CLEF eHealth 2020. In CLEF 2020 Working Notes. CEUR, 29."},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocab126"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2021.103960"},{"key":"e_1_3_2_89_2","unstructured":"Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes and Ajmal Mian. 2024. A comprehensive overview of large language models. arXiv:2307.06435. Retrieved from https:\/\/arxiv.org\/abs\/2307.06435"},{"key":"e_1_3_2_90_2","first-page":"24","volume-title":"4th Workshop on Building and Evaluating Resources for Health and Biomedical Text Processing (BioTxtM \u201914)","author":"N\u00e9v\u00e9ol Aur\u00e9lie","year":"2014","unstructured":"Aur\u00e9lie N\u00e9v\u00e9ol, Cyril Grouin, J\u00e9r\u00e9my Leixa, S. Rosset, and Pierre Zweigenbaum. 2014. The QUAERO French Medical Corpus: A resource for medical entity recognition and normalization. In 4th Workshop on Building and Evaluating Resources for Health and Biomedical Text Processing (BioTxtM \u201914). LREC, 24\u201330."},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13326-022-00269-1"},{"key":"e_1_3_2_92_2","unstructured":"OpenAI. 2023. GPT-4 technical report. arXiv:2303.08774. Retrieved from https:\/\/arxiv.org\/abs\/2303.08774"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.n71"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.n160"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073135"},{"key":"e_1_3_2_96_2","doi-asserted-by":"crossref","unstructured":"Cheng Peng Xi Yang Kaleb E. Smith Zehao Yu Aokun Chen Jiang Bian and Yonghui Wu. 2023. Model tuning or prompt tuning? A study of large language models for clinical concept and relation extraction. arXiv:2310.06239. Retrieved from https:\/\/arxiv.org\/abs\/2310.06239","DOI":"10.1016\/j.jbi.2024.104630"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocad107"},{"key":"e_1_3_2_98_2","doi-asserted-by":"crossref","unstructured":"Yifan Peng Qingyu Chen and Zhiyong Lu. 2020. An empirical study of multi-task learning on BERT for biomedical text mining. arXiv:2005.02799. Retrieved from https:\/\/arxiv.org\/abs\/2005.02799","DOI":"10.18653\/v1\/2020.bionlp-1.22"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.clinicalnlp-1.12"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-5006"},{"key":"e_1_3_2_101_2","first-page":"21","volume-title":"CEUR Workshop Proceedings","author":"Polignano Marco","year":"2021","unstructured":"Marco Polignano, Marco de Gemmis, and Giovanni Semeraro. 2021. Comparing transformer-based NER approaches for analysing textual medical diagnoses. In CEUR Workshop Proceedings. CEUR, 21\u201324."},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/S14-2007"},{"key":"e_1_3_2_103_2","unstructured":"Colin Raffel Noam Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li and Peter J. Liu. 2023. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv:1910.10683. Retrieved from https:\/\/arxiv.org\/abs\/1910.10683"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2023.104302"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.25747\/CD12-0636"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1093\/database\/baw100"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.435"},{"key":"e_1_3_2_108_2","first-page":"11","volume-title":"CEUR Workshop Proceedings","author":"Saiz Jos\u00e9 Javier","year":"2023","unstructured":"Jos\u00e9 Javier Saiz and Bego\u00f1a Altuna. 2023. End-to-end temporal relation extraction in the clinical domain [FULL]. In CEUR Workshop Proceedings. CEUR, 11 pages."},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.clinicalnlp-1.11"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-022-10338-7"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2023.104400"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz096"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-35176-1_7"},{"key":"e_1_3_2_114_2","first-page":"4","volume-title":"39th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Soldaini Luca","year":"2016","unstructured":"Luca Soldaini and Nazli Goharian. 2016. QuickUMLS: A fast, unsupervised approach for medical concept extraction. In 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, 4 pages."},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1152"},{"key":"e_1_3_2_116_2","doi-asserted-by":"publisher","DOI":"10.2196\/17984"},{"key":"e_1_3_2_117_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2019.103354"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICACRS58579.2023.10404906"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2013-001628"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40802-1_24"},{"key":"e_1_3_2_121_2","unstructured":"Madhumita Sushil Vanessa E. Kennedy Divneet Mandair Brenda Y. Miao Travis Zack and Atul J. Butte. 2023. Exploring zero-shot capability of large language models in inferences from medical oncology notes. arXiv:2308.03853. Retrieved from https:\/\/arxiv.org\/abs\/2308.03853"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.3233\/SHTI230281"},{"key":"e_1_3_2_123_2","unstructured":"Ruixiang Tang Xiaotian Han Xiaoqian Jiang and Xia Hu. 2023. Does synthetic data generation of LLMs help clinical text mining? arXiv:2303.04360. Retrieved from https:\/\/arxiv.org\/abs\/2303.04360"},{"key":"e_1_3_2_124_2","unstructured":"Yi Tay Mostafa Dehghani Vinh Q. Tran Xavier Garcia Jason Wei Xuezhi Wang Hyung Won Chung Siamak Shakeri Dara Bahri Tal Schuster et\u00a0al. 2023. UL2: Unifying language learning paradigms. arXiv:2205.05131. Retrieved from https:\/\/arxiv.org\/abs\/2205.05131"},{"key":"e_1_3_2_125_2","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re Naman Goyal Eric Hambro Faisal Azhar et\u00a0al. 2023. LLaMA: Open and efficient foundation language models. arXiv:2302.13971. Retrieved from https:\/\/arxiv.org\/abs\/2302.13971"},{"key":"e_1_3_2_126_2","unstructured":"Hangyu Tu Lifeng Han and Goran Nenadic. 2023. Extraction of medication and temporal relation from clinical text using neural language models. arXiv:2310.02229. Retrieved from https:\/\/arxiv.org\/abs\/2310.02229"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M2408"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2010.003947"},{"key":"e_1_3_2_129_2","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000203"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2012.04.004"},{"key":"e_1_3_2_131_2","first-page":"6000","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc., 6000\u20136010. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf"},{"key":"e_1_3_2_132_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2017.11.011"},{"key":"e_1_3_2_133_2","first-page":"1236","volume-title":"AMIA Annual Symposium Proceedings","author":"Wei Qiang","year":"2020","unstructured":"Qiang Wei. 2020. Relation extraction from clinical narratives using pre-trained language models. In AMIA Annual Symposium Proceedings. AMIA, 1236\u20131245."},{"key":"e_1_3_2_134_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2956831"},{"key":"e_1_3_2_135_2","doi-asserted-by":"publisher","DOI":"10.1109\/BIBM47256.2019.8983370"},{"key":"e_1_3_2_136_2","doi-asserted-by":"publisher","DOI":"10.3390\/app12136543"},{"key":"e_1_3_2_137_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocaa189"},{"key":"e_1_3_2_138_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-022-00742-2"},{"key":"e_1_3_2_139_2","doi-asserted-by":"publisher","DOI":"10.3390\/app12199691"},{"key":"e_1_3_2_140_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-023-02127-1"},{"key":"e_1_3_2_141_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCC-DSS-SmartCity-DependSys57074.2022.00308"},{"key":"e_1_3_2_142_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICEIEC49280.2020.9152257"},{"key":"e_1_3_2_143_2","unstructured":"Ningyu Zhang Mosha Chen Zhen Bi Xiaozhuan Liang Lei Li Xin Shang Kangping Yin Chuanqi Tan Jian Xu Fei Huang et\u00a0al. 2022. CBLUE: A Chinese biomedical language understanding evaluation benchmark. arXiv:2106.08087. Retrieved from https:\/\/arxiv.org\/abs\/2106.08087"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICICTA49267.2019.00043"},{"key":"e_1_3_2_145_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2019.103985"},{"key":"e_1_3_2_146_2","doi-asserted-by":"publisher","DOI":"10.3390\/ijerph192416590"},{"key":"e_1_3_2_147_2","doi-asserted-by":"publisher","unstructured":"Jinhong Zhong Zhanxiang Xuan Kang Wang and Zhou Cheng. 2023. A BERT-Span Model for Named Entity Recognition in Rehabilitation Medicine. Preprint (version 1) available at Research Square. DOI: 10.21203\/rs.3.rs-2588490\/v1","DOI":"10.21203\/rs.3.rs-2588490\/v1"},{"key":"e_1_3_2_148_2","doi-asserted-by":"publisher","unstructured":"Huixue Zhou Mingchen Li Yongkang Xiao Han Yang and Rui Zhang. 2023. LLM Instruction-Example Adaptive Prompting (LEAP) Framework for Clinical Relation Extraction. Preprint available at medRXiv. DOI: 10.1101\/2023.12.15.23300059","DOI":"10.1101\/2023.12.15.23300059"},{"key":"e_1_3_2_149_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocac040"},{"key":"e_1_3_2_150_2","doi-asserted-by":"crossref","unstructured":"Sicheng Zhou Nan Wang Liwei Wang Ju Sun Anne Blaes Hongfang Liu and Rui Zhang. 2023. A cross-institutional evaluation on breast cancer phenotyping NLP algorithms on electronic health records. arXiv:2303.08448. Retrieved from https:\/\/arxiv.org\/abs\/2303.08448","DOI":"10.1016\/j.csbj.2023.08.018"},{"key":"e_1_3_2_151_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITAIC58329.2023.10408910"}],"container-title":["ACM Transactions on Computing for Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3744660","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,2]],"date-time":"2026-03-02T15:50:33Z","timestamp":1772466633000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3744660"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,13]]},"references-count":150,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,10,31]]}},"alternative-id":["10.1145\/3744660"],"URL":"https:\/\/doi.org\/10.1145\/3744660","relation":{},"ISSN":["2691-1957","2637-8051"],"issn-type":[{"value":"2691-1957","type":"print"},{"value":"2637-8051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,13]]},"assertion":[{"value":"2024-06-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}