{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T22:48:44Z","timestamp":1773182924381,"version":"3.50.1"},"reference-count":108,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,6,12]],"date-time":"2025-06-12T00:00:00Z","timestamp":1749686400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Higher Education, Research and Innovation","award":["BFP\/RGP\/ICT\/22\/445"],"award-info":[{"award-number":["BFP\/RGP\/ICT\/22\/445"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>This systematic review evaluates recent literature from January 2021 to March 2024 on large language model (LLM) applications across diverse medical specialties. Searching PubMed, Web of Science, and Scopus, we included 84 studies. LLMs were applied to tasks such as clinical natural language processing, medical decision support, education, and aiding diagnostic processes. While studies reported benefits such as improved efficiency and, in some specific NLP tasks, high accuracy above 90%, significant challenges persist concerning reliability, ethical implications, and performance consistency, with accuracy in broader diagnostic support applications showing substantial variability, with some as low as 3%. The overall risk of bias in the reviewed literature was considerably low in 72 studies. Key findings highlight a substantial heterogeneity in LLM performance across different medical tasks and contexts, preventing meta-analysis due to a lack of standardized methodologies. Future efforts should prioritize developing domain-specific LLMs using robust medical data and establishing rigorous validation standards to ensure their safe and effective clinical integration. Trial registration: PROSPERO (CRD42024561381).<\/jats:p>","DOI":"10.3390\/info16060489","type":"journal-article","created":{"date-parts":[[2025,6,12]],"date-time":"2025-06-12T11:47:07Z","timestamp":1749728827000},"page":"489","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Systematic Review of Large Language Models in Medical Specialties: Applications, Challenges and Future Directions"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5507-4873","authenticated-orcid":false,"given":"Asma Musabah","family":"Alkalbani","sequence":"first","affiliation":[{"name":"Department of Information Technology, College of Computing and Information Sciences, University of Technology and Applied Sciences, Ibri 511, Oman"},{"name":"School of Computing, Macquarie University, Sydney, NSW 2109, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1990-5101","authenticated-orcid":false,"given":"Ahmed Salim","family":"Alrawahi","sequence":"additional","affiliation":[{"name":"Department of Information Technology, College of Computing and Information Sciences, University of Technology and Applied Sciences, Ibri 511, Oman"},{"name":"AI Applications Research Chair, University of Nizwa, Nizwa 616, Oman"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3433-7640","authenticated-orcid":false,"given":"Ahmad","family":"Salah","sequence":"additional","affiliation":[{"name":"Department of Information Technology, College of Computing and Information Sciences, University of Technology and Applied Sciences, Ibri 511, Oman"},{"name":"Department of Computer Science, Faculty of Computers and Informatics, Zagazig University, Zagazig 44519, Egypt"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5036-8984","authenticated-orcid":false,"given":"Venus","family":"Haghighi","sequence":"additional","affiliation":[{"name":"School of Computing, Macquarie University, Sydney, NSW 2109, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6821-2710","authenticated-orcid":false,"given":"Yang","family":"Zhang","sequence":"additional","affiliation":[{"name":"The Anuradha and Vikas Sinha Department of Data Science, University of North Texas, Denton, TX 76203, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6863-5748","authenticated-orcid":false,"given":"Salam","family":"Alkindi","sequence":"additional","affiliation":[{"name":"Department of Hematology, College of Medicine & Health Science, Sultan Qaboos University, Muscat 123, Oman"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3326-4147","authenticated-orcid":false,"given":"Quan Z.","family":"Sheng","sequence":"additional","affiliation":[{"name":"School of Computing, Macquarie University, Sydney, NSW 2109, Australia"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,12]]},"reference":[{"key":"ref_1","unstructured":"Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2018). Improving language understanding by generative pre-training. arXiv."},{"key":"ref_2","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2050","DOI":"10.1038\/s41467-024-46411-8","article-title":"Systematic analysis of ChatGPT, Google search and Llama 2 for clinical decision support tasks","volume":"15","author":"Sandmann","year":"2024","journal-title":"Nat. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Vrdoljak, J., Boban, Z., Vilovi\u0107, M., Kumri\u0107, M., and Bo\u017ei\u0107, J. (2025). A review of large language models in medical education, clinical decision support, and healthcare administration. Healthcare, 13.","DOI":"10.3390\/healthcare13060603"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Mishra, T., Sutanto, E., Rossanti, R., Pant, N., Ashraf, A., Raut, A., Uwabareze, G., Oluwatomiwa, A., and Zeeshan, B. (2024). Use of large language models as artificial intelligence tools in academic research and publishing among global clinical researchers. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-81370-6"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1007\/s10916-023-01961-0","article-title":"ChatGPT Performs on the Chinese National Medical Licensing Examination","volume":"47","author":"Wang","year":"2023","journal-title":"J. Med. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"123046","DOI":"10.1016\/j.eswa.2023.123046","article-title":"SOAP classifier for free-text clinical notes with domain-specific pre-trained language models","volume":"245","author":"Antunes","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"104956","DOI":"10.1016\/j.ijmedinf.2022.104956","article-title":"Predicting medical specialty from text based on a domain-specific pre-trained BERT","volume":"170","author":"Kim","year":"2023","journal-title":"Int. J. Med. Inform."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e47479","DOI":"10.2196\/47479","article-title":"Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument","volume":"25","author":"Walker","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e49324","DOI":"10.2196\/49324","article-title":"Large Language Models for Therapy Recommendations Across 3 Clinical Specialties: Comparative Study","volume":"25","author":"Wilhelm","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1080\/0142159X.2023.2249588","article-title":"ChatGPT-4: An assessment of an upgraded artificial intelligence chatbot in the United States Medical Licensing Examination","volume":"46","author":"Mihalache","year":"2023","journal-title":"Med. Teach."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Alkamli, S., Al-Yahya, M., and Alyahya, K. (2024, January 26\u201329). Ethical and Legal Considerations of Large Language Models: A Systematic Review of the Literature. Proceedings of the 2024 2nd International Conference on Foundation and Large Language Models (FLLM), Dubai, United Arab Emirates.","DOI":"10.1109\/FLLM63129.2024.10852451"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"110834","DOI":"10.1016\/j.knosys.2023.110834","article-title":"Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speech","volume":"277","author":"Ilias","year":"2023","journal-title":"Knowl. Based Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Pashangpour, S., and Nejat, G. (2024). The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare. Robotics, 13.","DOI":"10.3390\/robotics13080112"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Artsi, Y., Sorin, V., Konen, E., Glicksberg, B.S., Nadkarni, G., and Klang, E. (2024). Large language models for generating medical examinations: Systematic review. BMC Med. Educ., 24.","DOI":"10.1186\/s12909-024-05239-y"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Pressman, S.M., Borna, S., Gomez-Cabello, C.A., Haider, S.A., Haider, C.R., and Forte, A.J. (2024). Clinical and Surgical Applications of Large Language Models: A Systematic Review. J. Clin. Med., 13.","DOI":"10.3390\/jcm13113041"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e59607","DOI":"10.2196\/59607","article-title":"Examining the Role of Large Language Models in Orthopedics: Systematic Review","volume":"26","author":"Zhang","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1007\/s10462-024-10849-5","article-title":"Large language models in healthcare: From a systematic review on medical examinations to a comparative analysis on fundamentals of robotic surgery online test","volume":"57","author":"Moglia","year":"2024","journal-title":"Artif. Intell. Rev."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"e60807","DOI":"10.2196\/60807","article-title":"Performance of ChatGPT across different versions in medical licensing examinations worldwide: Systematic review and meta-analysis","volume":"26","author":"Liu","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e22769","DOI":"10.2196\/22769","article-title":"Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review","volume":"26","author":"Wang","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"n71","DOI":"10.1136\/bmj.n71","article-title":"The PRISMA 2020 statement: An updated guideline for reporting systematic reviews","volume":"372","author":"Page","year":"2021","journal-title":"BMJ"},{"key":"ref_22","unstructured":"(2024, April 01). Specialty Profiles. Available online: https:\/\/careersinmedicine.aamc.org\/."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1663","DOI":"10.1038\/s41591-021-01517-0","article-title":"A quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies: QUADAS-AI","volume":"27","author":"Sounderajah","year":"2021","journal-title":"Nat. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"116882","DOI":"10.1016\/j.eswa.2022.116882","article-title":"An intelligent early warning system of analyzing Twitter data using machine learning on COVID-19 surveillance in the US","volume":"198","author":"Zhang","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e27210","DOI":"10.2196\/27210","article-title":"A Question-and-Answer System to Extract Data From Free-Text Oncological Pathology Reports (CancerBERT Network): Development Study","volume":"24","author":"Mitchell","year":"2022","journal-title":"J. Med. Internet Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1111\/pcn.13656","article-title":"Comparing the performance of ChatGPT GPT-4, Bard, and Llama-2 in the Taiwan Psychiatric Licensing Examination and in differential diagnosis with multi-center psychiatrists","volume":"78","author":"Li","year":"2024","journal-title":"Psychiatry Clin. Neurosci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1484","DOI":"10.1002\/ohn.465","article-title":"Head-to-head comparison of ChatGPT versus Google search for medical knowledge acquisition","volume":"170","author":"Ayoub","year":"2024","journal-title":"Otolaryngol. Neck Surg."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"105173","DOI":"10.1016\/j.ijmedinf.2023.105173","article-title":"Performance and exploration of ChatGPT in medical examination, records and education in Chinese: Pave the way for medical AI","volume":"177","author":"Wang","year":"2023","journal-title":"Int. J. Med. Inform."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"e51501","DOI":"10.2196\/51501","article-title":"Text Dialogue Analysis for Primary Screening of Mild Cognitive Impairment: Development and Validation Study","volume":"25","author":"Wang","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1111\/his.15100","article-title":"Bridging bytes and biopsies: A comparative analysis of ChatGPT and histopathologists in pathology diagnosis and collaborative potential","volume":"84","author":"Oon","year":"2023","journal-title":"Histopathology"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"105219","DOI":"10.1016\/j.ijmedinf.2023.105219","article-title":"A comprehensive evaluation of ChatGPT consultation quality for augmentation mammoplasty: A comparative analysis between plastic surgeons and laypersons","volume":"179","author":"Yun","year":"2023","journal-title":"Int. J. Med. Inform."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"110077","DOI":"10.1016\/j.resuscitation.2023.110077","article-title":"Testing ChatGPT ability to answer laypeople questions about cardiac arrest and cardiopulmonary resuscitation","volume":"194","author":"Scquizzato","year":"2024","journal-title":"Resuscitation"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"825","DOI":"10.1093\/cid\/ciad632","article-title":"Can Chatbot Artificial Intelligence Replace Infectious Diseases Physicians in the Management of Bloodstream Infections? A Prospective Cohort Study","volume":"78","author":"Maillard","year":"2023","journal-title":"Clin. Infect. Dis."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/s10916-023-02019-x","article-title":"\u201cChatGPT, can you help me save my child\u2019s life?\u201d-Diagnostic Accuracy and Supportive Capabilities to lay rescuers by ChatGPT in prehospital Basic Life Support and Paediatric Advanced Life Support cases\u2013an in-silico analysis","volume":"47","author":"Bushuven","year":"2023","journal-title":"J. Med. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2108","DOI":"10.1093\/jamia\/ocab126","article-title":"MT-clinical BERT: Scaling clinical information extraction with multitask learning","volume":"28","author":"Mulyar","year":"2021","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1109\/TMM.2021.3132724","article-title":"Chestxraybert: A pretrained language model for chest radiology report summarization","volume":"25","author":"Cai","year":"2021","journal-title":"IEEE Trans. Multimed."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"104864","DOI":"10.1016\/j.ijmedinf.2022.104864","article-title":"Leveraging weak supervision to perform named entity recognition in electronic health records progress notes to identify the ophthalmology exam","volume":"167","author":"Wang","year":"2022","journal-title":"Int. J. Med. Inform."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"e48966","DOI":"10.2196\/48966","article-title":"ChatGPT vs Google for Queries Related to Dementia and Other Cognitive Decline: Comparison of Results","volume":"25","author":"Hristidis","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2887","DOI":"10.1007\/s00464-024-10739-5","article-title":"Dr. Google to Dr. ChatGPT: Assessing the content and quality of artificial intelligence-generated medical information on appendicitis","volume":"38","author":"Ghanem","year":"2024","journal-title":"Surg. Endosc."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1111\/bdi.13379","article-title":"A chat about bipolar disorder","volume":"26","author":"Parker","year":"2023","journal-title":"Bipolar Disord."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1366","DOI":"10.1227\/neu.0000000000002598","article-title":"Exploring the Intersection of Artificial Intelligence and Neurosurgery: Let us be Cautious With ChatGPT","volume":"93","author":"Mishra","year":"2023","journal-title":"Neurosurgery"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Shang, J., Tang, X., and Sun, Y. (2023). PhaTYP: Predicting the lifestyle for bacteriophages using BERT. Briefings Bioinform., 24.","DOI":"10.1093\/bib\/bbac487"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Zhu, G., Li, K., Li, F., Huang, L., Duan, M., and Zhou, F. (2022). HLAB: Learning the BiLSTM features from the ProtBert-encoded proteins for the class I HLA-peptide binding prediction. Briefings Bioinform., 23.","DOI":"10.1093\/bib\/bbac173"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"e25314","DOI":"10.2196\/25314","article-title":"Toward using Twitter for tracking COVID-19: A natural language processing pipeline and exploratory data set","volume":"23","author":"Klein","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1093\/jamia\/ocab243","article-title":"Natural language inference for curation of structured clinical registries from unstructured text","volume":"29","author":"Percha","year":"2021","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1362","DOI":"10.1136\/bjo-2023-324734","article-title":"Review of emerging trends and projection of future developments in large language models research in ophthalmology","volume":"108","author":"Wong","year":"2023","journal-title":"Br. J. Ophthalmol."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"17562848231218618","DOI":"10.1177\/17562848231218618","article-title":"Evaluating the role of ChatGPT in gastroenterology: A comprehensive systematic review of applications, benefits, and limitations","volume":"16","author":"Klang","year":"2023","journal-title":"Ther. Adv. Gastroenterol."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Younis, H.A., Eisa, T.A.E., Nasser, M., Sahib, T.M., Noor, A.A., Alyasiri, O.M., Salisu, S., Hayder, I.M., and Younis, H.A. (2024). A Systematic Review and Meta-Analysis of Artificial Intelligence Tools in Medicine and Healthcare: Applications, Considerations, Limitations, Motivation and Challenges. Diagnostics, 14.","DOI":"10.3390\/diagnostics14010109"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1111\/1471-0528.17641","article-title":"Performance of ChatGPT in medical examinations: A systematic review and a meta-analysis","volume":"131","author":"Levin","year":"2024","journal-title":"BJOG Int. J. Obstet. Gynaecol."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"e48933","DOI":"10.2196\/48933","article-title":"Applications of the natural language processing tool ChatGPT in clinical practice: Comparative study and augmented systematic review","volume":"11","author":"Schopow","year":"2023","journal-title":"JMIR Med. Inform."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"e34068","DOI":"10.1097\/MD.0000000000034068","article-title":"Assessing ChatGPT\u2019s capacity for clinical decision support in pediatrics: A comparative study with pediatricians using KIDMAP of Rasch analysis","volume":"102","author":"Kao","year":"2023","journal-title":"Medicine"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1208","DOI":"10.1093\/jamia\/ocac040","article-title":"CancerBERT: A cancer domain-specific language model for extracting breast cancer phenotypes from electronic health records","volume":"29","author":"Zhou","year":"2022","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Shool, S., Adimi, S., Saboori Amleshi, R., Bitaraf, E., Golpira, R., and Tara, M. (2025). A systematic review of large language model (LLM) evaluations in clinical medicine. BMC Med. Inform. Decis. Mak., 25.","DOI":"10.1186\/s12911-025-02954-4"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"1353","DOI":"10.1227\/neu.0000000000002551","article-title":"Performance of ChatGPT and GPT-4 on Neurosurgery Written Board Examinations","volume":"93","author":"Ali","year":"2023","journal-title":"Neurosurgery"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1173","DOI":"10.5435\/JAAOS-D-23-00396","article-title":"Comparison of ChatGPT\u20133.5, ChatGPT-4, and orthopaedic resident performance on orthopaedic assessment examinations","volume":"31","author":"Massey","year":"2023","journal-title":"JAAOS-J. Am. Acad. Orthop. Surg."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1077","DOI":"10.1002\/ksa.12133","article-title":"From technical to understandable: Artificial Intelligence Large Language Models improve the readability of knee radiology reports","volume":"32","author":"Butler","year":"2024","journal-title":"Knee Surgery Sport. Traumatol. Arthrosc."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"1519","DOI":"10.2106\/JBJS.23.00209","article-title":"Assessing ChatGPT Responses to Common Patient Questions Regarding Total Hip Arthroplasty","volume":"105","author":"Mika","year":"2023","journal-title":"J. Bone Jt. Surg."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"e19689","DOI":"10.2196\/19689","article-title":"Use of BERT (Bidirectional Encoder Representations from Transformers)-Based Deep Learning Method for Extracting Evidences in Chinese Radiology Reports: Development of a Computer-Aided Liver Cancer Diagnosis Framework","volume":"23","author":"Liu","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"104628","DOI":"10.1016\/j.ijmedinf.2021.104628","article-title":"Fine-grained spatial information extraction in radiology as two-turn question answering","volume":"158","author":"Datta","year":"2022","journal-title":"Int. J. Med. Inform."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"1657","DOI":"10.1093\/jamia\/ocad133","article-title":"Inferring cancer disease response from radiology reports using large language models with data augmentation and prompting","volume":"30","author":"Tan","year":"2023","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"102117","DOI":"10.1016\/j.arr.2023.102117","article-title":"ChatGPT\u2019s Dance with Neuropsychological Data: A case study in Alzheimer\u2019s Disease","volume":"92","author":"Mohamad","year":"2023","journal-title":"Ageing Res. Rev."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1111\/pcn.13580","article-title":"Diagnosing psychiatric disorders from history of present illness using a large-scale linguistic model","volume":"77","author":"Otsuka","year":"2023","journal-title":"Psychiatry Clin. Neurosci."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1016\/j.ajem.2024.02.008","article-title":"Human intelligence versus Chat-GPT: Who performs better in correctly classifying patients in triage?","volume":"79","author":"Zaboli","year":"2024","journal-title":"Am. J. Emerg. Med."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.ajem.2023.11.063","article-title":"Deep learning-based natural language processing for detecting medical symptoms and histories in emergency patient triage","volume":"77","author":"Lee","year":"2024","journal-title":"Am. J. Emerg. Med."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"CB, A., Mahesh, K., and Sanda, N. (2023). Ontology-based semantic data interestingness using BERT models. Connect. Sci., 35.","DOI":"10.1080\/09540091.2023.2190499"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"648","DOI":"10.1093\/bioinformatics\/btab712","article-title":"BERT-Kcr: Prediction of lysine crotonylation sites by a transfer learning method with pre-trained BERT models","volume":"38","author":"Qiao","year":"2021","journal-title":"Bioinformatics"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Vithanage, D., Zhu, Y., Zhang, Z., Deng, C., Yin, M., and Yu, P. (2024). Extracting Symptoms of Agitation in Dementia from Free-Text Nursing Notes Using Advanced Natural Language Processing. MEDINFO 2023\u2014The Future Is Accessible, IOS Press.","DOI":"10.3233\/SHTI231055"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"8548","DOI":"10.1002\/int.22955","article-title":"A medical question answering system using large language models and knowledge graphs","volume":"37","author":"Guo","year":"2022","journal-title":"Int. J. Intell. Syst."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"1856","DOI":"10.1093\/jamia\/ocae030","article-title":"Clinical risk prediction using language models: Benefits and considerations","volume":"31","author":"Acharya","year":"2024","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"e25113","DOI":"10.2196\/25113","article-title":"Disease Concept-Embedding Based on the Self-Supervised Method for Medical Information Extraction from Electronic Health Records and Disease Retrieval: Algorithm Development and Validation Study","volume":"23","author":"Chen","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"105283","DOI":"10.1016\/j.ijmedinf.2023.105283","article-title":"Influence on the accuracy in ChatGPT: Differences in the amount of information per medical field","volume":"180","author":"Haze","year":"2023","journal-title":"Int. J. Med. Inform."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1093\/jamia\/ocac018","article-title":"Extracting seizure frequency from epilepsy clinic notes: A machine reading approach to natural language processing","volume":"29","author":"Xie","year":"2022","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"116845","DOI":"10.1016\/j.eswa.2022.116845","article-title":"Deepluenza: Deep learning for influenza detection from Twitter","volume":"198","author":"Alkouz","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"e22860","DOI":"10.2196\/22860","article-title":"Revealing opinions for COVID-19 questions using a context retriever, opinion aggregator, and question-answering model: Model development study","volume":"23","author":"Lu","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1002\/path.6232","article-title":"Extracting structured information from unstructured histopathology reports using generative pre-trained transformer 4 (GPT-4)","volume":"262","author":"Truhn","year":"2023","journal-title":"J. Pathol."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.pediatrneurol.2023.08.035","article-title":"Leveraging ChatGPT in the Pediatric Neurology Clinic: Practical Considerations for Use to Improve Efficiency and Outcomes","volume":"148","author":"Karakas","year":"2023","journal-title":"Pediatr. Neurol."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"1237","DOI":"10.1093\/jamia\/ocad072","article-title":"Using AI-generated suggestions from ChatGPT to optimize clinical decision support","volume":"30","author":"Liu","year":"2023","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"e50882","DOI":"10.2196\/50882","article-title":"Quality and Dependability of ChatGPT and DingXiangYuan Forums for Remote Orthopedic Consultations: Comparative Analysis","volume":"26","author":"Xue","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"e49771","DOI":"10.2196\/49771","article-title":"Potential and Limitations of ChatGPT 3.5 and 4.0 as a Source of COVID-19 Information: Comprehensive Comparative Analysis of Generative and Authoritative Information","volume":"25","author":"Wang","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Ji, S., H\u00f6ltt\u00e4, M., and Marttinen, P. (2021). Does the magic of BERT apply to medical code assignment? A quantitative study. Comput. Biol. Med., 139.","DOI":"10.1016\/j.compbiomed.2021.104998"},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"108842","DOI":"10.1016\/j.asoc.2022.108842","article-title":"Pre-trained ensemble model for identification of emotion during COVID-19 based on emergency response support system dataset","volume":"122","author":"Nimmi","year":"2022","journal-title":"Appl. Soft Comput."},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"1995","DOI":"10.1093\/jamia\/ocad177","article-title":"A method to automate the discharge summary hospital course for neurology patients","volume":"30","author":"Hartman","year":"2023","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"e49074","DOI":"10.2196\/49074","article-title":"Understanding Mental Health Issues in Different Subdomains of Social Networking Services: Computational Analysis of Text-Based Reddit Posts","volume":"25","author":"Kim","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"105321","DOI":"10.1016\/j.ijmedinf.2023.105321","article-title":"Zero-shot information extraction from radiological reports using ChatGPT","volume":"183","author":"Hu","year":"2024","journal-title":"Int. J. Med. Inform."},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1093\/jamia\/ocad218","article-title":"AutoCriteria: A generalizable clinical trial eligibility criteria extraction system powered by large language models","volume":"31","author":"Datta","year":"2023","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_86","doi-asserted-by":"crossref","unstructured":"Bombieri, M., Rospocher, M., Ponzetto, S.P., and Fiorini, P. (2023). Machine understanding surgical actions from intervention procedure textbooks. Comput. Biol. Med., 152.","DOI":"10.1016\/j.compbiomed.2022.106415"},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"e47014","DOI":"10.2196\/47014","article-title":"Identifying Potential Lyme Disease Cases Using Self-Reported Worldwide Tweets: Deep Learning Modeling Approach Enhanced With Sentimental Words Through Emojis","volume":"25","author":"Laison","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Wang, X.F., Yu, C.Q., You, Z.H., Qiao, Y., Li, Z.W., and Huang, W.Z. (2023). An efficient circRNA-miRNA interaction prediction model by combining biological text mining and wavelet diffusion-based sparse network structure embedding. Comput. Biol. Med., 165.","DOI":"10.1016\/j.compbiomed.2023.107421"},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"e26777","DOI":"10.2196\/26777","article-title":"Natural language processing and machine learning methods to characterize unstructured patient-reported outcomes: Validation study","volume":"23","author":"Lu","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"121598","DOI":"10.1016\/j.eswa.2023.121598","article-title":"medBERT.de: A comprehensive German BERT model for the medical domain","volume":"237","author":"Bressem","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_91","doi-asserted-by":"crossref","unstructured":"Chizhikova, M., L\u00f3pez-\u00dabeda, P., Collado-Monta\u00f1ez, J., Mart\u00edn-Noguerol, T., D\u00edaz-Galiano, M.C., Luna, A., Ure\u00f1a-L\u00f3pez, L.A., and Mart\u00edn-Valdivia, M.T. (2023). CARES: A Corpus for classification of Spanish Radiological reports. Comput. Biol. Med., 154.","DOI":"10.1016\/j.compbiomed.2023.106581"},{"key":"ref_92","doi-asserted-by":"crossref","first-page":"104805","DOI":"10.1016\/j.ijmedinf.2022.104805","article-title":"Evaluation of clinical named entity recognition methods for Serbian electronic health records","volume":"164","author":"Kaplar","year":"2022","journal-title":"Int. J. Med. Inform."},{"key":"ref_93","doi-asserted-by":"crossref","first-page":"2574","DOI":"10.1109\/TCSVT.2021.3087641","article-title":"An OCR Post-Correction Approach Using Deep Learning for Processing Medical Reports","volume":"32","author":"Karthikeyan","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_94","doi-asserted-by":"crossref","first-page":"e49944","DOI":"10.2196\/49944","article-title":"A Natural Language Processing Model for COVID-19 Detection Based on Dutch General Practice Electronic Health Records by Using Bidirectional Encoder Representations From Transformers: Development and Validation Study","volume":"25","author":"Homburg","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Fonseca, \u0102., Ferreira, A., Ribeiro, L., Moreira, S., and Duque, C. (2024). Embracing the future\u2014Is artificial intelligence already better? A comparative study of artificial intelligence performance in diagnostic accuracy and decision-making. Eur. J. Neurol., 31.","DOI":"10.1111\/ene.16195"},{"key":"ref_96","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.ajem.2023.10.034","article-title":"Performance of Google bard and ChatGPT in mass casualty incidents triage","volume":"75","author":"Gan","year":"2024","journal-title":"Am. J. Emerg. Med."},{"key":"ref_97","doi-asserted-by":"crossref","first-page":"121186","DOI":"10.1016\/j.eswa.2023.121186","article-title":"Can ChatGPT provide intelligent diagnoses? A comparative study between predictive models and ChatGPT to define a new medical diagnostic bot","volume":"235","author":"Caruccio","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"1110","DOI":"10.1093\/postmj\/qgad053","article-title":"Artificial intelligence in orthopaedics: Can chat generative pre-trained transformer (ChatGPT) pass Section 1 of the fellowship of the royal college of surgeons (trauma & orthopaedics) examination?","volume":"99","author":"Cuthbert","year":"2023","journal-title":"Postgrad. Med J."},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"104736","DOI":"10.1016\/j.ijmedinf.2022.104736","article-title":"A hybrid model to identify fall occurrence from electronic health records","volume":"162","author":"Fu","year":"2022","journal-title":"Int. J. Med. Inform."},{"key":"ref_100","doi-asserted-by":"crossref","first-page":"5190","DOI":"10.1007\/s00167-023-07529-2","article-title":"Exploring the potential of ChatGPT as a supplementary tool for providing orthopaedic information","volume":"31","author":"Kaarre","year":"2023","journal-title":"Knee Surg. Sport. Traumatol. Arthrosc."},{"key":"ref_101","doi-asserted-by":"crossref","first-page":"e48145","DOI":"10.2196\/48145","article-title":"OpenDeID Pipeline for Unstructured Electronic Health Record Text Notes Based on Rules and Transformers: Deidentification Algorithm Development and Validation Study","volume":"25","author":"Liu","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"1388","DOI":"10.1093\/jamia\/ocae041","article-title":"Why do users override alerts? Utilizing large language model to summarize comments and optimize clinical decision support","volume":"31","author":"Liu","year":"2024","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_103","doi-asserted-by":"crossref","unstructured":"Song, H., Xia, Y., Luo, Z., Liu, H., Song, Y., Zeng, X., Li, T., Zhong, G., Li, J., and Chen, M. (2023). Evaluating the Performance of Different Large Language Models on Health Consultation and Patient Education in Urolithiasis. J. Med. Syst., 47.","DOI":"10.1007\/s10916-023-02021-3"},{"key":"ref_104","doi-asserted-by":"crossref","first-page":"1504","DOI":"10.1002\/ohn.506","article-title":"BPPV Information on Google VersusAI (ChatGPT)","volume":"170","author":"Bellinger","year":"2023","journal-title":"Otolaryngol. Neck Surg."},{"key":"ref_105","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1007\/s11192-017-2283-4","article-title":"How to reduce the number of rating scale items without predictability loss?","volume":"111","author":"Koczkodaj","year":"2017","journal-title":"Scientometrics"},{"key":"ref_106","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1089\/thy.2023.0491","article-title":"Evaluating ChatGPT responses on thyroid nodules for patient education","volume":"34","author":"Campbell","year":"2024","journal-title":"Thyroid"},{"key":"ref_107","doi-asserted-by":"crossref","first-page":"2193","DOI":"10.1093\/jamia\/ocab112","article-title":"Are synthetic clinical notes useful for real natural language processing tasks: A case study on clinical entity recognition","volume":"28","author":"Li","year":"2021","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_108","doi-asserted-by":"crossref","unstructured":"Tsoutsanis, P., and Tsoutsanis, A. (2024). Evaluation of Large language model performance on the Multi-Specialty Recruitment Assessment (MSRA) exam. Comput. Biol. Med., 168.","DOI":"10.1016\/j.compbiomed.2023.107794"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/489\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:51:05Z","timestamp":1760032265000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/489"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,12]]},"references-count":108,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["info16060489"],"URL":"https:\/\/doi.org\/10.3390\/info16060489","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,12]]}}}