{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T13:01:46Z","timestamp":1782738106648,"version":"3.54.5"},"reference-count":67,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,3,31]],"date-time":"2025-03-31T00:00:00Z","timestamp":1743379200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"},{"start":{"date-parts":[[2025,3,31]],"date-time":"2025-03-31T00:00:00Z","timestamp":1743379200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"DOI":"10.1186\/s12911-025-02871-6","type":"journal-article","created":{"date-parts":[[2025,3,31]],"date-time":"2025-03-31T11:17:09Z","timestamp":1743419829000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Leveraging large language models to mimic domain expert labeling in unstructured text-based electronic healthcare records in non-english languages"],"prefix":"10.1186","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3055-7355","authenticated-orcid":false,"given":"Izzet Turkalp","family":"Akbasli","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7131-3993","authenticated-orcid":false,"given":"Ahmet Ziya","family":"Birbilen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1856-0500","authenticated-orcid":false,"given":"Ozlem","family":"Teksam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,3,31]]},"reference":[{"issue":"5","key":"2871_CR1","doi-asserted-by":"crossref","first-page":"758","DOI":"10.1016\/j.ipm.2018.01.010","volume":"54","author":"MK Saggi","year":"2018","unstructured":"Saggi MK, Jain S. A survey towards an integration of big data analytics to big insights for value-creation. Inf Process Manag. 2018;54(5):758\u201390.","journal-title":"Inf Process Manag"},{"issue":"Suppl 3","key":"2871_CR2","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1093\/eurpub\/ckz168","volume":"29","author":"R Pastorino","year":"2019","unstructured":"Pastorino R, De Vito C, Migliara G, Glocker K, Binenbaum I, Ricciardi W, et al. Benefits and challenges of Big Data in healthcare: an overview of the European initiatives. Eur J Public Health. 2019;29(Suppl 3):23\u20137.","journal-title":"Eur J Public Health"},{"key":"2871_CR3","doi-asserted-by":"crossref","unstructured":"Mishra S, Tripathy HK, Mishra BK, Sahoo S. Usage and Analysis of Big Data in E-Health Domain. In: Research Anthology on Big Data Analytics, Architectures, and Applications. IGI Global; 2022 [cited 2024 Feb 8]. pp. 417\u201330. Available from: https:\/\/www.igi-global.com\/chapter\/usage-and-analysis-of-big-data-in-e-health-domain\/www.igi-global.com\/chapter\/usage-and-analysis-of-big-data-in-e-health-domain\/290994","DOI":"10.4018\/978-1-6684-3662-2.ch020"},{"issue":"4","key":"2871_CR4","doi-asserted-by":"crossref","first-page":"e25759","DOI":"10.2196\/25759","volume":"23","author":"J Yin","year":"2021","unstructured":"Yin J, Ngiam KY, Teo HH. Role of artificial intelligence applications in real-life clinical practice: systematic review. J Med Internet Res. 2021;23(4):e25759.","journal-title":"J Med Internet Res"},{"issue":"1","key":"2871_CR5","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1038\/s41746-021-00423-6","volume":"4","author":"DW Bates","year":"2021","unstructured":"Bates DW, Levine D, Syrowatka A, Kuznetsova M, Craig KJT, Rui A, et al. The potential of artificial intelligence to improve patient safety: a scoping review. NPJ Digit Med. 2021;4(1):54.","journal-title":"NPJ Digit Med"},{"key":"2871_CR6","doi-asserted-by":"crossref","unstructured":"Levine DM, Tuwani R, Kompa B, Varma A, Finlayson SG, Mehrotra A et al. The Diagnostic and Triage Accuracy of the GPT\u20133 Artificial Intelligence Model. medRxiv. 2023;2023.01.30.23285067.","DOI":"10.1101\/2023.01.30.23285067"},{"issue":"1","key":"2871_CR7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-020-00333-z","volume":"3","author":"B Mesk\u00f3","year":"2020","unstructured":"Mesk\u00f3 B, G\u00f6r\u00f6g M. A short guide for medical professionals in the era of artificial intelligence. Npj Digit Med. 2020;3(1):1\u20138.","journal-title":"Npj Digit Med"},{"issue":"4","key":"2871_CR8","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1038\/s41437-020-0303-2","volume":"124","author":"R Agrawal","year":"2020","unstructured":"Agrawal R, Prabakaran S. Big data in digital healthcare: lessons learnt and recommendations for general practice. Heredity. 2020;124(4):525\u201334.","journal-title":"Heredity"},{"issue":"6","key":"2871_CR9","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1001\/jama.2019.21579","volume":"323","author":"ME Matheny","year":"2020","unstructured":"Matheny ME, Whicher D, Israni ST. Artificial intelligence in health care: a report from the National Academy of Medicine. JAMA. 2020;323(6):509\u201310.","journal-title":"JAMA"},{"issue":"13","key":"2871_CR10","doi-asserted-by":"crossref","first-page":"1317","DOI":"10.1001\/jama.2017.18391","volume":"319","author":"AL Beam","year":"2018","unstructured":"Beam AL, Kohane IS. Big Data and Machine Learning in Health Care. JAMA. 2018;319(13):1317\u20138.","journal-title":"JAMA"},{"key":"2871_CR11","doi-asserted-by":"crossref","first-page":"baaa010","DOI":"10.1093\/database\/baaa010","volume":"2020","author":"Z Ahmed","year":"2020","unstructured":"Ahmed Z, Mohamed K, Zeeshan S, Dong X. Artificial intelligence with multi-functional machine learning platform development for better healthcare and precision medicine. Database. 2020;2020:baaa010.","journal-title":"Database"},{"issue":"3","key":"2871_CR12","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1071\/AH20062","volume":"45","author":"H Zhou","year":"2021","unstructured":"Zhou H, Albrecht MA, Roberts PA, Porter P, Della PR. Using machine learning to predict paediatric 30-day unplanned hospital readmissions: a case-control retrospective analysis of medical records, including written discharge documentation. Aust Health Rev Publ Aust Hosp Assoc. 2021;45(3):328\u201337.","journal-title":"Aust Health Rev Publ Aust Hosp Assoc"},{"issue":"1","key":"2871_CR13","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1055\/s-0039-1677908","volume":"28","author":"F Wang","year":"2019","unstructured":"Wang F, Preininger A. AI in Health: state of the art, challenges, and future directions. Yearb Med Inf. 2019;28(1):16\u201326.","journal-title":"Yearb Med Inf"},{"issue":"4","key":"2871_CR14","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1001\/jama.2019.20866","volume":"323","author":"AL Beam","year":"2020","unstructured":"Beam AL, Manrai AK, Ghassemi M. Challenges to the Reproducibility of Machine Learning Models in Health Care. JAMA. 2020;323(4):305\u20136.","journal-title":"JAMA"},{"key":"2871_CR15","doi-asserted-by":"crossref","first-page":"12339","DOI":"10.1038\/srep12339","volume":"5","author":"P Zhang","year":"2015","unstructured":"Zhang P, Wang F, Hu J, Sorrentino R. Label propagation prediction of drug-drug interactions based on Clinical Side effects. Sci Rep. 2015;5:12339.","journal-title":"Sci Rep"},{"issue":"5","key":"2871_CR16","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1016\/j.fertnstert.2020.09.159","volume":"114","author":"CL Curchoe","year":"2020","unstructured":"Curchoe CL, Flores-Saiffe Farias A, Mendizabal-Ruiz G, Chavez-Badiola A. Evaluating predictive models in reproductive medicine. Fertil Steril. 2020;114(5):921\u20136.","journal-title":"Fertil Steril"},{"key":"2871_CR17","doi-asserted-by":"crossref","unstructured":"Agrawal M, Hegselmann S, Lang H, Kim Y, Sontag D. Large language models are few-shot clinical information extractors. In: Goldberg Y, Kozareva Z, Zhang Y, editors. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics; 2022 [cited 2024 Feb 8]. pp. 1998\u20132022. Available from: https:\/\/aclanthology.org\/2022.emnlp-main.130","DOI":"10.18653\/v1\/2022.emnlp-main.130"},{"key":"2871_CR18","unstructured":"Goel A, Gueta A, Gilon O, Liu C, Erell S, Nguyen LH et al. LLMs Accelerate Annotation for Medical Information Extraction. In: Proceedings of the 3rd Machine Learning for Health Symposium. PMLR; 2023 [cited 2024 Feb 8]. pp. 82\u2013100. Available from: https:\/\/proceedings.mlr.press\/v225\/goel23a.html"},{"key":"2871_CR19","doi-asserted-by":"crossref","first-page":"101698","DOI":"10.1016\/j.csl.2024.101698","volume":"89","author":"B \u00dcnl\u00fctabak","year":"2025","unstructured":"\u00dcnl\u00fctabak B, Bal O. Theory of mind performance of large language models: a comparative analysis of Turkish and English. Comput Speech Lang. 2025;89:101698.","journal-title":"Comput Speech Lang"},{"key":"2871_CR20","unstructured":"Penedo G, Malartic Q, Hesslow D, Cojocaru R, Cappelli A, Alobeidli H et al. The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only. arXiv; 2023 [cited 2024 Sep 10]. Available from: http:\/\/arxiv.org\/abs\/2306.01116"},{"key":"2871_CR21","unstructured":"Ullman T. Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks. arXiv; 2023 [cited 2024 Sep 10]. Available from: http:\/\/arxiv.org\/abs\/2302.08399"},{"key":"2871_CR22","doi-asserted-by":"publisher","unstructured":"Nguyen-Dinh LV, Rossi M, Blanke U, Tr\u00f6ster G. Combining crowd-generated media and personal data: semi-supervised learning for context recognition. In: Proceedings of the 1st ACM international workshop on Personal data meets distributed multimedia. New York, NY, USA: Association for Computing Machinery; 2013 [cited 2024 Feb 7]. pp. 35\u20138. (PDM \u201913). Available from: https:\/\/doi.org\/10.1145\/2509352.2509396","DOI":"10.1145\/2509352.2509396"},{"issue":"6266","key":"2871_CR23","doi-asserted-by":"crossref","first-page":"1332","DOI":"10.1126\/science.aab3050","volume":"350","author":"BM Lake","year":"2015","unstructured":"Lake BM, Salakhutdinov R, Tenenbaum JB. Human-level concept learning through probabilistic program induction. Science. 2015;350(6266):1332\u20138.","journal-title":"Science"},{"key":"2871_CR24","doi-asserted-by":"crossref","unstructured":"Mozafari B, Sarkar P, Franklin M, Jordan M, Madden S. Scaling up crowd-sourcing to very large datasets: a case for active learning. Proc VLDB Endow. 2014 Ekim;8(2):125\u201336.","DOI":"10.14778\/2735471.2735474"},{"issue":"12","key":"2871_CR25","doi-asserted-by":"crossref","first-page":"255","DOI":"10.3390\/fi11120255","volume":"11","author":"L Qing","year":"2019","unstructured":"Qing L, Linhong W, Xuehai D. A novel neural network-based method for Medical text classification. Future Internet. 2019;11(12):255.","journal-title":"Future Internet"},{"issue":"1","key":"2871_CR26","doi-asserted-by":"crossref","first-page":"486","DOI":"10.1186\/s12859-022-05035-9","volume":"23","author":"EB Lee","year":"2022","unstructured":"Lee EB, Heo GE, Choi CM, Song M. MLM-based typographical error correction of unstructured medical texts for named entity recognition. BMC Bioinformatics. 2022;23(1):486.","journal-title":"BMC Bioinformatics"},{"issue":"5p2","key":"2871_CR27","doi-asserted-by":"crossref","first-page":"1620","DOI":"10.1111\/j.1475-6773.2005.00444.x","volume":"40","author":"KJ O\u2019Malley","year":"2005","unstructured":"O\u2019Malley KJ, Cook KF, Price MD, Wildes KR, Hurdle JF, Ashton CM. Measuring diagnoses: ICD Code Accuracy. Health Serv Res. 2005;40(5p2):1620\u201339.","journal-title":"Health Serv Res"},{"key":"2871_CR28","doi-asserted-by":"crossref","unstructured":"Kim J, Kim T, Choi JH, Choo J. End-to-end Multi-task Learning of Missing Value Imputation and Forecasting in Time-Series Data. In: 2020 25th International Conference on Pattern Recognition (ICPR). 2021 [cited 2024 Feb 8]. pp. 8849\u201356. Available from: https:\/\/ieeexplore.ieee.org\/document\/9412112","DOI":"10.1109\/ICPR48806.2021.9412112"},{"key":"2871_CR29","doi-asserted-by":"publisher","unstructured":"Muller M, Wolf CT, Andres J, Desmond M, Joshi NN, Ashktorab Z et al. Designing Ground Truth and the Social Life of Labels. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: Association for Computing Machinery; 2021 [cited 2024 Feb 7]. pp. 1\u201316. (CHI \u201921). Available from: https:\/\/doi.org\/10.1145\/3411764.3445402","DOI":"10.1145\/3411764.3445402"},{"key":"2871_CR30","doi-asserted-by":"crossref","first-page":"104403","DOI":"10.1016\/j.jbi.2023.104403","volume":"143","author":"L Murali","year":"2023","unstructured":"Murali L, Gopakumar G, Viswanathan DM, Nedungadi P. Towards electronic health record-based medical knowledge graph construction, completion, and applications: a literature study. J Biomed Inf. 2023;143:104403.","journal-title":"J Biomed Inf"},{"key":"2871_CR31","doi-asserted-by":"crossref","first-page":"102701","DOI":"10.1016\/j.artmed.2023.102701","volume":"146","author":"Jah Sim","year":"2023","unstructured":"Sim Jah, Huang X, Horan MR, Stewart CM, Robison LL, Hudson MM, et al. Natural language processing with machine learning methods to analyze unstructured patient-reported outcomes derived from electronic health records: a systematic review. Artif Intell Med. 2023;146:102701.","journal-title":"Artif Intell Med"},{"key":"2871_CR32","doi-asserted-by":"crossref","first-page":"100511","DOI":"10.1016\/j.cosrev.2022.100511","volume":"46","author":"I Li","year":"2022","unstructured":"Li I, Pan J, Goldwasser J, Verma N, Wong WP, Nuzumlal\u0131 MY, et al. Neural Natural Language Processing for unstructured data in electronic health records: a review. Comput Sci Rev. 2022;46:100511.","journal-title":"Comput Sci Rev"},{"issue":"1","key":"2871_CR33","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1007\/s10579-018-9431-1","volume":"54","author":"Y Wang","year":"2020","unstructured":"Wang Y, Afzal N, Fu S, Wang L, Shen F, Rastegar-Mojarad M, et al. MedSTS: a resource for clinical semantic textual similarity. Lang Resour Eval. 2020;54(1):57\u201372.","journal-title":"Lang Resour Eval"},{"issue":"1","key":"2871_CR34","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1109\/TCBB.2018.2849968","volume":"16","author":"Z Zeng","year":"2019","unstructured":"Zeng Z, Deng Y, Li X, Naumann T, Luo Y. Natural Language Processing for EHR-Based computational phenotyping. IEEE\/ACM Trans Comput Biol Bioinform. 2019;16(1):139\u201353.","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"key":"2871_CR35","doi-asserted-by":"crossref","unstructured":"Kundeti SR, Vijayananda J, Mujjiga S, Kalyan M. Clinical named entity recognition: Challenges and opportunities. In: 2016 IEEE International Conference on Big Data (Big Data). 2016 [cited 2024 Feb 11]. pp. 1937\u201345. Available from: https:\/\/ieeexplore.ieee.org\/abstract\/document\/7840814","DOI":"10.1109\/BigData.2016.7840814"},{"key":"2871_CR36","doi-asserted-by":"crossref","first-page":"105122","DOI":"10.1016\/j.ijmedinf.2023.105122","volume":"177","author":"D Fraile Navarro","year":"2023","unstructured":"Fraile Navarro D, Ijaz K, Rezazadegan D, Rahimi-Ardabili H, Dras M, Coiera E, et al. Clinical named entity recognition and relation extraction using natural language processing of medical free text: a systematic review. Int J Med Inf. 2023;177:105122.","journal-title":"Int J Med Inf"},{"issue":"9","key":"2871_CR37","doi-asserted-by":"crossref","first-page":"1268","DOI":"10.3390\/healthcare11091268","volume":"11","author":"PN Ahmad","year":"2023","unstructured":"Ahmad PN, Shah AM, Lee K. A review on Electronic Health Record text-mining for Biomedical Name Entity Recognition in Healthcare Domain. Healthcare. 2023;11(9):1268.","journal-title":"Healthcare"},{"key":"2871_CR38","unstructured":"Hersh WR, Campbell EM, Malveau SE. Assessing the feasibility of large-scale natural language processing in a corpus of ordinary medical records: a lexical analysis. Proc Conf Am Med Inform Assoc AMIA Fall Symp. 1997;580\u20134."},{"key":"2871_CR39","unstructured":"Zhou L, Mahoney LM, Shakurova A, Goss F, Chang FY, Bates DW et al. How Many Medication Orders are Entered through Free-text in EHRs? - A Study on Hypoglycemic Agents. AMIA Annu Symp Proc. 2012;2012:1079\u201388."},{"issue":"2","key":"2871_CR40","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1017\/S1351324922000110","volume":"29","author":"A Hamdi","year":"2023","unstructured":"Hamdi A, Pontes EL, Sidere N, Coustaty M, Doucet A. In-depth analysis of the impact of OCR errors on named entity recognition and linking. Nat Lang Eng. 2023;29(2):425\u201348.","journal-title":"Nat Lang Eng"},{"key":"2871_CR41","doi-asserted-by":"crossref","unstructured":"Fetahu B, Chen Z, Kar S, Rokhlenko O, Malmasi S. arXiv.org. 2023 [cited 2024 Feb 11]. MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition. Available from: https:\/\/arxiv.org\/abs\/2310.13213v1","DOI":"10.18653\/v1\/2023.findings-emnlp.134"},{"issue":"4","key":"2871_CR42","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1002\/hcs2.61","volume":"2","author":"R Yang","year":"2023","unstructured":"Yang R, Tan TF, Lu W, Thirunavukarasu AJ, Ting DSW, Liu N. Large language models in health care: development, applications, and challenges. Health Care Sci. 2023;2(4):255\u201363.","journal-title":"Health Care Sci"},{"issue":"1","key":"2871_CR43","doi-asserted-by":"crossref","first-page":"114","DOI":"10.3390\/digital4010005","volume":"4","author":"CEA Coello","year":"2024","unstructured":"Coello CEA, Alimam MN, Kouatly R. Effectiveness of ChatGPT in Coding: a comparative analysis of Popular large Language models. Digital. 2024;4(1):114\u201325.","journal-title":"Digital"},{"key":"2871_CR44","doi-asserted-by":"publisher","unstructured":"Knebel D, Priglinger S, Scherer N, Siedlecki J, Schworm B. Assessment of ChatGPT in the preclinical management of ophthalmological emergencies\u2013 an analysis of ten fictional case vignettes. medRxiv; 2023 [cited 2024 Feb 8]. p. 2023.04.16.23288645. Available from: https:\/\/www.medrxiv.org\/content\/https:\/\/doi.org\/10.1101\/2023.04.16.23288645v1","DOI":"10.1101\/2023.04.16.23288645v1"},{"key":"2871_CR45","doi-asserted-by":"publisher","unstructured":"Nastasi AJ, Courtright KR, Halpern SD, Weissman GE. Does ChatGPT Provide Appropriate and Equitable Medical Advice? A Vignette-Based, Clinical Evaluation Across Care Contexts. medRxiv; 2023 [cited 2024 Feb 8]. p. 2023.02.25.23286451. Available from: https:\/\/www.medrxiv.org\/content\/https:\/\/doi.org\/10.1101\/2023.02.25.23286451v1","DOI":"10.1101\/2023.02.25.23286451v1"},{"key":"2871_CR46","doi-asserted-by":"publisher","unstructured":"Rao A, Pang M, Kim J, Kamineni M, Lie W, Prasad AK et al. Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow. medRxiv; 2023 [cited 2024 Feb 8]. p. 2023.02.21.23285886. Available from: https:\/\/www.medrxiv.org\/content\/https:\/\/doi.org\/10.1101\/2023.02.21.23285886v1","DOI":"10.1101\/2023.02.21.23285886v1"},{"issue":"1","key":"2871_CR47","doi-asserted-by":"crossref","first-page":"e49995","DOI":"10.2196\/49995","volume":"11","author":"H Fraser","year":"2023","unstructured":"Fraser H, Crossland D, Bacher I, Ranney M, Madsen T, Hilliard R. Comparison of Diagnostic and Triage Accuracy of Ada Health and WebMD Symptom checkers, ChatGPT, and Physicians for patients in an Emergency Department: Clinical Data Analysis Study. JMIR MHealth UHealth. 2023;11(1):e49995.","journal-title":"JMIR MHealth UHealth"},{"key":"2871_CR48","doi-asserted-by":"publisher","unstructured":"Takita H, Walston SL, Tatekawa H, Saito K, Tsujimoto Y, Miki Y et al. Diagnostic Performance of Generative AI and Physicians: A Systematic Review and Meta-Analysis. medRxiv; 2024 [cited 2024 Feb 11]. p. 2024.01.20.24301563. Available from: https:\/\/www.medrxiv.org\/content\/https:\/\/doi.org\/10.1101\/2024.01.20.24301563v1","DOI":"10.1101\/2024.01.20.24301563v1"},{"key":"2871_CR49","doi-asserted-by":"publisher","unstructured":"Roso\u0142 M, G\u0105sior JS, \u0141aba J, Korzeniewski K, M\u0142y\u0144czak M. Evaluation of the performance of GPT\u20133.5 and GPT\u20134 on the Medical Final Examination. medRxiv; 2023 [cited 2024 Feb 10]. p. 2023.06.04.23290939. Available from: https:\/\/www.medrxiv.org\/content\/https:\/\/doi.org\/10.1101\/2023.06.04.23290939v2","DOI":"10.1101\/2023.06.04.23290939v2"},{"key":"2871_CR50","unstructured":"Nori H, King N, McKinney SM, Carignan D, Horvitz E. Capabilities of gpt\u20134 on medical challenge problems. ArXiv Prepr ArXiv230313375. 2023."},{"key":"2871_CR51","unstructured":"Singhal K, Tu T, Gottweis J, Sayres R, Wulczyn E, Hou L et al. Towards Expert-Level Medical Question Answering with Large Language Models. arXiv; 2023 [cited 2024 Feb 12]. Available from: http:\/\/arxiv.org\/abs\/2305.09617"},{"key":"2871_CR52","unstructured":"Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y et al. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv; 2023 [cited 2024 Feb 12]. Available from: http:\/\/arxiv.org\/abs\/2307.09288"},{"issue":"1","key":"2871_CR53","doi-asserted-by":"crossref","first-page":"bbad493","DOI":"10.1093\/bib\/bbad493","volume":"25","author":"S Tian","year":"2024","unstructured":"Tian S, Jin Q, Yeganova L, Lai PT, Zhu Q, Chen X, et al. Opportunities and challenges for ChatGPT and large language models in biomedicine and health. Brief Bioinform. 2024;25(1):bbad493.","journal-title":"Brief Bioinform"},{"key":"2871_CR54","doi-asserted-by":"crossref","unstructured":"Johnson D, Goodman R, Patrinely J, Stone C, Zimmerman E, Donald R et al. Assessing the accuracy and reliability of AI-generated medical responses: an evaluation of the Chat-GPT model. Res Sq. 2023.","DOI":"10.21203\/rs.3.rs-2566942\/v1"},{"key":"2871_CR55","doi-asserted-by":"crossref","unstructured":"Latif E, Zhai X. Fine-tuning chatgpt for automatic scoring. Comput Educ Artif Intell. 2024;100210.","DOI":"10.1016\/j.caeai.2024.100210"},{"issue":"7972","key":"2871_CR56","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","volume":"620","author":"K Singhal","year":"2023","unstructured":"Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172\u201380.","journal-title":"Nature"},{"key":"2871_CR57","first-page":"9459","volume":"33","author":"P Lewis","year":"2020","unstructured":"Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Adv Neural Inf Process Syst. 2020;33:9459\u201374.","journal-title":"Adv Neural Inf Process Syst"},{"key":"2871_CR58","unstructured":"Guu K, Lee K, Tung Z, Pasupat P, Chang M. Retrieval Augmented Language Model Pre-Training. In: International Conference on Machine Learning. PMLR; 2020 [cited 2024 Feb 12]. pp. 3929\u201338. Available from: https:\/\/proceedings.mlr.press\/v119\/guu20a.html"},{"key":"2871_CR59","doi-asserted-by":"crossref","unstructured":"Cuconasu F, Trappolini G, Siciliano F, Filice S, Campagnano C, Maarek Y et al. The Power of Noise: Redefining Retrieval for RAG Systems. arXiv; 2024 [cited 2024 Feb 12]. Available from: http:\/\/arxiv.org\/abs\/2401.14887","DOI":"10.1145\/3626772.3657834"},{"key":"2871_CR60","unstructured":"Zhang L, Jijo K, Setty S, Chung E, Javid F, Vidra N et al. Enhancing Large Language Model Performance To Answer Questions and Extract Information More Accurately. arXiv; 2024 [cited 2024 Feb 12]. Available from: http:\/\/arxiv.org\/abs\/2402.01722"},{"issue":"6","key":"2871_CR61","doi-asserted-by":"crossref","first-page":"bbac409","DOI":"10.1093\/bib\/bbac409","volume":"23","author":"R Luo","year":"2022","unstructured":"Luo R, Sun L, Xia Y, Qin T, Zhang S, Poon H, et al. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Brief Bioinform. 2022;23(6):bbac409.","journal-title":"Brief Bioinform"},{"key":"2871_CR62","doi-asserted-by":"crossref","unstructured":"Naik A, Parasa S, Feldman S, Wang LL, Hope T. Literature-Augmented Clinical Outcome Prediction. arXiv; 2022 [cited 2024 Feb 12]. Available from: http:\/\/arxiv.org\/abs\/2111.08374","DOI":"10.18653\/v1\/2022.findings-naacl.33"},{"issue":"2","key":"2871_CR63","doi-asserted-by":"crossref","first-page":"AIoa2300068","DOI":"10.1056\/AIoa2300068","volume":"1","author":"C Zakka","year":"2024","unstructured":"Zakka C, Shad R, Chaurasia A, Dalal AR, Kim JL, Moor M, et al. Almanac \u2014 Retrieval-Augmented Language models for Clinical Medicine. NEJM AI. 2024;1(2):AIoa2300068.","journal-title":"NEJM AI"},{"key":"2871_CR64","unstructured":"Balaguer A, Benara V, Cunha RL, de Filho F, de Hendry R, Holstein T. D, RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture. arXiv; 2024 [cited 2024 Feb 12]. Available from: http:\/\/arxiv.org\/abs\/2401.08406"},{"key":"2871_CR65","doi-asserted-by":"crossref","unstructured":"Wang S, Liu Y, Xu Y, Zhu C, Zeng M. Want To Reduce Labeling Cost? GPT\u20133 Can Help. arXiv; 2021 [cited 2024 Feb 11]. Available from: http:\/\/arxiv.org\/abs\/2108.13487","DOI":"10.18653\/v1\/2021.findings-emnlp.354"},{"key":"2871_CR66","unstructured":"Guti\u00e9rrez BJ, McNeal N, Washington C, Chen Y, Li L, Sun H et al. Thinking about GPT\u20133 In-Context Learning for Biomedical IE? Think Again. arXiv; 2022 [cited 2024 Jun 9]. Available from: http:\/\/arxiv.org\/abs\/2203.08410"},{"key":"2871_CR67","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1038\/s41746-023-00873-0","volume":"6","author":"B Mesk\u00f3","year":"2023","unstructured":"Mesk\u00f3 B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med. 2023;6:120.","journal-title":"NPJ Digit Med"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-025-02871-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-025-02871-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-025-02871-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,31]],"date-time":"2025-03-31T11:18:31Z","timestamp":1743419911000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-025-02871-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,31]]},"references-count":67,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["2871"],"URL":"https:\/\/doi.org\/10.1186\/s12911-025-02871-6","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-4014476\/v1","asserted-by":"object"}]},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,31]]},"assertion":[{"value":"4 March 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 January 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 March 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The Hacettepe University Clinical Research Ethics Committee approved our study\u2019s design and procedures under protocol number GO-23\/508, ensuring adherence to the ethical standards in clinical research. The data sourced from Hacettepe University \u0130hsan Do\u011framac\u0131 Children\u2019s Hospital, which underwent a de-identification process through the redaction of protected health information, received approval for utilization in a quality improvement project by the hospital. In this context, the Hacettepe University Research Ethics Board granted a waiver for the necessity of its approval and the procurement of informed consent for this study. Furthermore, all procedures complied with the relevant guidelines and standards outlined in the Declaration of Helsinki.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"154"}}