{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T10:24:57Z","timestamp":1785839097917,"version":"3.56.0"},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2024,12,20]],"date-time":"2024-12-20T00:00:00Z","timestamp":1734652800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100012986","name":"Universidad Francisco de Vitoria","doi-asserted-by":"publisher","award":["UFV2024-05"],"award-info":[{"award-number":["UFV2024-05"]}],"id":[{"id":"10.13039\/100012986","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,2,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>We evaluate the effectiveness of large language models (LLMs), specifically GPT-based (GPT-3.5 and GPT-4) and Llama-2 models (13B and 7B architectures), in autonomously assessing clinical records (CRs) to enhance medical education and diagnostic skills.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>Various techniques, including prompt engineering, fine-tuning (FT), and low-rank adaptation (LoRA), were implemented and compared on Llama-2 7B. These methods were assessed using prompts in both English and Spanish to determine their adaptability to different languages. Performance was benchmarked against GPT-3.5, GPT-4, and Llama-2 13B.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>GPT-based models, particularly GPT-4, demonstrated promising performance closely aligned with specialist evaluations. Application of FT on Llama-2 7B improved text comprehension in Spanish, equating its performance to that of Llama-2 13B with English prompts. Low-rank adaptation significantly enhanced performance, surpassing GPT-3.5 results when combined with FT. This indicates LoRA\u2019s effectiveness in adapting open-source models for specific tasks.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>While GPT-4 showed superior performance, FT and LoRA on Llama-2 7B proved crucial in improving language comprehension and task-specific accuracy. Identified limitations highlight the need for further research.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>This study underscores the potential of LLMs in medical education, providing an innovative, effective approach to CR correction. Low-rank adaptation emerged as the most effective technique, enabling open-source models to perform on par with proprietary models. Future research should focus on overcoming current limitations to further improve model performance.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae302","type":"journal-article","created":{"date-parts":[[2024,12,21]],"date-time":"2024-12-21T04:34:34Z","timestamp":1734755674000},"page":"341-348","source":"Crossref","is-referenced-by-count":7,"title":["Application of large language models in clinical record correction: a comprehensive study on various retraining methods"],"prefix":"10.1093","volume":"32","author":[{"given":"Ana M","family":"Maitin","sequence":"first","affiliation":[{"name":"CEIEC, Universidad Francisco de Vitoria , Pozuelo de Alarc\u00f3n, 28223 Madrid,","place":["Spain"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alberto","family":"Nogales","sequence":"additional","affiliation":[{"name":"CEIEC, Universidad Francisco de Vitoria , Pozuelo de Alarc\u00f3n, 28223 Madrid,","place":["Spain"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sergio","family":"Fern\u00e1ndez-Rinc\u00f3n","sequence":"additional","affiliation":[{"name":"CEIEC, Universidad Francisco de Vitoria , Pozuelo de Alarc\u00f3n, 28223 Madrid,","place":["Spain"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Enrique","family":"Aranguren","sequence":"additional","affiliation":[{"name":"CEIEC, Universidad Francisco de Vitoria , Pozuelo de Alarc\u00f3n, 28223 Madrid,","place":["Spain"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emilio","family":"Cervera-Barba","sequence":"additional","affiliation":[{"name":"Facultad de Medicina, Universidad Francisco de Vitoria , Pozuelo de Alarc\u00f3n, 28223 Madrid,","place":["Spain"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sophia","family":"Denizon-Arranz","sequence":"additional","affiliation":[{"name":"Facultad de Medicina, Universidad Francisco de Vitoria , Pozuelo de Alarc\u00f3n, 28223 Madrid,","place":["Spain"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alonso","family":"Mateos-Rodr\u00edguez","sequence":"additional","affiliation":[{"name":"Facultad de Medicina, Universidad Francisco de Vitoria , Pozuelo de Alarc\u00f3n, 28223 Madrid,","place":["Spain"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"\u00c1lvaro J","family":"Garc\u00eda-Tejedor","sequence":"additional","affiliation":[{"name":"CEIEC, Universidad Francisco de Vitoria , Pozuelo de Alarc\u00f3n, 28223 Madrid,","place":["Spain"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,12,20]]},"reference":[{"key":"2025012119040174100_ocae302-B1","doi-asserted-by":"crossref","first-page":"329","DOI":"10.7861\/clinmedicine.3-4-329","article-title":"Standards in medical record keeping","volume":"3","author":"Mann","year":"2003","journal-title":"Clin Med (Lond)"},{"key":"2025012119040174100_ocae302-B2","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1192\/apt.12.4.280","article-title":"Improving standards in clinical record-keeping","volume":"12","author":"Pullen","year":"2006","journal-title":"Adv Psychiatr Treat"},{"key":"2025012119040174100_ocae302-B3","doi-asserted-by":"crossref","first-page":"1539","DOI":"10.1016\/S0140-6736(98)08308-1","article-title":"How to limit clinical errors in interpretation of data","volume":"352","author":"Wright","year":"1998","journal-title":"Lancet"},{"key":"2025012119040174100_ocae302-B4","doi-asserted-by":"crossref","first-page":"255","DOI":"10.7861\/clinmedicine.4-3-255","article-title":"The need for mandatory clinical recording standards","volume":"4","author":"Huston","year":"2004","journal-title":"Clin Med (Lond)"},{"key":"2025012119040174100_ocae302-B5","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1080\/01421590903002821","article-title":"The use of simulated patients in medical education: AMEE Guide No 42","volume":"31","author":"Cleland","year":"2009","journal-title":"Med Teach"},{"key":"2025012119040174100_ocae302-B6","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1097\/SIH.0b013e318262007e","article-title":"Characteristics of simulation activities at North American medical schools and teaching hospitals: an AAMC-SSH-ASPE-AACN collaboration","volume":"7","author":"Huang","year":"2012","journal-title":"Simul Healthc J Soc Simul Healthc"},{"key":"2025012119040174100_ocae302-B7","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2025012119040174100_ocae302-B8","author":"Vaswani","year":"2017"},{"key":"2025012119040174100_ocae302-B9","first-page":"374","author":"Li","year":"2023"},{"key":"2025012119040174100_ocae302-B10","first-page":"5","author":"Singla","year":"2023"},{"key":"2025012119040174100_ocae302-B11","author":"McDuff","year":"2023"},{"key":"2025012119040174100_ocae302-B12","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1272\/jnms.JNMS.2024_91-205","article-title":"Performance of a large language model on Japanese Emergency Medicine Board Certification Examinations","volume":"91","author":"Igarashi","year":"2024","journal-title":"J Nippon Med Sch"},{"key":"2025012119040174100_ocae302-B13","author":"Liu","year":"2023"},{"key":"2025012119040174100_ocae302-B14","doi-asserted-by":"crossref","first-page":"109713","DOI":"10.1016\/j.isci.2024.109713","article-title":"The application of large language models in medicine: a scoping review","volume":"27","author":"Meng","year":"2024","journal-title":"iScience"},{"key":"2025012119040174100_ocae302-B15","author":"Chuang","year":"2023"},{"key":"2025012119040174100_ocae302-B16"},{"key":"2025012119040174100_ocae302-B17","author":"OpenAI","year":"2023"},{"key":"2025012119040174100_ocae302-B18","author":"Touvron","year":"2023"},{"key":"2025012119040174100_ocae302-B19","doi-asserted-by":"crossref","first-page":"3041","DOI":"10.3390\/jcm13113041","article-title":"Clinical and surgical applications of large language models: a systematic review","volume":"13","author":"Pressman","year":"2024","journal-title":"J Clin Med"},{"key":"2025012119040174100_ocae302-B20","doi-asserted-by":"crossref","first-page":"1097","DOI":"10.3390\/biomedinformatics4020062","article-title":"Recent advances in large language models for healthcare","volume":"4","author":"Nassiri","year":"2024","journal-title":"BioMedInformatics"},{"key":"2025012119040174100_ocae302-B21","doi-asserted-by":"crossref","first-page":"57","DOI":"10.3390\/informatics11030057","article-title":"Large language models in healthcare and medical domain: a review","volume":"11","author":"Nazi","year":"2024","journal-title":"Informatics"},{"key":"2025012119040174100_ocae302-B22","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1002\/hcs2.61","article-title":"Large language models in health care: development, applications, and challenges","volume":"2","author":"Yang","year":"2023","journal-title":"Health Care Sci"},{"key":"2025012119040174100_ocae302-B23","author":"Sallam","year":"2023"},{"key":"2025012119040174100_ocae302-B24","doi-asserted-by":"crossref","first-page":"776","DOI":"10.1093\/jamia\/ocad252","article-title":"Harnessing the potential of large language models in medical education: promise and pitfalls","volume":"31","author":"Ben\u00edtez","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"2025012119040174100_ocae302-B25","doi-asserted-by":"crossref","first-page":"e48291","DOI":"10.2196\/48291","article-title":"Large language models in medical education: opportunities, challenges, and future directions","volume":"9","author":"Abd-Alrazaq","year":"2023","journal-title":"JMIR Med Educ"},{"key":"2025012119040174100_ocae302-B26","author":"Labrak","year":"2024"},{"key":"2025012119040174100_ocae302-B27","first-page":"387","article-title":"Origin and development of the book Medical Semiology","volume":"146","author":"Goic","year":"2018","journal-title":"Rev M\u00e9dica Chile"},{"key":"2025012119040174100_ocae302-B28","author":"Radford"},{"key":"2025012119040174100_ocae302-B29"},{"key":"2025012119040174100_ocae302-B30","first-page":"1877","author":"Brown","year":"2020"},{"key":"2025012119040174100_ocae302-B31","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei","year":"2022","journal-title":"Adv Neural Inf Process Syst"},{"key":"2025012119040174100_ocae302-B32","author":"Dodge","year":"2020"},{"key":"2025012119040174100_ocae302-B33","author":"Hu","year":"2021"},{"key":"2025012119040174100_ocae302-B34","author":"Singh","year":"2023"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/2\/341\/61250513\/ocae302.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/2\/341\/61250513\/ocae302.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T19:04:13Z","timestamp":1737486253000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/32\/2\/341\/7929961"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,20]]},"references-count":34,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,12,20]]},"published-print":{"date-parts":[[2025,2,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae302","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,2]]},"published":{"date-parts":[[2024,12,20]]}}}