{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T13:51:14Z","timestamp":1784641874385,"version":"3.55.0"},"reference-count":22,"publisher":"BMJ","issue":"4","license":[{"start":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T00:00:00Z","timestamp":1734566400000},"content-version":"unspecified","delay-in-days":18,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100030888","name":"Fondation priv\u00e9e des H\u00f4pitaux universitaires de Gen\u00e8ve","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100030888","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["RMD Open"],"accepted":{"date-parts":[[2024,11,27]]},"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>We propose and test a framework to detect disease diagnosis using a recent large language model (LLM), Meta\u2019s Llama-3-8B, on French-language electronic health record (EHR) documents. Specifically, it focuses on detecting gout (\u2018goutte\u2019 in French), a ubiquitous French term that has multiple meanings beyond the disease. The study compares the performance of the LLM-based framework with traditional natural language processing techniques and tests its dependence on the parameter used.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>The framework was developed using a training and testing set of 700 paragraphs assessing \u2018gout\u2019 from a random selection of EHR documents from a tertiary university hospital in Geneva, Switzerland. All paragraphs were manually reviewed and classified by two healthcare professionals into disease (true gout) and non-disease (gold standard). The LLM\u2019s accuracy was tested using few-shot and chain-of-thought prompting and compared with a regular expression (regex)-based method, focusing on the effects of model parameters and prompt structure. The framework was further validated on 600 paragraphs assessing \u2018Calcium Pyrophosphate Deposition Disease (CPPD)\u2019.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The LLM-based algorithm outperformed the regex method, achieving a 92.7% (88.7%\u201395.4%) positive predictive value, a 96.6% (94.6%\u201397.8%) negative predictive value and an accuracy of 95.4% (93.6%\u201396.7%) for gout. In the validation set on CPPD, accuracy was 94.1% (90.2%\u201397.6%). The LLM framework performed well over a wide range of parameter values.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>LLMs accurately detected disease diagnoses from EHRs, even in non-English languages. They could facilitate creating large disease registers in any language, improving disease care assessment and patient recruitment for clinical trials.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1136\/rmdopen-2024-005003","type":"journal-article","created":{"date-parts":[[2024,12,21]],"date-time":"2024-12-21T03:59:45Z","timestamp":1734753585000},"page":"e005003","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":12,"title":["Large language models for accurate disease detection in electronic health records: the examples of crystal arthropathies"],"prefix":"10.1136","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-3375-1985","authenticated-orcid":false,"given":"Nils","family":"B\u00fcrgisser","sequence":"first","affiliation":[{"name":"Division of Rheumatology, Geneva University Hospitals, Geneva, Switzerland"},{"name":"Division of Internal Medicine, Geneva University Hospitals, Geneva, Switzerland"},{"name":"Faculty of Medicine, University of Geneva, Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Etienne","family":"Chalot","sequence":"additional","affiliation":[{"name":"Information Systems Directorate, Geneva University Hospitals, Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samia","family":"Mehouachi","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Geneva University Hospitals, Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Clement P.","family":"Buclin","sequence":"additional","affiliation":[{"name":"Division of Internal Medicine, Geneva University Hospitals, Geneva, Switzerland"},{"name":"Faculty of Medicine, University of Geneva, Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4315-9009","authenticated-orcid":false,"given":"Kim","family":"Lauper","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Geneva University Hospitals, Geneva, Switzerland"},{"name":"Faculty of Medicine, University of Geneva, Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1956-2607","authenticated-orcid":false,"given":"Delphine S.","family":"Courvoisier","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Geneva University Hospitals, Geneva, Switzerland"},{"name":"Quality of Care Division, Geneva University Hospitals, Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4801-8395","authenticated-orcid":false,"given":"Denis","family":"Mongin","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Geneva University Hospitals, Geneva, Switzerland"},{"name":"Faculty of Medicine, University of Geneva, Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2024,12,20]]},"reference":[{"key":"2025090312555198000_10.4.e005003.1","unstructured":"Devlin J , Chang MW , Lee K , et al . BERT: pre-training of deep bidirectional transformers for language understanding. arXiv 2019. doi:10.48550\/arXiv.1810.04805"},{"key":"2025090312555198000_10.4.e005003.2","unstructured":"Fan L , Li L , Ma Z , et al . A Bibliometric Review of Large Language Models Research from 2017 to 2023. arXiv 2023. doi:10.48550\/arXiv.2304.02020"},{"key":"2025090312555198000_10.4.e005003.3","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2023.46721","article-title":"Performance of Large Language Models on a Neurology Board-Style Examination","volume":"6","author":"Schubert","year":"2023","journal-title":"JAMA Netw Open"},{"key":"2025090312555198000_10.4.e005003.4","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2023.30320","article-title":"Comparison of Ophthalmologist and Large Language Model Chatbot Responses to Online Patient Eye Care Questions","volume":"6","author":"Bernstein","year":"2023","journal-title":"JAMA Netw Open"},{"key":"2025090312555198000_10.4.e005003.5","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-022-00742-2","article-title":"A large language model for electronic health records","volume":"5","author":"Yang","year":"2022","journal-title":"NPJ Digit Med"},{"key":"2025090312555198000_10.4.e005003.6","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1053\/j.gastro.2023.12.019","article-title":"A Comparison of a Large Language Model vs Manual Chart Review for the Extraction of Data Elements From the Electronic Health Record","volume":"166","author":"Ge","year":"2024","journal-title":"Gastroenterology"},{"key":"2025090312555198000_10.4.e005003.7","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2024.13208","article-title":"Use of a Large Language Model to Identify and Classify Injuries With Free-Text Emergency Department Data","volume":"7","author":"Lorenzoni","year":"2024","journal-title":"JAMA Netw Open"},{"key":"2025090312555198000_10.4.e005003.8","doi-asserted-by":"crossref","DOI":"10.1136\/rmdopen-2024-004120","article-title":"Development and validation of a self-updating gout register from electronic health records data","volume":"10","author":"B\u00fcrgisser","year":"2024","journal-title":"RMD Open"},{"key":"2025090312555198000_10.4.e005003.9","unstructured":"Naveed H , Khan AU , Qiu S , et al . A comprehensive overview of large language models. arXiv 2024. doi:10.48550\/arXiv.2307.06435"},{"key":"2025090312555198000_10.4.e005003.10","unstructured":"Touvron H , Lavril T , Izacard G , et al . LLaMA: open and efficient foundation language models. arXiv 2023. doi:10.48550\/arXiv.2302.13971"},{"key":"2025090312555198000_10.4.e005003.11","unstructured":"Brown T , Mann B , Ryder N , et al . Language models are few-shot learners. In: Advances in neural information processing systems. Curran Associates, Inc, 2020: 1877\u2013901."},{"key":"2025090312555198000_10.4.e005003.12","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2023.35924","article-title":"Evaluating the Application of Large Language Models in Clinical Research Contexts","volume":"6","author":"Perlis","year":"2023","journal-title":"JAMA Netw Open"},{"key":"2025090312555198000_10.4.e005003.13","doi-asserted-by":"crossref","unstructured":"Sahoo P , Singh AK , Saha S , et al . A systematic survey of prompt engineering in large language models: techniques and applications. arXiv 2024. doi:10.48550\/arXiv.2402.07927","DOI":"10.1007\/979-8-8688-0569-1_4"},{"key":"2025090312555198000_10.4.e005003.14","doi-asserted-by":"publisher","DOI":"10.2307\/2276774"},{"key":"2025090312555198000_10.4.e005003.15","unstructured":"R Core Team . R: a language and environment for statistical ## computing. Vienna, Austria R Foundation for Statistical Computing; 2021. Available: https:\/\/www.R-project.org\/"},{"key":"2025090312555198000_10.4.e005003.16","doi-asserted-by":"crossref","first-page":"1740","DOI":"10.1002\/acr.22324","article-title":"Using Natural Language Processing and Machine Learning to Identify Gout Flares From Electronic Clinical Notes","volume":"66","author":"Zheng","year":"2014","journal-title":"Arthritis Care Res (Hoboken)"},{"key":"2025090312555198000_10.4.e005003.17","first-page":"973","article-title":"Identification of Gout Flares in Chief Complaint Text Using Natural Language Processing","author":"Osborne","year":"2020","journal-title":"AMIA Annu Symp Proc"},{"key":"2025090312555198000_10.4.e005003.18","doi-asserted-by":"crossref","DOI":"10.1007\/s11926-019-0847-4","article-title":"Issues in CPPD Nomenclature and Classification","volume":"21","author":"Tedeschi","year":"2019","journal-title":"Curr Rheumatol Rep"},{"key":"2025090312555198000_10.4.e005003.19","unstructured":"Li Z , Shi Y , Liu Z , et al . Quantifying multilingual performance of large language models across languages. arXiv 2024. doi:10.48550\/arXiv.2404.11553"},{"key":"2025090312555198000_10.4.e005003.20","doi-asserted-by":"crossref","first-page":"2557","DOI":"10.1002\/art.39254","article-title":"2015 Gout Classification Criteria: an American College of Rheumatology\/European League Against Rheumatism collaborative initiative","volume":"67","author":"Neogi","year":"2015","journal-title":"Arthritis Rheumatol"},{"key":"2025090312555198000_10.4.e005003.21","doi-asserted-by":"crossref","first-page":"1703","DOI":"10.1002\/art.42619","article-title":"The 2023 American College of Rheumatology\/European Alliance of Associations for Rheumatology Classification Criteria for Calcium Pyrophosphate Deposition (CPPD) Disease","volume":"75","author":"Abhishek","year":"2023","journal-title":"Arthritis Rheumatol"},{"key":"2025090312555198000_10.4.e005003.22","doi-asserted-by":"crossref","first-page":"52","DOI":"10.3390\/gucdd2010005","article-title":"A New Era for Calcium Pyrophosphate Deposition Disease Research: The First-Ever Calcium Pyrophosphate Deposition Disease Classification Criteria and Considerations for Measuring Outcomes in Calcium Pyrophosphate Deposition Disease","volume":"2","author":"Tedeschi","year":"2024","journal-title":"GUCDD"}],"container-title":["RMD Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/rmdopen-2024-005003","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T19:56:01Z","timestamp":1756929361000},"score":1,"resource":{"primary":{"URL":"https:\/\/rmdopen.bmj.com\/lookup\/doi\/10.1136\/rmdopen-2024-005003"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12]]},"references-count":22,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,12,20]]},"published-print":{"date-parts":[[2024,12]]}},"alternative-id":["10.1136\/rmdopen-2024-005003"],"URL":"https:\/\/doi.org\/10.1136\/rmdopen-2024-005003","relation":{},"ISSN":["2056-5933"],"issn-type":[{"value":"2056-5933","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12]]}}}