{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:43:12Z","timestamp":1785544992365,"version":"3.56.0"},"reference-count":49,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Unstructured clinical text remains a major barrier to interoperable data reuse and large-scale secondary analysis in health care. Large language models (LLMs) have the potential to automate the extraction of structured clinical information; however, their application is limited by the scarcity of high-quality annotated training data.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n To address these limitations, this study aims to develop and validate a scalable, privacy-preserving framework that uses synthetic data generated from structured Fast Healthcare Interoperability Resources (FHIR) to fine-tune open-source LLMs for the effective extraction of interoperable clinical information from unstructured text.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We evaluated an LLM-based framework for extracting structured clinical information from cancer-related discharge letters and mapping it to representations compatible with FHIR. To enable large-scale supervised training, we developed a random sample generator that creates synthetic discharge letters using Qwen3-235B by randomly sampling and aggregating structured FHIR data from 41,175 patients with cancer. The resulting synthetic discharge letters (n=75,000) were paired with their originating structured data, forming a large-scale dataset for fine-tuning MedGemma 27B, a 27-billion-parameter medical language model. Evaluation was conducted on the synthetic test dataset (n=7500), real-world discharge letters (n=30), which were evaluated by physicians and a medical student, and a comparative one-shot approach using open-source models (Qwen3, LLaMA, and GPT-OSS).<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n The fine-tuned model achieved high extraction performance across multiple clinical entities on the synthetic test set, with\n F<\/jats:italic>\n 1<\/jats:sub>\n -scores of 0.84 for full\n International Classification of Diseases<\/jats:italic>\n diagnosis codes, 0.99 for tumor-related information, 0.99 for laboratory values, 0.99 for medication names and dosages, and 0.94 for Anatomical Therapeutic Chemical medication codes. The extraction of procedure-related information was more challenging, with\n F<\/jats:italic>\n 1<\/jats:sub>\n -scores of 0.63 for OPS codes and 0.90 for procedure descriptions. The fine-tuned model consistently outperformed general-purpose LLMs in a one-shot comparison across nearly all extraction categories. When evaluated by physicians on real-world discharge letters, the model achieved case-level correctness rates of 78.9% for\n International Classification of Diseases<\/jats:italic>\n diagnoses, 86.1% for tumor-related information, 93.0% for medications, and 61.3% for procedures.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n These results demonstrate that synthetic text generation from structured clinical data enables the effective and scalable training of LLMs for extracting interoperable, multientity clinical information from unstructured documentation.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/92413","type":"journal-article","created":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T07:15:07Z","timestamp":1779866107000},"page":"e92413-e92413","source":"Crossref","is-referenced-by-count":0,"title":["Extracting Medical Information From Unstructured Clinical Text Using Large Language Models to Enhance Health Care Interoperability: Proof-of-Concept Study"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-8743-4751","authenticated-orcid":false,"given":"Bahad\u0131r","family":"Ery\u0131lmaz","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6074-804X","authenticated-orcid":false,"given":"Kamyar","family":"Arzideh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0866-4023","authenticated-orcid":false,"given":"Mikel","family":"Bahn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7464-4293","authenticated-orcid":false,"given":"Hendrik","family":"Damm","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2262-2655","authenticated-orcid":false,"given":"Sina","family":"Warmer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4123-0406","authenticated-orcid":false,"given":"Henning","family":"Sch\u00e4fer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1507-9690","authenticated-orcid":false,"given":"Ahmad","family":"Idrissi-Yaghir","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-9802-7167","authenticated-orcid":false,"given":"Tabea M G","family":"Pakull","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0307-6006","authenticated-orcid":false,"given":"Lea Jessica","family":"Albrecht","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8686-0682","authenticated-orcid":false,"given":"Jens","family":"Kleesiek","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6930-2694","authenticated-orcid":false,"given":"Georg","family":"Lodde","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7906-0038","authenticated-orcid":false,"given":"Christoph M","family":"Friedrich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8279-9239","authenticated-orcid":false,"given":"Elisabeth","family":"Livingstone","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3524-7858","authenticated-orcid":false,"given":"Dirk","family":"Schadendorf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6987-6041","authenticated-orcid":false,"given":"Katarzyna","family":"Borys","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5811-7100","authenticated-orcid":false,"given":"Felix","family":"Nensa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1760-2342","authenticated-orcid":false,"given":"Ren\u00e9","family":"Hosch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,7,2]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1136\/amiajnl-2012-001145","article-title":"Next-generation phenotyping of electronic health records","volume":"20","author":"Hripcsak","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"R2","doi-asserted-by":"publisher","DOI":"10.1002\/wics.1549","article-title":"Challenges and opportunities beyond structured data in analysis of electronic health records","volume":"13","author":"Tayefi","journal-title":"WIREs Comput Stat"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4258\/hir.2019.25.1.1","article-title":"Managing unstructured big data in healthcare system","volume":"25","author":"Kong","journal-title":"Healthc Inform Res"},{"issue":"7","key":"R4","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1097\/CIN.0000000000000609","article-title":"Challenges frequently encountered in the secondary use of electronic medical record data for research","volume":"38","author":"Edmondson","journal-title":"Comput Inform Nurs"},{"key":"R5","doi-asserted-by":"publisher","DOI":"10.2196\/69853","article-title":"Lessons learned from building a data platform for longitudinal, analytical use cases and scaling to 77 German hospitals: implementation report","volume":"13","author":"Bockhacker","journal-title":"JMIR Med Inform"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.2196\/63902","article-title":"Exploring physicians\u2019 dual perspectives on the transition from free text to structured and standardized documentation practices: interview and participant observational study","volume":"9","author":"Golburean","journal-title":"JMIR Form Res"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2017.11.011","article-title":"Clinical information extraction applications: a literature review","volume":"77","author":"Wang","journal-title":"J Biomed Inform"},{"key":"R8","doi-asserted-by":"crossref","unstructured":"Bender D Sartipi K . HL7 FHIR: an Agile and RESTful approach to healthcare information exchange. Presented at: Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems; Jun 20-22, 2013; Porto, Portugal. [doi: 10.1109\/CBMS.2013.6627810]","DOI":"10.1109\/CBMS.2013.6627810"},{"issue":"S 01","key":"R9","doi-asserted-by":"publisher","first-page":"e50","DOI":"10.3414\/ME18-03-0003","article-title":"German Medical Informatics Initiative","volume":"57","author":"Semler","journal-title":"Methods Inf Med"},{"key":"R10","unstructured":"Regulation (EU) 2025\/327 of the European Parliament and of the Council of 11 February 2025 on the European Health Data Space and amending Directive 2011\/24\/EU and Regulation (EU) 2024\/2847 (Text with EEA relevance). European Union; 2025. URL: http:\/\/data.europa.eu\/eli\/reg\/2025\/327\/oj [Accessed 22-12-2025]"},{"issue":"8","key":"R11","doi-asserted-by":"publisher","DOI":"10.1056\/aics2300301","article-title":"FHIR-GPT enhances health interoperability with large language models","volume":"1","author":"Li","journal-title":"NEJM AI"},{"key":"R12","doi-asserted-by":"crossref","unstructured":"Wang J Mathews WC Pham HA Xu H Zhang Y . Opioid2FHIR: a system for extracting FHIR-compatible opioid prescriptions from clinical text. Presented at: 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM); Dec 16-19, 2020; Seoul, Korea (South). [doi: 10.1109\/BIBM49941.2020.9313258]","DOI":"10.1109\/BIBM49941.2020.9313258"},{"key":"R13","doi-asserted-by":"crossref","unstructured":"Ghali MK Farrag A Sakai H . GAMedX: generative AI-based medical entity data extractor using large language models. arXiv. Preprint posted online on May 31, 2024. [doi: 10.48550\/arXiv.2405.20585]","DOI":"10.2139\/ssrn.5063216"},{"issue":"5","key":"R14","doi-asserted-by":"publisher","first-page":"858","DOI":"10.1136\/amiajnl-2013-002190","article-title":"MedXN: an open source medication extraction and normalization tool for clinical text","volume":"21","author":"Sohn","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"R15","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","journal-title":"J Am Med Inform Assoc"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.3233\/SHTI250793","article-title":"LLM-Integrated Normalization and Knowledge for FHIR (LINK-FHIR)","volume":"329","author":"Hou","journal-title":"Stud Health Technol Inform"},{"key":"R17","doi-asserted-by":"crossref","unstructured":"Frei J Feldhus N Raithel L Roller R Meyer A Kramer F . Infherno: end-to-end agent-based FHIR resource synthesis from free-form clinical notes. Presented at: Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics; Mar 24-29, 2026; Rabat, Marocco. [doi: 10.18653\/v1\/2026.eacl-demo.13]","DOI":"10.18653\/v1\/2026.eacl-demo.13"},{"key":"R18","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2025.110665","article-title":"From BERT to generative AI\u2014comparing encoder-only vs. large language models in a cohort of lung cancer patients for named entity recognition in unstructured medical reports","volume":"195","author":"Arzideh","journal-title":"Comput Biol Med"},{"key":"R19","volume":"2024","author":"Majid","journal-title":"AMIA Annu Symp Proc"},{"issue":"1","key":"R20","doi-asserted-by":"publisher","DOI":"10.1186\/s13040-025-00463-8","article-title":"Can open source large language models be used for tumor documentation in Germany?\u2014An evaluation on urological doctors\u2019 notes","volume":"18","author":"Lenz","journal-title":"BioData Min"},{"key":"R21","unstructured":"Spiegel S Yimam SM Breitfeld P \u00dcckert F . Adaption and evaluation of generative large language models for German medical information extraction. Presented at: Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Long and Short Papers; Sep 9-12, 2025; Hannover, Germany. URL: https:\/\/aclanthology.org\/2025.konvens-1.4.pdf [Accessed 12-06-2026]"},{"issue":"17","key":"R22","doi-asserted-by":"publisher","first-page":"3509","DOI":"10.3390\/electronics13173509","article-title":"A systematic review of synthetic data generation techniques using generative AI","volume":"13","author":"Goyal","journal-title":"Electronics"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2025.3589503","article-title":"Synthetic data generation using large language models: advances in text and code","volume":"13","author":"Nad\u01ce\u015f","journal-title":"IEEE Access"},{"key":"R24","unstructured":"Baumel T Manoel A Jones D et al. Controllable synthetic clinical note generation with privacy guarantees. arXiv. Preprint posted online on Sep 12, 2024. [doi: 10.48550\/arXiv.2409.07809]"},{"key":"R25","unstructured":"Yang A Li A Yang B et al. Qwen3 technical report. arXiv. Preprint posted online on May 14, 2025. [doi: 10.48550\/arXiv.2505.09388]"},{"key":"R26","unstructured":"unslothai\/unsloth. GitHub. URL: https:\/\/github.com\/unslothai\/unsloth [Accessed 16-11-2025]"},{"key":"R27","doi-asserted-by":"crossref","unstructured":"Ina T Idomura Y Imamura T Onodera N . A new data conversion method for mixed precision Krylov solvers with FP16\/BF16 Jacobi preconditioners. Presented at: Proceedings of the International Conference on High Performance Computing in Asia-Pacific Region; Feb 27 to Mar 2, 2023; Singapore, Singapore. [doi: 10.1145\/3578178.3578222]","DOI":"10.1145\/3578178.3578222"},{"issue":"142","key":"R28","doi-asserted-by":"publisher","first-page":"547","DOI":"10.5169\/seals-266450","article-title":"\u00c9tude de la distribution florale dans une portion des Alpes et du Jura [Article in French]","volume":"37","author":"Jaccard","journal-title":"Bull Soc Vaudoise Sci Nat"},{"key":"R29","unstructured":"Lewis P Perez E Piktus A et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. Presented at: Proceedings of the 34th International Conference on Neural Information Processing Systems; Dec 6-12, 2020; Vancouver, BC, Canada. URL: https:\/\/dl.acm.org\/doi\/abs\/10.5555\/3495724.3496517 [Accessed 12-06-2026]"},{"issue":"2","key":"R30","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1109\/TBDATA.2025.3618474","article-title":"The Faiss Library","volume":"12","author":"Douze","journal-title":"IEEE Trans Big Data"},{"key":"R31","unstructured":"ICD\u2011Code\u2011Suche [Article in German]. gesund.bund.de. URL: https:\/\/gesund.bund.de\/icd-code-suche [Accessed 16-11-2025]"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","author":"Cohen","journal-title":"Educ Psychol Meas"},{"issue":"1","key":"R33","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","journal-title":"Biometrics"},{"issue":"5","key":"R34","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooaf109","article-title":"Leveraging open-source large language models for clinical information extraction in resource-constrained settings","volume":"8","author":"Builtjes","journal-title":"JAMIA Open"},{"key":"R35","doi-asserted-by":"publisher","DOI":"10.1016\/j.cosrev.2026.100999","article-title":"The rise of small language models in healthcare: a comprehensive survey","volume":"62","author":"Garg","journal-title":"Comput Sci Rev"},{"key":"R36","doi-asserted-by":"crossref","unstructured":"Wang Z Wu J Teitge B Holodinsky J Drew S . Small language models for emergency departments decision support: a benchmark study. Presented at: 2025 IEEE Smart World Congress (SWC); Aug 18-22, 2025; Calgary, AB, Canada. [doi: 10.1109\/SWC65939.2025.00239]","DOI":"10.1109\/SWC65939.2025.00239"},{"issue":"1","key":"R37","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-023-00370-1","article-title":"The future landscape of large language models in medicine","volume":"3","author":"Clusmann","journal-title":"Commun Med (Lond)"},{"issue":"3","key":"R38","doi-asserted-by":"publisher","first-page":"553","DOI":"10.1093\/jamia\/ocaf213","article-title":"Information extraction from clinical notes: are we ready to switch to large language models?","volume":"33","author":"Hu","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"R39","doi-asserted-by":"publisher","first-page":"e35","DOI":"10.1016\/S2589-7500(24)00246-2","article-title":"The potential of Generative Pre-trained Transformer 4 (GPT-4) to analyse medical notes in three different languages: a retrospective model-evaluation study","volume":"7","author":"Menezes","journal-title":"Lancet Digit Health"},{"issue":"12","key":"R40","doi-asserted-by":"publisher","first-page":"1418","DOI":"10.1038\/s42256-024-00944-1","article-title":"LLM-based agentic systems in medicine and healthcare","volume":"6","author":"Qiu","journal-title":"Nat Mach Intell"},{"issue":"1","key":"R41","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01476-7","article-title":"Implementing large language models in healthcare while balancing control, collaboration, costs and security","volume":"8","author":"Dennst\u00e4dt","journal-title":"NPJ Digit Med"},{"key":"R42","unstructured":"AI privacy risks & mitigations large language models (LLMs). European Data Protection Board (EDPB); 2025. URL: https:\/\/www.edpb.europa.eu\/system\/files\/2025-04\/ai-privacy-risks-and-mitigations-in-llms.pdf [Accessed 12-12-2025]"},{"issue":"6","key":"R43","doi-asserted-by":"publisher","first-page":"3379","DOI":"10.3390\/app15063379","article-title":"Evaluating the effectiveness of large language models in converting clinical data to FHIR format","volume":"15","author":"Delaunay","journal-title":"Appl Sci"},{"key":"R44","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2025.3573030","article-title":"The effectiveness of large language models in transforming unstructured text to standardized formats","volume":"13","author":"Brach","journal-title":"IEEE Access"},{"issue":"2","key":"R45","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1177\/18333583221106509","article-title":"International Classification of Diseases clinical coding training: an international survey","volume":"53","author":"Otero Varela","journal-title":"Health Inf Manag"},{"key":"R46","unstructured":"Kodierleitfaden H\u00e4matologie, Onkologie und Stammzelltransplantation, Version 2020 [Report in German]. Deutsche Gesellschaft f\u00fcr H\u00e4matologie und Medizinische Onkologie e.V. (DGHO e.V.); 2020. URL: https:\/\/www.dgho.de\/arbeitskreise\/a-g\/drg-gesundheitsoekonomie\/kodierleitfaden\/dgho-kodierleitfaden_2020_buch_ak2_final.pdf [Accessed 12-06-2026]"},{"key":"R47","doi-asserted-by":"publisher","DOI":"10.2196\/72153","article-title":"Clinical large language model evaluation by expert review (CLEVER): framework development and validation","volume":"4","author":"Kocaman","journal-title":"JMIR AI"},{"key":"R48","doi-asserted-by":"crossref","unstructured":"Laskar MTR Jahan I Dolatabadi E Peng C Hoque E Huang JX . Improving automatic evaluation of large language models (LLMs) in biomedical relation extraction via LLMs-as-the-judge. Presented at: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Jul 27 to Aug 1, 2025; Vienna, Austria. [doi: 10.18653\/v1\/2025.acl-long.1238]","DOI":"10.18653\/v1\/2025.acl-long.1238"},{"key":"R49","unstructured":"UMEssen\/PIGEON. GitHub. URL: https:\/\/github.com\/UMEssen\/PIGEON [Accessed 17-06-2026]"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2026\/1\/e92413","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:23:56Z","timestamp":1785536636000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e92413"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,2]]},"references-count":49,"URL":"https:\/\/doi.org\/10.2196\/92413","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,2]]},"article-number":"v28i1e92413"}}