{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T16:51:00Z","timestamp":1774716660772,"version":"3.50.1"},"reference-count":57,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T00:00:00Z","timestamp":1736812800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Generating physician letters is a time-consuming task in daily clinical practice.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>This study investigates local fine-tuning of large language models (LLMs), specifically LLaMA models, for physician letter generation in a privacy-preserving manner within the field of radiation oncology.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Our findings demonstrate that base LLaMA models, without fine-tuning, are inadequate for effectively generating physician letters. The QLoRA algorithm provides an efficient method for local intra-institutional fine-tuning of LLMs with limited computational resources (i.e., a single 48 GB GPU workstation within the hospital). The fine-tuned LLM successfully learns radiation oncology-specific information and generates physician letters in an institution-specific style. ROUGE scores of the generated summary reports highlight the superiority of the 8B LLaMA-3 model over the 13B LLaMA-2 model. Further multidimensional physician evaluations of 10 cases reveal that, although the fine-tuned LLaMA-3 model has limited capacity to generate content beyond the provided input data, it successfully generates salutations, diagnoses and treatment histories, recommendations for further treatment, and planned schedules. Overall, clinical benefit was rated highly by the clinical experts (average score of 3.4 on a 4-point scale).<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>With careful physician review and correction, automated LLM-based physician letter generation has significant practical value.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2024.1493716","type":"journal-article","created":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T06:13:19Z","timestamp":1736835199000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Fine-tuning a local LLaMA-3 large language model for automated privacy-preserving physician letter generation in radiation oncology"],"prefix":"10.3389","volume":"7","author":[{"given":"Yihao","family":"Hou","sequence":"first","affiliation":[]},{"given":"Christoph","family":"Bert","sequence":"additional","affiliation":[]},{"given":"Ahmed","family":"Gomaa","sequence":"additional","affiliation":[]},{"given":"Godehard","family":"Lahmer","sequence":"additional","affiliation":[]},{"given":"Daniel","family":"H\u00f6fler","sequence":"additional","affiliation":[]},{"given":"Thomas","family":"Weissmann","sequence":"additional","affiliation":[]},{"given":"Raphaela","family":"Voigt","sequence":"additional","affiliation":[]},{"given":"Philipp","family":"Schubert","sequence":"additional","affiliation":[]},{"given":"Charlotte","family":"Schmitter","sequence":"additional","affiliation":[]},{"given":"Alina","family":"Depardon","sequence":"additional","affiliation":[]},{"given":"Sabine","family":"Semrau","sequence":"additional","affiliation":[]},{"given":"Andreas","family":"Maier","sequence":"additional","affiliation":[]},{"given":"Rainer","family":"Fietkau","sequence":"additional","affiliation":[]},{"given":"Yixing","family":"Huang","sequence":"additional","affiliation":[]},{"given":"Florian","family":"Putz","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2025,1,14]]},"reference":[{"key":"B1","first-page":"7319","article-title":"\u201cIntrinsic dimensionality explains the effectiveness of language model fine-tuning,\u201d","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing","author":"Aghajanyan","year":"2021"},{"key":"B2","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1016\/j.annemergmed.2024.01.039","article-title":"Harnessing the power of generative AI for clinical summaries: perspectives from emergency physicians","volume":"84","author":"Barak-Corren","year":"2024","journal-title":"Ann. Emerg. Med"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ejmp.2021.02.011","article-title":"The EU medical device regulation: Implications for artificial intelligence-based medical device software in medical physics","volume":"83","author":"Beckers","year":"2021","journal-title":"Phys. Med"},{"key":"B4","doi-asserted-by":"publisher","first-page":"262","DOI":"10.1016\/j.ijrobp.2023.03.055","article-title":"An end-to-end natural language processing system for automatically extracting radiation therapy events from clinical texts","volume":"117","author":"Bitterman","year":"2023","journal-title":"Int. J. Radiat. Oncol. Biol. Phys"},{"key":"B5","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B6","doi-asserted-by":"publisher","first-page":"209","DOI":"10.3857\/roj.2023.00633","article-title":"Developing prompts from large language model for extracting clinical information from pathology and ultrasound reports in breast cancer","volume":"41","author":"Choi","year":"2023","journal-title":"Radiat. Oncol. J"},{"key":"B7","first-page":"4302","article-title":"Deep reinforcement learning from human preferences","volume":"30","author":"Christiano","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B8","doi-asserted-by":"publisher","first-page":"101400","DOI":"10.1016\/j.adro.2023.101400","article-title":"Exploring capabilities of large language models such as chatgpt in radiation oncology","volume":"9","author":"Dennst\u00e4dt","year":"2024","journal-title":"Adv. Radiat. Oncol"},{"key":"B9","first-page":"10088","article-title":"QLoRA: Efficient finetuning of quantized LLMs","volume":"36","author":"Dettmers","year":"2024","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2407.21783","article-title":"The Llama 3 herd of models","author":"Dubey","year":"2024","journal-title":"arXiv"},{"key":"B11","doi-asserted-by":"publisher","first-page":"977","DOI":"10.1016\/j.ijrobp.2023.03.075","article-title":"Chatgpt: Can a natural language processing tool be trusted for radiation oncology use?","volume":"116","author":"Ebrahimi","year":"2023","journal-title":"Int. J. Radiat. Oncol. Biol. Phys"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00066-024-02262-2","article-title":"Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives","volume":"2024","author":"Erdur","year":"2024","journal-title":"Strahlentherapie und Onkologie"},{"key":"B13","doi-asserted-by":"publisher","first-page":"22021","DOI":"10.1609\/aaai.v38i20.30205","article-title":"Medalign: a clinician-generated dataset for instruction following with electronic medical records","volume":"38","author":"Fleming","year":"2024","journal-title":"Proc. AAAI Conf. Artif. Intellig"},{"key":"B14","first-page":"82","article-title":"\u201cLLMS accelerate annotation for medical information extraction,\u201d","volume-title":"Machine Learning for Health (ML4H)","author":"Goel","year":"2023"},{"key":"B15","doi-asserted-by":"publisher","first-page":"109894","DOI":"10.1016\/j.radonc.2023.109894","article-title":"Potential of chatgpt in facilitating research in radiation oncology?","volume":"188","author":"Guckenberger","year":"2023","journal-title":"Radiother. Oncol"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.bionlp-1.58","article-title":"QUB-Cirdan at \"discharge me!\": Zero shot discharge letter generation by open-source LLM","author":"Guo","year":"2024","journal-title":"arXiv"},{"key":"B17","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1109\/ICCCBDA56900.2023.10154864","article-title":"\u201cFederated learning differential privacy preservation method based on differentiated noise addition,\u201d","volume-title":"2023 8th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA)","author":"Han","year":"2023"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1219326","DOI":"10.3389\/fonc.2023.1219326","article-title":"Evaluating large language models on a highly-specialized topic, radiation oncology physics","volume":"13","author":"Holmes","year":"2023","journal-title":"Front. Oncol"},{"key":"B19","first-page":"2790","article-title":"\u201cParameter-efficient transfer learning for NLP,\u201d","volume-title":"International Conference on Machine Learning","author":"Houlsby","year":"2019"},{"key":"B20","first-page":"1","article-title":"\u201cLoRA: Low-rank adaptation of large language models,\u201d","volume-title":"International Conference on Learning Representations","author":"Hu","year":"2022"},{"key":"B21","doi-asserted-by":"publisher","first-page":"5773","DOI":"10.1002\/mp.15863","article-title":"Deep learning for brain metastasis detection and segmentation in longitudinal mri data","volume":"49","author":"Huang","year":"2022","journal-title":"Med. Phys"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00066-024-02272-0","article-title":"Principles of artificial intelligence in radiooncology","volume":"2024","author":"Huang","year":"","journal-title":"Strahlentherapie und Onkologie"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1265024","DOI":"10.3389\/fonc.2023.1265024","article-title":"Benchmarking chatgpt-4 on a radiation oncology in-training exam and red journal gray zone cases: potentials and challenges for ai-assisted medical education and decision making in radiation oncology","volume":"13","author":"Huang","year":"2023","journal-title":"Front. Oncol"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.radonc.2024.110419","article-title":"Multicenter privacy-preserving model training for deep learning brain metastases autosegmentation","volume":"198","author":"Huang","year":"","journal-title":"Radiother. Oncol"},{"key":"B25","doi-asserted-by":"publisher","first-page":"116674","DOI":"10.1109\/ACCESS.2023.3325396","article-title":"FL-ODP: An optimized differential privacy enabled privacy preserving federated learning","volume":"11","author":"Iqbal","year":"2023","journal-title":"IEEE Access"},{"key":"B26","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1109\/ICTC61510.2024.10602253","article-title":"\u201cGemini-the most powerful llm: Myth or truth,\u201d","volume-title":"2024 5th Information Communication Technologies Conference (ICTC)","author":"Islam","year":"2024"},{"key":"B27","doi-asserted-by":"publisher","first-page":"497","DOI":"10.1007\/s00066-023-02182-7","article-title":"Survey in radiation oncology departments in germany, austria, and switzerland: state of digitalization by 2023","volume":"200","author":"Janssen","year":"2024","journal-title":"Strahlentherapie und Onkologie"},{"key":"B28","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1038\/s41514-024-00148-2","article-title":"Healthcare on the brink: navigating the challenges of an aging society in the united states","volume":"10","author":"Jones","year":"2024","journal-title":"NPJ Aging"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s12559-024-10285-1","article-title":"Chatgpt needs spade (sustainability, privacy, digital divide, and ethics) evaluation: a review","volume":"2024","author":"Khowaja","year":"2024","journal-title":"Cognit. Comp"},{"key":"B30","doi-asserted-by":"publisher","first-page":"3411","DOI":"10.1109\/TPAMI.2022.3178101","article-title":"Low dimensional trajectory hypothesis is true: DNNS can be trained in tiny subspaces","volume":"45","author":"Li","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B31","first-page":"4582","article-title":"\u201cPrefix-tuning: Optimizing continuous prompts for generation,\u201d","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Li","year":"2021"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.7759\/cureus.40895","article-title":"Chatdoctor: A medical chat model fine-tuned on a large language model Meta-AI (LLaMA) using medical domain knowledge","volume":"15","author":"Li","year":"2023","journal-title":"Cureus"},{"key":"B33","first-page":"74","article-title":"\u201cRouge: a package for automatic evaluation of summaries,\u201d","volume-title":"Text Summarization Branches Out","author":"Lin","year":"2004"},{"key":"B34","doi-asserted-by":"publisher","first-page":"65","DOI":"10.2147\/PGPM.S396971","article-title":"Natural language processing for radiation oncology: Personalizing treatment pathways","volume":"17","author":"Lin","year":"2024","journal-title":"Pharmacogen. Personal. Med"},{"key":"B35","doi-asserted-by":"crossref","first-page":"441","DOI":"10.18653\/v1\/2020.findings-emnlp.41","article-title":"\u201cExploring versatile generative language model via parameter-efficient transfer learning,\u201d","volume-title":"Findings of the Association for Computational Linguistics: EMNLP","author":"Lin","year":"2020"},{"key":"B36","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1016\/j.aiopen.2023.08.012","article-title":"GPT understands, too","volume":"5","author":"Liu","year":"2024","journal-title":"AI Open."},{"key":"B37","volume-title":"The Complexities of Physician Supply and Demand: Projections from 2015 to 2030","author":"Markit","year":"2017"},{"key":"B38","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1007\/s00066-023-02190-7","article-title":"The role of artificial intelligence in informed patient consent for radiotherapy treatments\u2013a case report","volume":"200","author":"Moll","year":"2024","journal-title":"Strahlentherapie und Onkologie"},{"key":"B39","doi-asserted-by":"publisher","first-page":"1953","DOI":"10.1093\/jamia\/ocae073","article-title":"Distilling large language models for matching patients to clinical trials","volume":"31","author":"Nievas","year":"2024","journal-title":"J. Am. Med. Inform. Assoc"},{"key":"B40","volume-title":"The 2020 Long-Term Budget Outlook","author":"Office","year":"2020"},{"key":"B41","unstructured":"New Ways to Manage Your Data in ChatGPT\n          \n          2024"},{"key":"B42","doi-asserted-by":"publisher","first-page":"900","DOI":"10.1016\/j.ijrobp.2023.11.062","article-title":"Exploring the capabilities and limitations of large language models for radiation oncology decision support","volume":"118","author":"Putz","year":"2024","journal-title":"Int. J. Radiat. Oncol. Biol. Phys"},{"key":"B43","article-title":"Learning multiple visual domains with residual adapters","author":"Rebuffi","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.37829\/HF-2021-RC18","article-title":"Health and social care funding projections 2021","author":"Rocks","year":"2021","journal-title":"Health Found."},{"key":"B45","first-page":"137","article-title":"\u201cExperiments in automated generation of discharge summaries in Italian,\u201d","author":"Ruinelli","year":"2024","journal-title":"Proceedings of the First Workshop on Patient-Oriented Language Processing (CL4Health)"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4835935","author":"Schoonbeek","year":"2024","journal-title":"Completeness, Correctness and Conciseness of Physician-Written Versus Large Language Model Generated Patient Summaries Integrated in Electronic Health Records."},{"key":"B47","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"B48","doi-asserted-by":"publisher","first-page":"120","DOI":"10.3390\/healthcare8020120","article-title":"Integrated natural language processing and machine learning models for standardizing radiotherapy structure names","volume":"8","author":"Syed","year":"2020","journal-title":"Healthcare"},{"key":"B49","doi-asserted-by":"publisher","first-page":"2647","DOI":"10.1007\/s10439-023-03284-0","article-title":"ChatGPT, bard, and large language models for biomedical research: opportunities and pitfalls","volume":"51","author":"Thapa","year":"2023","journal-title":"Ann. Biomed. Eng"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.09288","article-title":"Llama 2: Open foundation and fine-tuned chat models","author":"Touvron","year":"2023","journal-title":"arXiv"},{"key":"B51","doi-asserted-by":"publisher","first-page":"e57721","DOI":"10.2196\/57721","article-title":"Comparison of the quality of discharge letters written by large language models and junior clinicians: Single-blinded study","volume":"26","author":"Tung","year":"2024","journal-title":"J. Med. Internet Res"},{"key":"B52","first-page":"30","article-title":"\u201cAttention is all you need,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"B53","doi-asserted-by":"publisher","first-page":"1","DOI":"10.48550\/arXiv.2304.06975","article-title":"Huatuo: Tuning llama model with chinese medical knowledge","volume":"6975","author":"Wang","year":"2023","journal-title":"arXiv"},{"key":"B54","doi-asserted-by":"publisher","first-page":"1115258","DOI":"10.3389\/fonc.2023.1115258","article-title":"Deep learning for automatic head and neck lymph node level delineation provides expert-level accuracy","volume":"13","author":"Weissmann","year":"2023","journal-title":"Front. Oncol"},{"key":"B55","doi-asserted-by":"publisher","first-page":"e244630","DOI":"10.1001\/jamanetworkopen.2024.4630","article-title":"Quality of large language model responses to radiation oncology patient care questions","volume":"7","author":"Yalamanchili","year":"2024","journal-title":"JAMA Netw. Open"},{"key":"B56","doi-asserted-by":"publisher","first-page":"104096","DOI":"10.1016\/j.jbi.2022.104096","article-title":"Data protection-compliant broad consent for secondary use of health care data and human biosamples for (bio)medical research: Towards a new german national standard","volume":"131","author":"Zenker","year":"2022","journal-title":"J. Biomed. Inform"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1909.08593","article-title":"Fine-tuning language models from human preferences","author":"Ziegler","year":"2019","journal-title":"arXiv"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1493716\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T06:13:28Z","timestamp":1736835208000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1493716\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,14]]},"references-count":57,"alternative-id":["10.3389\/frai.2024.1493716"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1493716","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,14]]},"article-number":"1493716"}}