{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T08:19:28Z","timestamp":1783585168880,"version":"3.55.0"},"reference-count":30,"publisher":"American Association for the Advancement of Science (AAAS)","issue":"2","funder":[{"DOI":"10.13039\/501100020083","name":"Forschungsrat des Kantonsspitals Aarau","doi-asserted-by":"publisher","award":["FR1410.000.214"],"award-info":[{"award-number":["FR1410.000.214"]}],"id":[{"id":"10.13039\/501100020083","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Hightech Zentrum Aargau"}],"content-domain":{"domain":["spj.science.org"],"crossmark-restriction":true},"short-container-title":["Comput Struct Biotechnol J"],"published-print":{"date-parts":[[2026,1,1]]},"abstract":"<jats:p>Writing discharge notes (DNs) is a time-intensive task in inpatient care that contributes substantially to physician workload and burnout. Large language models (LLMs) offer potential opportunities to automate clinical documentation. However, concerns regarding accuracy, reliability, data privacy protection laws and data governance, and compliance with healthcare privacy remain. This study evaluated the performance of locally deployed, domain-adapted LLMs for generating German-language DN in internal medicine. In this retrospective study, electronic health record data from a Swiss tertiary care hospital were used. Clinical information from 3 inpatient scenarios served as input for the 3 open source LLMs (Mixtral 8\u00d77B, Mixtral 8\u00d722B, and LLaMA 3.1 70B). Zero-shot prompting, 4-shot in-context learning, and supervised fine-tuning using QLoRA were compared. Generated DNs were evaluated using automated metrics, blinded physician assessment with direct comparison of all relevant clinical information using the modified Physician Documentation Quality Instrument, and additionally an LLM-as-a-judge framework and a Turing Test. Across all evaluation methods, the fine-tuned Mixtral 8\u00d77B model achieved the best overall performance compared with prompting-based approaches and larger nonspecialized models. Simpler cases consistently received higher-quality ratings. Physician inter-rater agreement was low, highlighting the challenges of objective quality assessment for clinical documentation. In a Turing Test, physicians were unable to reliably distinguish artificial intelligence-generated from clinician-authored DNs, performing at chance level, demonstrating that high-quality DN generation is feasible using locally deployed LLMs within secure hospital infrastructures. Domain-specific fine-tuning substantially improves performance and may enable practical deployment. Future work should address further hallucination mitigation, robustness across medical specialties, and real-world integration.<\/jats:p>","DOI":"10.34133\/csbj.0151","type":"journal-article","created":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T22:14:15Z","timestamp":1780956855000},"update-policy":"https:\/\/doi.org\/10.34133\/aaas_crossmark_01","source":"Crossref","is-referenced-by-count":0,"title":["Toward Responsible and Privacy-Preserving Generation of Inpatient Clinical Discharge Notes Using Large Language Models"],"prefix":"10.34133","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4362-162X","authenticated-orcid":true,"given":"Felice","family":"Burn","sequence":"first","affiliation":[{"name":"Center of Excellence for AI and Data Science, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-8408-9917","authenticated-orcid":false,"given":"Chantal","family":"Zwick","sequence":"additional","affiliation":[{"name":"Institute for Data Science, \rFHNW Brugg-Windisch, Windisch, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-1971-0771","authenticated-orcid":false,"given":"Joseph","family":"Weibel","sequence":"additional","affiliation":[{"name":"Institute for Data Science, \rFHNW Brugg-Windisch, Windisch, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-1461-8406","authenticated-orcid":false,"given":"Cyril","family":"Fischer","sequence":"additional","affiliation":[{"name":"Center of Excellence for AI and Data Science, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rick","family":"Veer","sequence":"additional","affiliation":[{"name":"University Department, Division of General Internal and Emergency Medicine, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lennart","family":"Essmann","sequence":"additional","affiliation":[{"name":"University Department, Division of General Internal and Emergency Medicine, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3720-2584","authenticated-orcid":false,"given":"Maurin","family":"Lampart","sequence":"additional","affiliation":[{"name":"University Department, Division of General Internal and Emergency Medicine, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Chamo","sequence":"additional","affiliation":[{"name":"University Department, Division of General Internal and Emergency Medicine, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0553-3357","authenticated-orcid":false,"given":"Carola","family":"George","sequence":"additional","affiliation":[{"name":"University Department, Division of General Internal and Emergency Medicine, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sandra","family":"Gafner","sequence":"additional","affiliation":[{"name":"University Department, Division of General Internal and Emergency Medicine, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-7093-7188","authenticated-orcid":false,"given":"Simone","family":"Wildisen","sequence":"additional","affiliation":[{"name":"University Department, Division of General Internal and Emergency Medicine, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Vanecko","sequence":"additional","affiliation":[{"name":"Notfall Zurich 3, NZ3, Zurich, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kimberly","family":"Ordo\u00f1ez","sequence":"additional","affiliation":[{"name":"Notfall Zurich 3, NZ3, Zurich, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8878-9547","authenticated-orcid":false,"given":"Monica Alexandra","family":"Vieira","sequence":"additional","affiliation":[{"name":"Notfall Zurich 3, NZ3, Zurich, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0208-3220","authenticated-orcid":false,"given":"Christian","family":"Eisenring","sequence":"additional","affiliation":[{"name":"Notfall Zurich 3, NZ3, Zurich, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1986-2511","authenticated-orcid":false,"given":"Beat","family":"Mueller","sequence":"additional","affiliation":[{"name":"University Department, Division of General Internal and Emergency Medicine, \rCantonal Hospital Aarau, Aarau, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2688-4105","authenticated-orcid":false,"given":"Daniel Olivier","family":"Perruchoud","sequence":"additional","affiliation":[{"name":"Institute for Data Science, \rFHNW Brugg-Windisch, Windisch, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0199-0184","authenticated-orcid":false,"given":"Tristan","family":"Struja","sequence":"additional","affiliation":[{"name":"University Department, Division of General Internal and Emergency Medicine, \rCantonal Hospital Aarau, Aarau, Switzerland."},{"name":"Laboratory for Computational Physiology, Institute for Medical Engineering and Science, \rMassachusetts Institute of Technology, Cambridge MA, USA."},{"name":"Hospital Muri,Aargau,Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"221","published-online":{"date-parts":[[2026,7,9]]},"reference":[{"issue":"5","key":"e_1_3_4_2_2","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1370\/afm.2121","article-title":"Tethered to the EHR: Primary care physician workload assessment using EHR event log data and time-motion observations","volume":"15","author":"Arndt BG","year":"2017","unstructured":"Arndt BG, Beasley JW, Watkinson MD, Temte JL, Tuan WJ, Sinsky CA, Gilchrist VJ. Tethered to the EHR: Primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med. 2017;15(5):419\u2013426.","journal-title":"Ann Fam Med"},{"issue":"1","key":"e_1_3_4_3_2","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.ijmedinf.2009.09.002","article-title":"Accuracy of medication documentation in hospital discharge summaries: A retrospective analysis of medication transcription errors in manual and electronic discharge summaries","volume":"79","author":"Callen J","year":"2010","unstructured":"Callen J, McIntosh J, Li J. Accuracy of medication documentation in hospital discharge summaries: A retrospective analysis of medication transcription errors in manual and electronic discharge summaries. Int J Med Inform. 2010;79(1):58\u201364.","journal-title":"Int J Med Inform"},{"issue":"1","key":"e_1_3_4_4_2","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1186\/s12913-019-3989-1","article-title":"A systematic literature review and narrative synthesis on the risks of medical discharge letters for patients\u2019 safety","volume":"19","author":"Schwarz CM","year":"2019","unstructured":"Schwarz CM, Hoffmann M, Schwarz P, Kamolz LP, Brunner G, Sendlhofer G. A systematic literature review and narrative synthesis on the risks of medical discharge letters for patients\u2019 safety. BMC Health Serv Res. 2019;19(1):158.","journal-title":"BMC Health Serv Res"},{"issue":"2","key":"e_1_3_4_5_2","doi-asserted-by":"crossref","first-page":"149","DOI":"10.7861\/fhj.2019-0046","article-title":"Improving the quality of discharge summaries through a direct feedback system","volume":"7","author":"Earnshaw CH","year":"2020","unstructured":"Earnshaw CH, Pedersen A, Evans J, Cross T, Gaillemin O, Vilches-Moraga A. Improving the quality of discharge summaries through a direct feedback system. Future Healthc J. 2020;7(2):149\u2013154.","journal-title":"Future Healthc J"},{"key":"e_1_3_4_6_2","doi-asserted-by":"crossref","first-page":"1514896","DOI":"10.3389\/frai.2024.1514896","article-title":"Evaluating accuracy and reproducibility of large language model performance on critical care assessments in pharmacy education","volume":"7","author":"Yang H","year":"2025","unstructured":"Yang H, Hu M, Most A, Hawkins WA, Murray B, Smith SE, Li S, Sikora A. Evaluating accuracy and reproducibility of large language model performance on critical care assessments in pharmacy education. Front Artif Intell. 2025;7:1514896.","journal-title":"Front Artif Intell"},{"issue":"7","key":"e_1_3_4_7_2","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1001\/jamainternmed.2025.0821","article-title":"Physician- and large language model-generated hospital discharge summaries","volume":"185","author":"Williams CY","year":"2025","unstructured":"Williams CY, Subramanian CR, Ali SS, Apolinario M, Askin E, Barish P, Cheng M, Deardorff WJ, Donthi N, Ganeshan S, et al. Physician- and large language model-generated hospital discharge summaries. JAMA Intern Med. 2025;185(7):818\u2013825.","journal-title":"JAMA Intern Med"},{"key":"e_1_3_4_8_2","doi-asserted-by":"crossref","DOI":"10.2196\/57721","article-title":"Comparison of the quality of discharge letters written by large language models and junior clinicians: Single-blinded study","volume":"26","author":"Tung JYM","year":"2024","unstructured":"Tung JYM, Gill SR, Sng GG, Lim DY, Ke Y, Tan TF, Jin L, Elangovan K, Ong JC, Abdullah HR, et al. Comparison of the quality of discharge letters written by large language models and junior clinicians: Single-blinded study. J Med Internet Res. 2024;26: Article e57721.","journal-title":"J Med Internet Res"},{"issue":"1","key":"e_1_3_4_9_2","doi-asserted-by":"crossref","first-page":"16466","DOI":"10.1038\/s41598-025-01618-7","article-title":"Automated generation of discharge summaries: Leveraging large language models with clinical data","volume":"15","author":"Ganzinger M","year":"2025","unstructured":"Ganzinger M, Kunz N, Fuchs P, Lyu CK, Loos M, Dugas M, Pausch TM. Automated generation of discharge summaries: Leveraging large language models with clinical data. Sci Rep. 2025;15(1):16466.","journal-title":"Sci Rep"},{"key":"e_1_3_4_10_2","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1038\/s41746-024-01336-w","article-title":"The quality and safety of using generative AI to produce patient-centred discharge instructions","volume":"7","author":"Stanceski K","year":"2024","unstructured":"Stanceski K, Zhong S, Zhang X, Khadra S, Tracy M, Koria L, Lo S, Naganathan V, Kim J, Dunn AG, et al. The quality and safety of using generative AI to produce patient-centred discharge instructions. NPJ Digit Med. 2024;7:329.","journal-title":"NPJ Digit Med"},{"key":"e_1_3_4_11_2","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijmedinf.2024.105654","article-title":"Large language models can support generation of standardized discharge summaries\u2014A retrospective study utilizing ChatGPT-4 and electronic health records","volume":"192","author":"Schwieger A","year":"2024","unstructured":"Schwieger A, Angst K, De Bardeci M, Burrer A, Cathomas F, Ferrea S, Gr\u00e4tz F, Knorr M, Kronenberg G, Spiller T, et al. Large language models can support generation of standardized discharge summaries\u2014A retrospective study utilizing ChatGPT-4 and electronic health records. Int J Med Inform. 2024;192: Article 105654.","journal-title":"Int J Med Inform"},{"key":"e_1_3_4_12_2","article-title":"Integration of customised LLM for discharge summary generation in real-world clinical settings: A pilot study on RUSSELL GPT","volume":"51","author":"Chua CE","year":"2024","unstructured":"Chua CE, Clara NL, Furqan MS, Kit JL, Makmur A, Tham YC, Santosa A, Ngiam KY. Integration of customised LLM for discharge summary generation in real-world clinical settings: A pilot study on RUSSELL GPT. Lancet Reg Health West Pac. 2024;51: Article 101211.","journal-title":"Lancet Reg Health West Pac"},{"key":"e_1_3_4_13_2","volume-title":"Guide to protecting the confidentiality of personally identifiable information","author":"McCallister E","year":"2010","unstructured":"McCallister E. Guide to protecting the confidentiality of personally identifiable information. Diane Publishing; 2010."},{"key":"e_1_3_4_14_2","unstructured":"Jiang AQ Sablayrolles A Roux A Mensch A Savary B Bamford C Chaplot DS de las Casas D Bou Hanna E Bressand F et al. Mixtral of experts. arXiv. 2024. https:\/\/doi.org\/10.48550\/arXiv.2401.04088"},{"key":"e_1_3_4_15_2","unstructured":"Grattafiori A Dubey A Jauhri A Pandey A Kadian A Al-Dahle A Letman A Mathur A Schelten A Vaughan A et al. The llama 3 herd of models. arXiv. 2024. https:\/\/doi.org\/10.48550\/arXiv.2407.21783"},{"key":"e_1_3_4_16_2","doi-asserted-by":"crossref","first-page":"10088","DOI":"10.52202\/075280-0441","article-title":"Qlora: Efficient finetuning of quantized llms","volume":"36","author":"Dettmers T","year":"2023","unstructured":"Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L. Qlora: Efficient finetuning of quantized llms. Adv Neural Inf Proces Syst. 2023;36:10088\u201310115.","journal-title":"Adv Neural Inf Proces Syst"},{"key":"e_1_3_4_17_2","unstructured":"Dabre R Fujita A. Softmax tempering for training neural machine translation models. arXiv. 2020. https:\/\/doi.org\/10.48550\/arXiv.2009.09372"},{"key":"e_1_3_4_18_2","doi-asserted-by":"crossref","unstructured":"Papineni K Roukos S Ward T Zhu W-J. Bleu: A method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics. Association for Computational Linguistics; 2022. p. 311\u2013318.","DOI":"10.3115\/1073083.1073135"},{"key":"e_1_3_4_19_2","unstructured":"Lin C-Y. In: Text summarization branches out. p. 74\u201381."},{"key":"e_1_3_4_20_2","unstructured":"Tianyi Z Varsha K Felix W Kilian QW Yoav A. BERTScore: Evaluating text generation with BERT. The Lancet Regional Health\u2013Western Pacific 51 (2020)."},{"issue":"2","key":"e_1_3_4_21_2","doi-asserted-by":"crossref","first-page":"274","DOI":"10.34197\/ats-scholar.2023-0094OC","article-title":"Validating the physician documentation quality instrument for intensive care unit\u2013ward transfer notes","volume":"5","author":"Lyons PG","year":"2024","unstructured":"Lyons PG, Rojas JC, Bewley AF, Malone SM, Santhosh L. Validating the physician documentation quality instrument for intensive care unit\u2013ward transfer notes. ATS Sch. 2024;5(2):274\u2013285.","journal-title":"ATS Sch"},{"key":"e_1_3_4_22_2","doi-asserted-by":"crossref","DOI":"10.1016\/j.mex.2023.102545","article-title":"K-Alpha calculator\u2014Krippendorff\u2019s alpha calculator: A user-friendly tool for computing Krippendorff\u2019s alpha inter-rater reliability coefficient","volume":"12","author":"Marzi G","year":"2024","unstructured":"Marzi G, Balzano M, Marchiori D. K-Alpha calculator\u2014Krippendorff\u2019s alpha calculator: A user-friendly tool for computing Krippendorff\u2019s alpha inter-rater reliability coefficient. MethodsX. 2024;12: Article 102545.","journal-title":"MethodsX"},{"key":"e_1_3_4_23_2","unstructured":"Yang A Yang B Zhang B Hui B Zheng B Yu B Li C Liu D Huang F Wei H et al. Qwen2.5 technical report. arXiv. 2025. https:\/\/doi.org\/10.48550\/arXiv.2412.15115"},{"key":"e_1_3_4_24_2","doi-asserted-by":"crossref","unstructured":"Turing AM. Computing machinery and intelligence. Front Artificial Intelligence. 2007;7:23\u201365.","DOI":"10.1007\/978-1-4020-6710-5_3"},{"key":"e_1_3_4_25_2","doi-asserted-by":"crossref","unstructured":"Raunak V Menezes A Junczys-Dowmunt M. The curious case of hallucinations in neural machine translation. arXiv. 2021. https:\/\/doi.org\/10.48550\/arXiv.2104.06683","DOI":"10.18653\/v1\/2021.naacl-main.92"},{"key":"e_1_3_4_26_2","doi-asserted-by":"crossref","unstructured":"Agrawal G Kumarage T Alghamdi Z Liu H. Can knowledge graphs reduce hallucinations in LLMs?: A survey. arXiv. 2023. https:\/\/doi.org\/10.48550\/arXiv.2311.07914","DOI":"10.18653\/v1\/2024.naacl-long.219"},{"key":"e_1_3_4_27_2","doi-asserted-by":"crossref","unstructured":"Song Z Yan B Liu Y Fang M Li M Yan R Chen X. Injecting domain-specific knowledge into large language models: A comprehensive survey. arXiv. 2025. https:\/\/doi.org\/10.48550\/arXiv.2502.10708","DOI":"10.18653\/v1\/2025.findings-emnlp.1379"},{"key":"e_1_3_4_28_2","unstructured":"Nori H King N McKinney SM Carignan D Horvitz E. Capabilities of gpt-4 on medical challenge problems. arXiv. 2023. https:\/\/doi.org\/10.48550\/arXiv.2303.13375"},{"key":"e_1_3_4_29_2","unstructured":"Lehman E Hernandez E Mahajan D Wulff J Smith MJ Ziegler Z Nadler D Szolovits P Johnson A Alsentzer E. Do we still need clinical language models? arXiv. 2023. https:\/\/doi.org\/10.48550\/arXiv.2302.08091"},{"issue":"1","key":"e_1_3_4_30_2","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1038\/s44401-024-00011-2","article-title":"Current and future state of evaluation of large language models for medical summarization tasks","volume":"2","author":"Croxford E","year":"2025","unstructured":"Croxford E, Gao Y, Pellegrino N, Wong K, Wills G, First E, Liao F, Goswami C, Patterson B, Afshar M. Current and future state of evaluation of large language models for medical summarization tasks. NPJ Health Syst. 2025;2(1):6.","journal-title":"NPJ Health Syst"},{"key":"e_1_3_4_31_2","doi-asserted-by":"crossref","unstructured":"Kruse M Hu S Derby N Wu Y Stonbraker S Yao B Wang D Goldberg E Gao Y. Zero-shot large language models for long clinical text summarization with temporal reasoning. arXiv. 2025. https:\/\/doi.org\/10.48550\/arXiv.2501.18724","DOI":"10.1101\/2025.07.21.25331947"}],"container-title":["Computational and Structural Biotechnology Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/spj.science.org\/doi\/pdf\/10.34133\/csbj.0151","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T08:01:21Z","timestamp":1783584081000},"score":1,"resource":{"primary":{"URL":"https:\/\/spj.science.org\/doi\/10.34133\/csbj.0151"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,1]]},"references-count":30,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,1,1]]}},"alternative-id":["10.34133\/csbj.0151"],"URL":"https:\/\/doi.org\/10.34133\/csbj.0151","relation":{},"ISSN":["2001-0370"],"issn-type":[{"value":"2001-0370","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,1]]},"assertion":[{"value":"2026-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-08","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-07-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"0151"}}