{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T14:27:03Z","timestamp":1787063223959,"version":"3.56.0"},"reference-count":22,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,9,30]],"date-time":"2025-09-30T00:00:00Z","timestamp":1759190400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large language models (LLMs) offer promise for enhancing clinical care by automating documentation, supporting decision-making, and improving communication. However, their integration into real-world healthcare workflows remains limited and under characterized. This systematic review aims to evaluate the literature on real-world implementation of LLMs in clinical workflows, including their use cases, clinical settings, observed outcomes, and challenges.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We searched MEDLINE, Scopus, Web of Science, and Google Scholar for studies published between January 2015 and April 2025 that assessed LLMs in real-world clinical applications. Inclusion criteria were peer-reviewed, full-text studies in English reporting empirical implementation of LLMs in clinical settings. Study quality and risk of bias were assessed using the PROBAST tool.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Four studies published between 2024 and 2025 met inclusion criteria. All used generative pre-trained transformers (GPTs). Reported applications included outpatient communication, mental health support, inbox message drafting, and clinical data extraction. LLM deployment was associated with improvements in operational efficiency, user satisfaction, and reduced workload. However, challenges included performance variability across data types, limitations in generalizability, regulatory delays, and lack of post-deployment monitoring.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Early evidence suggests that LLMs can enhance clinical workflows, but real-world adoption remains constrained by systemic, technical, and regulatory barriers. To support safe and scalable use, future efforts should prioritize standardized evaluation metrics, multi-site validation, human oversight, and implementation frameworks tailored to clinical settings.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Systematic Review Registration<\/jats:title>\n                    <jats:p>\n                      <jats:ext-link>https:\/\/www.crd.york.ac.uk\/PROSPERO\/recorddashboard<\/jats:ext-link>\n                      , PROSPERO CRD420251030069.\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fdgth.2025.1659134","type":"journal-article","created":{"date-parts":[[2025,9,30]],"date-time":"2025-09-30T12:37:52Z","timestamp":1759235872000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":60,"title":["Large language models in real-world clinical workflows: a systematic review of applications and implementation"],"prefix":"10.3389","volume":"7","author":[{"given":"Yaara","family":"Artsi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vera","family":"Sorin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin S.","family":"Glicksberg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Panagiotis","family":"Korfiatis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Girish N.","family":"Nadkarni","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eyal","family":"Klang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,9,30]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"125","DOI":"10.3390\/healthcare12020125","article-title":"Transformative potential of AI in healthcare: definitions, applications, and navigating the ethical landscape and public perspectives","volume":"12","author":"Bekbolatova","year":"2024","journal-title":"Healthcare"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e70312","DOI":"10.1002\/hsr2.70312","article-title":"The impact of artificial intelligence on healthcare: a comprehensive review of advancements in diagnostics, treatment, and operational efficiency","volume":"8","author":"Faiyazuddin","year":"2025","journal-title":"Health Sci Rep"},{"key":"B3","doi-asserted-by":"publisher","first-page":"AI-S2300233","DOI":"10.1056\/AI-S2300233","article-title":"Multimodal generative AI for precision health","volume":"1","author":"Poon","year":"2023","journal-title":"NEJM AI"},{"key":"B4","doi-asserted-by":"publisher","first-page":"AIp2300031","DOI":"10.1056\/AIp2300031","article-title":"Making the most of AI in health care: strategic principles for implementation","volume":"1","author":"Beam","year":"2023","journal-title":"NEJM AI"},{"key":"B5","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1038\/s41746-025-01542-0","article-title":"Red teaming ChatGPT in medicine to yield real-world insights on model behavior","volume":"8","author":"Wang","year":"2025","journal-title":"NPJ Digit Med"},{"key":"B6","doi-asserted-by":"publisher","first-page":"109713","DOI":"10.1016\/j.isci.2024.109713","article-title":"The application of large language models in medicine: a scoping review","volume":"27","author":"Meng","year":"2024","journal-title":"iScience"},{"key":"B7","doi-asserted-by":"publisher","first-page":"20552076241311144","DOI":"10.1177\/20552076241311144","article-title":"Overcoming barriers and enabling artificial intelligence adoption in allied health clinical practice: a qualitative study","volume":"11","author":"Hoffman","year":"2025","journal-title":"Digit Health"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1873","DOI":"10.1038\/s41591-025-03626-6","article-title":"Sociodemographic biases in medical decision making by large language models","volume":"31","author":"Omar","year":"2025","journal-title":"Nat Med"},{"key":"B9","doi-asserted-by":"publisher","first-page":"366","DOI":"10.1186\/s12911-024-02709-7","article-title":"Analyzing evaluation methods for large language models in the medical field: a scoping review","volume":"24","author":"Lee","year":"2024","journal-title":"BMC Med Inform Decis Mak"},{"key":"B10","doi-asserted-by":"publisher","first-page":"100287","DOI":"10.1016\/j.ibmed.2025.100287","article-title":"Large language models in radiology reporting\u2014a systematic review of performance, limitations, and clinical implications","volume":"12","author":"Artsi","year":"2025","journal-title":"Intell Based Med"},{"key":"B11","article-title":"Transparent reporting of systematic reviews and meta-analyses [Internet]","year":""},{"key":"B12","doi-asserted-by":"publisher","first-page":"712","DOI":"10.1093\/ehjdh\/ztae066","article-title":"Enhancing the interoperability and transparency of real-world data extraction in clinical research: evaluating the feasibility and impact of a ChatGLM implementation in Chinese hospital settings","volume":"5","author":"Wang","year":"2024","journal-title":"Eur Heart J Digit Health"},{"key":"B13","doi-asserted-by":"publisher","first-page":"2878","DOI":"10.1038\/s41591-024-03148-7","article-title":"Outpatient reception via collaboration between nurses and a large language model: a randomized controlled trial","volume":"30","author":"Wan","year":"2024","journal-title":"Nat Med"},{"key":"B14","doi-asserted-by":"publisher","first-page":"e60435","DOI":"10.2196\/60435","article-title":"Generative AI-enabled therapy support tool for improved clinical outcomes and patient engagement in group therapy: real-world observational study","volume":"27","author":"Habicht","year":"2025","journal-title":"J Med Internet Res"},{"key":"B15","doi-asserted-by":"publisher","first-page":"e243201","DOI":"10.1001\/jamanetworkopen.2024.3201","article-title":"Artificial intelligence-generated draft replies to patient inbox messages","volume":"7","author":"Garcia","year":"2024","journal-title":"JAMA Netw Open"},{"key":"B16","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1007\/s42979-021-00592-x","article-title":"Machine learning: algorithms, real-world applications and research directions","volume":"2","author":"Sarker","year":"2021","journal-title":"SN Comput Sci"},{"key":"B17","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1186\/s12911-024-02757-z","article-title":"Qualitative metrics from the biomedical literature for evaluating large language models in clinical decision-making: a narrative review","volume":"24","author":"Ho","year":"2024","journal-title":"BMC Med Inform Decis Mak"},{"key":"B18","doi-asserted-by":"publisher","first-page":"e2400223","DOI":"10.1056\/AIp2400223","article-title":"A call for artificial intelligence implementation science centers to evaluate clinical effectiveness","volume":"1","author":"Longhurst","year":"2024","journal-title":"NEJM AI"},{"key":"B19","doi-asserted-by":"publisher","first-page":"787","DOI":"10.1001\/jama.2024.12630","article-title":"The compelling need for shared responsibility of AI oversight: lessons from health IT certification","volume":"332","author":"Ratwani","year":"2024","journal-title":"J Am Med Assoc"},{"key":"B20","doi-asserted-by":"publisher","first-page":"100184","DOI":"10.1016\/j.mcpdig.2024.11.005","article-title":"Fine-tuning large language models for specialized use cases","volume":"3","author":"Anisuzzaman","year":"2024","journal-title":"Mayo Clin Proc Digit Health"},{"key":"B21","doi-asserted-by":"publisher","first-page":"652","DOI":"10.3233\/SHTI240498","article-title":"Pubmed retrieval with RAG techniques","volume":"316","author":"Thomo","year":"2024","journal-title":"Stud Health Technol Inform"},{"key":"B22","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s11604-023-01474-3","article-title":"Fairness of artificial intelligence in healthcare: review and recommendations","volume":"42","author":"Ueda","year":"2024","journal-title":"Jpn J Radiol"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2025.1659134\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,30]],"date-time":"2025-09-30T12:37:53Z","timestamp":1759235873000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2025.1659134\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,30]]},"references-count":22,"alternative-id":["10.3389\/fdgth.2025.1659134"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2025.1659134","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,30]]},"article-number":"1659134"}}