{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T22:58:29Z","timestamp":1781650709896,"version":"3.54.5"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T00:00:00Z","timestamp":1776988800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T00:00:00Z","timestamp":1776988800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Med Syst"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Clinical documentation accounts for a substantial share of clinicians\u2019 working time, contributing to administrative burden and reduced patient-facing care. Artificial intelligence has stimulated the development of digital scribes that combine speech recognition (ASR) and large language models (LLMs) to generate clinical notes from patient\u2013provider conversations with the aim to automate and support this process and reduce this burden. This scoping review explores how digital scribes are currently validated, both technical and clinical, and whether they reliably support clinical workflows. Using the Technology Readiness Level (TRL) framework, we show that most systems remain in early development stages (typically TRL 3&amp;4), with only a small number progressing to workflow integration. While digital scribes show potential to improve documentation efficiency, validation methods are highly heterogeneous, most studies rely on simulated or retrospective data, and real-world testing is limited. Consequently, cross-system comparisons and conclusions about clinical performance remain limited. We identified three motivational frames: human-, performance-, and system-oriented, which shape evaluation practices and outcome expectations. These findings suggest that successful implementation depends not only on scribes\u2019 technical capability but also on alignment with clinical needs and documentation styles. Overall, our review underscores the need for standardised validation frameworks and prospective real-world studies to ensure that digital scribes progress beyond their current low TRL and move from experimental promise to safe, effective, and sustainable integration into clinical care.<\/jats:p>","DOI":"10.1007\/s10916-026-02392-3","type":"journal-article","created":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T02:10:54Z","timestamp":1776996654000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Validating Digital Scribes: A Scoping Review of Evaluation Practices and Clinical Use"],"prefix":"10.1007","volume":"50","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-4331-5285","authenticated-orcid":false,"given":"Ekin","family":"Kerimo\u011flu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4185-849X","authenticated-orcid":false,"given":"Fleur V.","family":"Notermans","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8279-1341","authenticated-orcid":false,"given":"Milou E. W. M.","family":"Silkens","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3447-5920","authenticated-orcid":false,"given":"Marleen","family":"de Mul","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9973-3746","authenticated-orcid":false,"given":"Kees C. T. B.","family":"Ahaus","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6340-2590","authenticated-orcid":false,"given":"Robert M. A.","family":"van der Boon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,4,24]]},"reference":[{"key":"2392_CR1","doi-asserted-by":"publisher","first-page":"e60020","DOI":"10.2196\/60020","volume":"3","author":"MM van Buchem","year":"2024","unstructured":"van Buchem MM, Kant IMJ, King L, Kazmaier J, Steyerberg EW, Bauer MP (2024) Impact of a digital scribe system on clinical documentation time and quality: usability study. JMIR AI 3:e60020. https:\/\/doi.org\/10.2196\/60020","journal-title":"JMIR AI"},{"key":"2392_CR2","doi-asserted-by":"publisher","first-page":"2906","DOI":"10.1177\/1460458220951719","volume":"26","author":"JC Quiroz","year":"2020","unstructured":"Quiroz JC, Laranjo L, Kocaballi AB, Briatore A, Berkovsky S, Rezazadegan D, Coiera E (2020) Identifying relevant information in medical conversations to summarize a clinician\u2013patient encounter. Health Inform J 26:2906\u20132914. https:\/\/doi.org\/10.1177\/1460458220951719","journal-title":"Health Inform J"},{"key":"2392_CR3","doi-asserted-by":"publisher","first-page":"2995","DOI":"10.1007\/s11606-024-08870-z","volume":"39","author":"H Bundy","year":"2024","unstructured":"Bundy H, Gerhart J, Baek S, Connor CD, Isreal M, Dharod A, Stephens C, Liu TL, Hetherington T, Cleveland J (2024) Can the administrative loads of physicians be alleviated by AI-facilitated clinical documentation? J Gen Intern Med 39:2995\u20133000. https:\/\/doi.org\/10.1007\/s11606-024-08870-z","journal-title":"J Gen Intern Med"},{"key":"2392_CR4","doi-asserted-by":"publisher","first-page":"228","DOI":"10.3122\/jabfm.2023.230211R2","volume":"37","author":"ST Florig","year":"2024","unstructured":"Florig ST, Corby S, Devara T, Weiskopf NG, Gold JA, Mohan V (2024) Variable impact of medical scribes on physician electronic health record documentation practices: A quantitative analysis across a large, integrated health-system. J Am Board Fam Med 37:228\u2013241. https:\/\/doi.org\/10.3122\/jabfm.2023.230211R2","journal-title":"J Am Board Fam Med"},{"key":"2392_CR5","doi-asserted-by":"publisher","first-page":"2625","DOI":"10.1007\/s11606-024-08924-2","volume":"39","author":"JL Galloway","year":"2024","unstructured":"Galloway JL, Munroe D, Vohra-Khullar PD, Holland C, Solis MA, Moore MA, Dbouk RH (2024) Impact of an artificial intelligence-based solution on clinicians\u2019 clinical documentation experience: initial findings using ambient listening technology. J Gen Intern Med 39:2625\u20132627. https:\/\/doi.org\/10.1007\/s11606-024-08924-2","journal-title":"J Gen Intern Med"},{"key":"2392_CR6","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1038\/s41746-019-0190-1","volume":"2","author":"JC Quiroz","year":"2019","unstructured":"Quiroz JC, Laranjo L, Kocaballi AB, Berkovsky S, Rezazadegan D, Coiera E (2019) Challenges of developing a digital scribe to reduce clinical documentation burden. npj Digit Med 2:114. https:\/\/doi.org\/10.1038\/s41746-019-0190-1","journal-title":"npj Digit Med"},{"key":"2392_CR7","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","volume":"29","author":"AJ Thirunavukarasu","year":"2023","unstructured":"Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW (2023) Large language models in medicine. Nat Med 29:1930\u20131940. https:\/\/doi.org\/10.1038\/s41591-023-02448-8","journal-title":"Nat Med"},{"key":"2392_CR8","doi-asserted-by":"publisher","unstructured":"Suenghataiphorn T, Tribuddharat N, Danpanichkul P, Kulthamrongsri N (2025) Bias in large language models across clinical applications: a systematic review. arXiv 2504.02917. https:\/\/doi.org\/10.48550\/arXiv.2504.02917","DOI":"10.48550\/arXiv.2504.02917"},{"key":"2392_CR9","doi-asserted-by":"publisher","first-page":"467","DOI":"10.7326\/M18-0850","volume":"169","author":"AC Tricco","year":"2018","unstructured":"Tricco AC, Lillie E, Zarin W, O\u2019Brien KK, Colquhoun H et al. (2018) PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 169:467\u2013473. https:\/\/doi.org\/10.7326\/M18-0850","journal-title":"Ann Intern Med"},{"key":"2392_CR10","doi-asserted-by":"publisher","DOI":"10.1007\/s00134-020-06045-y","author":"LM Fleuren","year":"2020","unstructured":"Fleuren LM, Thoral P, Shillan D, Ercole A, Elbers PWG, Right Data Right Now Collaborators et al (2020) Machine learning in intensive care medicine: ready for take-off? Intensive Care Med 46:1486\u20131488. https:\/\/doi.org\/10.1007\/s00134-020-06045-y"},{"key":"2392_CR11","doi-asserted-by":"publisher","first-page":"ooaf013","DOI":"10.1093\/jamiaopen\/ooaf013","volume":"8","author":"M Albrecht","year":"2025","unstructured":"Albrecht M, Shanks D, Shah T, Hudson T, Thompson J, Filardi T, Wright K, Ator GA, Smith TR (2025) Enhancing clinical documentation with ambient artificial intelligence: a quality improvement survey assessing clinician perspectives on work burden, burnout, and job satisfaction. JAMIA open, 8:ooaf013. https:\/\/doi.org\/10.1093\/jamiaopen\/ooaf013","journal-title":"JAMIA open"},{"key":"2392_CR12","doi-asserted-by":"publisher","unstructured":"Knoll T, Moramarco F, Korfiatis AP, Young R, Ruffini C et al (2022) User-driven research of medical note generation software. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL, Seattle, pp 385\u2013394. https:\/\/doi.org\/10.18653\/v1\/2022.naacl-main.29","DOI":"10.18653\/v1\/2022.naacl-main.29"},{"key":"2392_CR13","doi-asserted-by":"publisher","first-page":"205520762412939","DOI":"10.1177\/20552076241293932","volume":"10","author":"X Chen","year":"2024","unstructured":"Chen X, Zhou W, Hoda R, Li A, Bain C, Poon P (2024) Exploring the opportunities of large language models for summarizing palliative care consultations: a pilot comparative study. Digit Health 10: 20552076241293932. https:\/\/doi.org\/10.1177\/20552076241293932","journal-title":"Digit Health"},{"key":"2392_CR14","doi-asserted-by":"publisher","first-page":"1195","DOI":"10.1038\/s41598-024-84850-x","volume":"15","author":"D Fraile Navarro","year":"2025","unstructured":"Fraile Navarro D, Coiera E, Hambly TW, Triplett Z, Asif N et al (2025) Expert evaluation of large language models for clinical dialogue summarization. Sci Rep 15:1195. https:\/\/doi.org\/10.1038\/s41598-024-84850-x","journal-title":"Sci Rep"},{"key":"2392_CR15","doi-asserted-by":"publisher","first-page":"e54419","DOI":"10.2196\/54419","volume":"26","author":"A Kernberg","year":"2024","unstructured":"Kernberg A, Gold JA, Mohan V (2024) Using ChatGPT-4 to create structured medical notes from audio recordings of physician\u2013patient encounters: comparative study. J Med Internet Res 26:e54419. https:\/\/doi.org\/10.2196\/54419","journal-title":"J Med Internet Res"},{"key":"2392_CR16","doi-asserted-by":"publisher","first-page":"e48235","DOI":"10.7759\/cureus.48235","volume":"15","author":"Y Kaneda","year":"2023","unstructured":"Kaneda Y, Takita M, Hamaki T, Ozaki A, Tanimoto T (2023) ChatGPT\u2019s potential in enhancing physician efficiency: a Japanese case study. Cureus 15:e48235. https:\/\/doi.org\/10.7759\/cureus.48235","journal-title":"Cureus"},{"key":"2392_CR17","doi-asserted-by":"publisher","first-page":"100157","DOI":"10.1016\/j.fhj.2024.100157","volume":"11","author":"J Balloch","year":"2024","unstructured":"Balloch J, Sridharan S, Oldham G, Wray J, Gough P et al (2024) Use of an ambient artificial intelligence tool to improve quality of clinical documentation. Future Healthc J 11:100157. https:\/\/doi.org\/10.1016\/j.fhj.2024.100157","journal-title":"Future Healthc J"},{"key":"2392_CR18","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","volume":"30","author":"D Van Veen","year":"2024","unstructured":"Van Veen D, Van Uden C, Blankemeier L, Delbrouck JB, Aali A et al (2024) Adapted large language models can outperform medical experts in clinical text summarization. Nat Med 30:1134\u20131142. https:\/\/doi.org\/10.1038\/s41591-024-02855-5","journal-title":"Nat Med"},{"key":"2392_CR19","doi-asserted-by":"publisher","unstructured":"Li B, Crampton N, Yeates T, Xia Y, Tian X, Truong K (2021) Automating clinical documentation with digital scribes: understanding the impact on physicians. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1\u201312. https:\/\/doi.org\/10.1145\/3411764.3445172","DOI":"10.1145\/3411764.3445172"},{"key":"2392_CR20","doi-asserted-by":"publisher","unstructured":"Han J, Park J, Huh J, Oh U, Do J, Kim D (2024) AscleAI: a LLM-based clinical note management system for enhancing clinician productivity. In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. ACM, New York, pp 1\u20137. https:\/\/doi.org\/10.1145\/3613905.3650784","DOI":"10.1145\/3613905.3650784"},{"key":"2392_CR21","doi-asserted-by":"publisher","unstructured":"Moramarco F, Korfiatis AP, Perera M, Juric D, Flann J et al (2022) Human evaluation and correlation with automatic metrics in consultation note generation. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. ACL, Dublin, pp 5739\u20135754. https:\/\/doi.org\/10.18653\/v1\/2022.acl-long.394","DOI":"10.18653\/v1\/2022.acl-long.394"},{"key":"2392_CR22","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1055\/a-2327-4121","volume":"15","author":"E Sezgin","year":"2024","unstructured":"Sezgin E, Sirrianni JW, Kranz K (2024) Evaluation of a digital scribe: conversation summarization for emergency department consultation calls. Appl Clin Inform, 15:600\u2013611. https:\/\/doi.org\/10.1055\/a-2327-4121","journal-title":"Appl Clin Inform"},{"key":"2392_CR23","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1038\/s41746-021-00432-5","volume":"4","author":"MM Van Buchem","year":"2021","unstructured":"Van Buchem MM, Boosman H, Bauer MP, Kant IMJ, Cammel SA, Steyerberg EW (2021) The digital scribe in clinical practice: a scoping review and research agenda. npj Digit Med 4:57. https:\/\/doi.org\/10.1038\/s41746-021-00432-5","journal-title":"npj Digit Med"},{"key":"2392_CR24","doi-asserted-by":"publisher","first-page":"358","DOI":"10.1038\/s41746-025-01622-1","volume":"8","author":"H Wang","year":"2025","unstructured":"Wang H, Yang R, Alwakeel M, Kayastha A, Chowdhury A et al (2025) An evaluation framework for ambient digital scribing tools in clinical applications. npj Digit Med 8:358. https:\/\/doi.org\/10.1038\/s41746-025-01622-1","journal-title":"npj Digit Med"},{"key":"2392_CR25","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1186\/s12911-025-03061-0","volume":"25","author":"JJW Ng","year":"2025","unstructured":"Ng JJW, Wang E, Zhou X, Zhou KX, Goh CXL et al (2025) Evaluating the performance of artificial intelligence-based speech recognition for clinical documentation: a systematic review. BMC Med Inform Decis Mak 25:236. https:\/\/doi.org\/10.1186\/s12911-025-03061-0","journal-title":"BMC Med Inform Decis Mak"},{"key":"2392_CR26","unstructured":"Pesheva E, Lee J, Kim J et al (2025) How good are AI \u2018clinicians\u2019 at medical conversations? Department Of Biomedical Informatics. https:\/\/dbmi.hms.harvard.edu\/news\/how-good-are-ai-clinicians-medical-conversations. Accessed 2 September 2025"},{"key":"2392_CR27","doi-asserted-by":"publisher","first-page":"m3164","DOI":"10.1136\/bmj.m3164","volume":"370","author":"X Liu","year":"2020","unstructured":"Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK et al (2020) Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. BMJ 370:m3164. https:\/\/doi.org\/10.1136\/bmj.m3164","journal-title":"BMJ"},{"key":"2392_CR28","doi-asserted-by":"publisher","first-page":"e070904","DOI":"10.1136\/bmj-2022-070904","volume":"377","author":"B Vasey","year":"2022","unstructured":"Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS et al (2022) Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ 377:e070904. https:\/\/doi.org\/10.1136\/bmj-2022-070904","journal-title":"BMJ"},{"key":"2392_CR29","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1038\/s41591-024-03425-5","volume":"31","author":"J Gallifant","year":"2025","unstructured":"Gallifant J, Afshar M, Ameen S, Aphinyanaphongs Y, Chen S et al (2025) The TRIPOD-LLM reporting guideline for studies using large language models. Nat Med 31:60\u201369. https:\/\/doi.org\/10.1038\/s41591-024-03425-5","journal-title":"Nat Med"},{"key":"2392_CR30","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1038\/s41746-019-0111-3","volume":"2","author":"SC Mathews","year":"2019","unstructured":"Mathews SC, McShea MJ, Hanley CL, Ravitz A, Labrique AB, Cohen AB (2019) Digital health: a path to validation. npj Digit Med 2:38. https:\/\/doi.org\/10.1038\/s41746-019-0111-3","journal-title":"npj Digit Med"},{"key":"2392_CR31","unstructured":"European Parliament and Council (2017) Regulation (EU) 2017\/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001\/83\/EC, Regulation (EC) No 178\/2002 and Regulation (EC) No 1223\/2009 and repealing Council Directives 90\/385\/EEC and 93\/42\/EEC. Official Journal of the European Union, L 117, 5 May 2017, pp 1\u2013175. https:\/\/eur-lex.europa.eu\/eli\/reg\/2017\/745\/oj. Accessed 22 Jan 2026"},{"key":"2392_CR32","unstructured":"U.S. Food and Drug Administration (2026) Clinical decision support software and AI tools in healthcare: Updated regulatory guidance. U.S. Department of Health and Human Services. https:\/\/www.fda.gov\/regulatory-information\/search-fda-guidance-documents\/clinical-decision-support-software. Accessed 22 Jan 2026"},{"key":"2392_CR33","unstructured":"NHS England (2026) Guidance on the use of AI-enabled ambient scribing products in health and care settings. NHS England. https:\/\/www.england.nhs.uk\/long-read\/guidance-on-the-use-of-ai-enabled-ambient-scribing-products-in-health-and-care-settings. Accessed 22 Jan 2026"},{"key":"2392_CR34","doi-asserted-by":"publisher","first-page":"614","DOI":"10.1377\/hlthaff.2010.0007","volume":"29","author":"DW Bates","year":"2010","unstructured":"Bates DW, Bitton A (2010) The future of health information technology in the patient-centered medical home. Health Affairs 29:614\u2013621. https:\/\/doi.org\/10.1377\/hlthaff.2010.0007","journal-title":"Health Affairs"}],"container-title":["Journal of Medical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-026-02392-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10916-026-02392-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-026-02392-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T22:38:09Z","timestamp":1781649489000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10916-026-02392-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,24]]},"references-count":34,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["2392"],"URL":"https:\/\/doi.org\/10.1007\/s10916-026-02392-3","relation":{"references":[{"id-type":"doi","id":"10.1007\/s00134-020-06045-y","asserted-by":"subject"}]},"ISSN":["1573-689X"],"issn-type":[{"value":"1573-689X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,24]]},"assertion":[{"value":"24 March 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 April 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 April 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"RB received an independent research grant from Abbott to the institute and speaker fees from Abbott, AstraZeneca Bayer, Boehringer Ingelheim, and Novartis outside the scope of the submitted work.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Generative AI was used for formatting the reference list, AI output has been manually checked.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Ethical approval was not required.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"declaration: not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Clinical Trial Number"}},{"value":"The authors declare no competing interests.","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"62"}}