{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T21:57:19Z","timestamp":1787003839054,"version":"build-2736575974"},"reference-count":88,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T00:00:00Z","timestamp":1780272000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T00:00:00Z","timestamp":1780272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Hong Kong Research Grants Council (RGC) General Research Fund","award":["14220622"],"award-info":[{"award-number":["14220622"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Health Syst."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Inpatient pathways require complex clinical decision-making based on comprehensive patient information, yet research on medical LLMs is limited in this area due to the lack of large-scale datasets. Existing medical benchmarks primarily focused on question-answering and examinations, overlooking the multifaceted nature of inpatient decision-making. To address this gap, we developed the IPDS benchmark, comprising 51,274 cases across 9 triage departments, 17 major disease categories, and 16 treatment options. We further proposed the Multi-Agent Inpatient Pathways (MAP) framework, containing three specialized clinical agents: a triage agent for patient admission, a diagnosis agent for diagnostic decision-making, and a treatment agent for care planning. A chief agent guides and promotes these agents to ensure coordination. Experiments demonstrated that MAP achieved superior alignment with operational protocols compared to state-of-the-art LLMs. The MAP sets a foundation for advancing inpatient support systems, offering significant potential for enhancing operational efficiency and resource planning in healthcare facilities.<\/jats:p>","DOI":"10.1038\/s44401-026-00085-0","type":"journal-article","created":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T07:42:28Z","timestamp":1780299748000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["MAP: evaluation and multi-agent enhancement of large language models for inpatient pathways"],"prefix":"10.1038","volume":"3","author":[{"given":"Zhen","family":"Chen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhihao","family":"Peng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xusheng","family":"Liang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cheng","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peigan","family":"Liang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Linsheng","family":"Zeng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Minjie","family":"Ju","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yixuan","family":"Yuan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,6,1]]},"reference":[{"key":"85_CR1","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1197\/j.aem.2007.01.003","volume":"14","author":"K Alagappan","year":"2007","unstructured":"Alagappan, K. et al. International emergency medicine and the role for academic emergency medicine. Acad. Emerg. Med. 14, 451\u2013456 (2007).","journal-title":"Acad. Emerg. Med."},{"key":"85_CR2","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","volume":"25","author":"EJ Topol","year":"2019","unstructured":"Topol, E. J. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25, 44\u201356 (2019).","journal-title":"Nat. Med."},{"key":"85_CR3","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1038\/s41591-021-01614-0","volume":"28","author":"P Rajpurkar","year":"2022","unstructured":"Rajpurkar, P., Chen, E., Banerjee, O. & Topol, E. J. AI in health and medicine. Nat. Med. 28, 31\u201338 (2022).","journal-title":"Nat. Med."},{"key":"85_CR4","doi-asserted-by":"publisher","first-page":"AIe2400472","DOI":"10.1056\/AIe2400472","volume":"1","author":"R Avram","year":"2024","unstructured":"Avram, R. & Fearon, W. F. AI-RISE to the challenge\u2014artificial intelligence reduces time to treatment in STEMI. NEJM AI 1, AIe2400472 (2024).","journal-title":"NEJM AI"},{"key":"85_CR5","doi-asserted-by":"publisher","first-page":"2613","DOI":"10.1038\/s41591-024-03097-1","volume":"30","author":"P Hager","year":"2024","unstructured":"Hager, P. et al. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nat. Med. 30, 2613\u20132622 (2024).","journal-title":"Nat. Med."},{"key":"85_CR6","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","volume":"30","author":"D Van Veen","year":"2024","unstructured":"Van Veen, D. et al. Adapted large language models can outperform medical experts in clinical text summarization. Nat. Med. 30, 1134\u20131142 (2024).","journal-title":"Nat. Med."},{"key":"85_CR7","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1093\/intqhc\/mzr038","volume":"23","author":"T Schoenfelder","year":"2011","unstructured":"Schoenfelder, T., Klewer, J. & Kugler, J. Determinants of patient satisfaction: a study among 39 hospitals in an in-patient setting in Germany. Int. J. Qual. Health Care 23, 503\u2013509 (2011).","journal-title":"Int. J. Qual. Health Care"},{"key":"85_CR8","doi-asserted-by":"publisher","first-page":"138ra78","DOI":"10.1126\/scitranslmed.3003544","volume":"4","author":"TJ Dixon-Salazar","year":"2012","unstructured":"Dixon-Salazar, T. J. et al. Exome sequencing can improve diagnosis and alter patient management. Sci. Transl. Med. 4, 138ra78\u2013138ra78 (2012).","journal-title":"Sci. Transl. Med."},{"key":"85_CR9","doi-asserted-by":"crossref","unstructured":"Xu Hua, D. D. F. Natural Language Processing in Biomedicine: A Practical Guide (Springer International Publishing, 2024).","DOI":"10.1007\/978-3-031-55865-8"},{"key":"85_CR10","doi-asserted-by":"publisher","first-page":"1257","DOI":"10.1038\/s41591-024-02970-3","volume":"30","author":"C Varghese","year":"2024","unstructured":"Varghese, C., Harrison, E. M., O\u2019Grady, G. & Topol, E. J. Artificial intelligence in surgery. Nat. Med. 30, 1257\u20131268 (2024).","journal-title":"Nat. Med."},{"key":"85_CR11","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1515\/dx-2017-0012","volume":"4","author":"ML Graber","year":"2017","unstructured":"Graber, M. L., Byrne, C. & Johnston, D. The impact of electronic health records on diagnosis. Diagnosis 4, 211\u2013223 (2017).","journal-title":"Diagnosis"},{"key":"85_CR12","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-17431-x","volume":"11","author":"SM Lauritsen","year":"2020","unstructured":"Lauritsen, S. M. et al. Explainable artificial intelligence model to predict acute critical illness from electronic health records. Nat. Commun. 11, 3852 (2020).","journal-title":"Nat. Commun."},{"key":"85_CR13","doi-asserted-by":"publisher","first-page":"1384","DOI":"10.1038\/s41551-022-00872-8","volume":"6","author":"G Erion","year":"2022","unstructured":"Erion, G. et al. CoAI: cost-aware artificial intelligence for health care. Nat. Biomed. Eng. 6, 1384 (2022).","journal-title":"Nat. Biomed. Eng."},{"key":"85_CR14","doi-asserted-by":"crossref","unstructured":"Underdahl, L., Ditri, M. & Duthely, L. M. Physician burnout: evidence-based roadmaps to prioritizing and supporting personal wellbeing. J. Healthc. Leadersh. 15\u201327 (2024).","DOI":"10.2147\/JHL.S389245"},{"key":"85_CR15","doi-asserted-by":"crossref","unstructured":"Ong, J. C. L. et al. Artificial intelligence, ChatGPT, and other large language models for social determinants of health: current state and future directions. Cell Rep. Med. 5 (2024).","DOI":"10.1016\/j.xcrm.2023.101356"},{"key":"85_CR16","doi-asserted-by":"crossref","unstructured":"Chen, E. et al. Multimodal clinical benchmark for emergency care (MC-BEC): a comprehensive benchmark for evaluating foundation models in emergency medicine. Adv. Neural Inf. Process. Syst. 36 (2024).","DOI":"10.52202\/075280-1984"},{"key":"85_CR17","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01029-4","volume":"7","author":"L Wang","year":"2024","unstructured":"Wang, L. et al. Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs. npj Digital Med. 7, 41 (2024).","journal-title":"npj Digital Med."},{"key":"85_CR18","doi-asserted-by":"publisher","first-page":"1296","DOI":"10.1038\/s41591-023-02341-4","volume":"29","author":"C Preiksaitis","year":"2023","unstructured":"Preiksaitis, C., Sinsky, C. A. & Rose, C. Chatgpt is not the solution to physicians\u2019 documentation burden. Nat. Med. 29, 1296\u20131297 (2023).","journal-title":"Nat. Med."},{"key":"85_CR19","doi-asserted-by":"publisher","first-page":"AIra2400012","DOI":"10.1056\/AIra2400012","volume":"1","author":"J Wu","year":"2024","unstructured":"Wu, J. et al. Clinical text datasets for medical artificial intelligence and large language models\u2014a systematic review. NEJM AI 1, AIra2400012 (2024).","journal-title":"NEJM AI"},{"key":"85_CR20","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","volume":"616","author":"M Moor","year":"2023","unstructured":"Moor, M. et al. Foundation models for generalist medical artificial intelligence. Nature 616, 259\u2013265 (2023).","journal-title":"Nature"},{"key":"85_CR21","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-50043-3","volume":"15","author":"J Zhou","year":"2024","unstructured":"Zhou, J. et al. Pre-trained multimodal large language model enhances dermatological diagnosis using skingpt-4. Nat. Commun. 15, 5649 (2024).","journal-title":"Nat. Commun."},{"key":"85_CR22","doi-asserted-by":"publisher","unstructured":"Kanithi, P. K. et al. Medic: towards a comprehensive framework for evaluating LLMs in clinical applications. arXiv https:\/\/doi.org\/10.48550\/arXiv.2409.07314 (2024).","DOI":"10.48550\/arXiv.2409.07314"},{"key":"85_CR23","doi-asserted-by":"publisher","first-page":"2838","DOI":"10.1038\/s41591-024-03113-4","volume":"30","author":"Y Yang","year":"2024","unstructured":"Yang, Y., Zhang, H., Gichoya, J. W., Katabi, D. & Ghassemi, M. The limits of fair medical imaging ai in real-world generalization. Nat. Med. 30, 2838\u20132848 (2024).","journal-title":"Nat. Med."},{"key":"85_CR24","doi-asserted-by":"publisher","first-page":"e45312","DOI":"10.2196\/45312","volume":"9","author":"A Gilson","year":"2023","unstructured":"Gilson, A. et al. How does ChatGPT perform on the United States Medical Licensing Examination (USMLE)? The implications of large language models for medical education and knowledge assessment. JMIR Med. Educ. 9, e45312 (2023).","journal-title":"JMIR Med. Educ."},{"key":"85_CR25","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1038\/s41591-024-03328-5","volume":"31","author":"S Johri","year":"2025","unstructured":"Johri, S. et al. An evaluation framework for clinical use of large language models in patient interaction tasks. Nat. Med. 31, 77\u201386 (2025).","journal-title":"Nat. Med."},{"key":"85_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"AE Johnson","year":"2016","unstructured":"Johnson, A. E. et al. MIMIC-III, a freely accessible critical care database. Sci. Data 3, 1\u20139 (2016).","journal-title":"Sci. Data"},{"key":"85_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2018.178","volume":"5","author":"TJ Pollard","year":"2018","unstructured":"Pollard, T. J. et al. The eicu collaborative research database, a freely available multi-center database for critical care research. Sci. Data 5, 1\u201313 (2018).","journal-title":"Sci. Data"},{"key":"85_CR28","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1038\/s41591-020-0789-4","volume":"26","author":"SL Hyland","year":"2020","unstructured":"Hyland, S. L. et al. Early prediction of circulatory failure in the intensive care unit using machine learning. Nat. Med. 26, 364\u2013373 (2020).","journal-title":"Nat. Med."},{"key":"85_CR29","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-020-0355-4","volume":"7","author":"X Zeng","year":"2020","unstructured":"Zeng, X. et al. PIC, a paediatric-specific intensive care database. Sci. Data 7, 14 (2020).","journal-title":"Sci. Data"},{"key":"85_CR30","doi-asserted-by":"publisher","first-page":"e563","DOI":"10.1097\/CCM.0000000000004916","volume":"49","author":"PJ Thoral","year":"2021","unstructured":"Thoral, P. J. et al. Sharing ICU patient data responsibly under the Society of Critical Care Medicine\/European Society of Intensive Care Medicine Joint Data Science Collaboration: the Amsterdam University Medical Centers Database (AmsterdamUMCdb) example. Crit. Care Med. 49, e563\u2013e577 (2021).","journal-title":"Crit. Care Med."},{"key":"85_CR31","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-022-01899-x","volume":"10","author":"AE Johnson","year":"2023","unstructured":"Johnson, A. E. et al. MIMIC-IV, a freely accessible electronic health record dataset. Sci. Data 10, 1 (2023).","journal-title":"Sci. Data"},{"key":"85_CR32","unstructured":"Johnson, A., Pollard, T., Horng, S., Celi, L. & Mark IV, R.MIMIC-IV-note: deidentified free-text clinical notes (version 2.2). physionet. (2023)."},{"key":"85_CR33","unstructured":"Henry, J., Pylypchuk, Y., Searcy, T. & Patel, V. Adoption of electronic health record systems among US non-federal acute care hospitals: 2008-2015. ASTP Health IT Data Brief (2016)."},{"key":"85_CR34","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1093\/jamia\/ocx084","volume":"25","author":"AE Johnson","year":"2018","unstructured":"Johnson, A. E., Stone, D. J., Celi, L. A. & Pollard, T. J. The mimic code repository: enabling reproducibility in critical care research. J. Am. Med. Inform. Assoc. 25, 32\u201339 (2018).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"85_CR35","doi-asserted-by":"publisher","first-page":"6421","DOI":"10.3390\/app11146421","volume":"11","author":"D Jin","year":"2021","unstructured":"Jin, D. et al. What disease does this patient have? A large-scale open domain question answering dataset from medical exams. Appl. Sci. 11, 6421 (2021).","journal-title":"Appl. Sci."},{"key":"85_CR36","unstructured":"Pal, A., Umapathi, L. K. & Sankarasubbu, M. Medmcqa: a large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on Health, Inference, And Learning, 248\u2013260 (PMLR, 2022)."},{"key":"85_CR37","first-page":"67125","volume":"36","author":"M Wornow","year":"2023","unstructured":"Wornow, M., Thapa, R., Steinberg, E., Fries, J. & Shah, N. EHRSHOT: an EHR benchmark for few-shot evaluation of foundation models. Adv. Neural Inf. Process. Syst. 36, 67125\u201367137 (2023).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"85_CR38","first-page":"17709","volume":"38","author":"Y Cai","year":"2024","unstructured":"Cai, Y. et al. Medbench: a large-scale chinese benchmark for evaluating medical large language models. Proc. AAAI Conf. Artif. Intell. 38, 17709\u201317717 (2024).","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"85_CR39","doi-asserted-by":"crossref","unstructured":"Bae, S. et al. EHRXQA: a multi-modal question answering dataset for electronic health records with chest X-ray images. Adv. Neural Inf. Process. Syst. 36 (2024).","DOI":"10.52202\/075280-0170"},{"key":"85_CR40","unstructured":"Organization, W. H. International Statistical Classification of Diseases and related health problems: Alphabetical Index, Vol. 3 (World Health Organization, 2004)."},{"key":"85_CR41","doi-asserted-by":"publisher","unstructured":"Chen, J. et al. HuatuoGPT-II, one-stage training for medical adaption of LLMs. arXiv https:\/\/doi.org\/10.48550\/arXiv.2311.09774 (2023).","DOI":"10.48550\/arXiv.2311.09774"},{"key":"85_CR42","doi-asserted-by":"publisher","unstructured":"Chen, Z. et al. Meditron-70b: scaling medical pretraining for large language models. arXiv https:\/\/doi.org\/10.48550\/arXiv.2311.16079 (2023).","DOI":"10.48550\/arXiv.2311.16079"},{"key":"85_CR43","doi-asserted-by":"publisher","unstructured":"Toma, A. et al. Clinical Camel: an open-source expert-level medical language model with dialogue-based knowledge encoding. arXiv https:\/\/doi.org\/10.48550\/arXiv.2305.12031 (2023).","DOI":"10.48550\/arXiv.2305.12031"},{"key":"85_CR44","unstructured":"Cai, Z. et al. InternLM2 technical report. (2024)."},{"key":"85_CR45","doi-asserted-by":"publisher","unstructured":"Dubey, A. et al. The Llama 3 herd of models. arXiv https:\/\/doi.org\/10.48550\/arXiv.2407.21783 (2024).","DOI":"10.48550\/arXiv.2407.21783"},{"key":"85_CR46","doi-asserted-by":"publisher","first-page":"1179","DOI":"10.1007\/s11739-017-1718-5","volume":"12","author":"S Alerhand","year":"2017","unstructured":"Alerhand, S., Wood, S., Long, B. & Koyfman, A. The time-sensitive challenge of diagnosing spinal epidural abscess in the emergency department. Intern. Emerg. Med. 12, 1179\u20131183 (2017).","journal-title":"Intern. Emerg. Med."},{"key":"85_CR47","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/j.annemergmed.2009.12.013","volume":"56","author":"BG Carr","year":"2010","unstructured":"Carr, B. G., Conway, P. H., Meisel, Z. F., Steiner, C. A. & Clancy, C. Defining the emergency care sensitive condition: a health policy research agenda in emergency medicine. Ann. Emerg. Med. 56, 49\u201351 (2010).","journal-title":"Ann. Emerg. Med."},{"key":"85_CR48","doi-asserted-by":"publisher","first-page":"490","DOI":"10.1097\/ACM.0b013e3181ccb628","volume":"85","author":"DP Sklar","year":"2010","unstructured":"Sklar, D. P. et al. The future of emergency medicine: an evolutionary perspective. Acad. Med. 85, 490\u2013495 (2010).","journal-title":"Acad. Med."},{"key":"85_CR49","doi-asserted-by":"publisher","first-page":"1212","DOI":"10.1197\/j.aem.2006.04.010","volume":"13","author":"DJ France","year":"2006","unstructured":"France, D. J. & Levin, S. System complexity as a measure of safe capacity for the emergency department. Acad. Emerg. Med. 13, 1212\u20131219 (2006).","journal-title":"Acad. Emerg. Med."},{"key":"85_CR50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1472-6947-8-32","volume":"8","author":"I Neamatullah","year":"2008","unstructured":"Neamatullah, I. et al. Automated de-identification of free-text medical records. BMC Med. Inform. Decis. Mak. 8, 1\u201317 (2008).","journal-title":"BMC Med. Inform. Decis. Mak."},{"key":"85_CR51","doi-asserted-by":"crossref","unstructured":"Johnson, A. E., Bulgarelli, L. & Pollard, T. J. Deidentification of free-text medical records using pre-trained bidirectional transformers. In Proceedings of the ACM Conference on Health, Inference, and Learning, 214\u2013221 (2020).","DOI":"10.1145\/3368555.3384455"},{"key":"85_CR52","doi-asserted-by":"publisher","first-page":"113172","DOI":"10.1016\/j.socscimed.2020.113172","volume":"260","author":"J Morley","year":"2020","unstructured":"Morley, J. et al. The ethics of AI in health care: a mapping review. Soc. Sci. Med. 260, 113172 (2020).","journal-title":"Soc. Sci. Med."},{"key":"85_CR53","doi-asserted-by":"publisher","first-page":"e36671","DOI":"10.1097\/MD.0000000000036671","volume":"102","author":"C Elendu","year":"2023","unstructured":"Elendu, C. et al. Ethical implications of ai and robotics in healthcare: a review. Medicine 102, e36671 (2023).","journal-title":"Medicine"},{"key":"85_CR54","unstructured":"ruslanmv. Medical-llama3\u20138b-16bit: fine-tuned llama3 for medical q&a. huggingface (2024)."},{"key":"85_CR55","doi-asserted-by":"publisher","unstructured":"Chen, J. et al. HuatuoGPT-o1, towards medical complex reasoning with llms. arXiv https:\/\/doi.org\/10.48550\/arXiv.2412.18925 (2024).","DOI":"10.48550\/arXiv.2412.18925"},{"key":"85_CR56","doi-asserted-by":"publisher","first-page":"681","DOI":"10.1007\/s11023-020-09548-1","volume":"30","author":"L Floridi","year":"2020","unstructured":"Floridi, L. & Chiriatti, M. Gpt-3: its nature, scope, limits, and consequences. Minds Mach. 30, 681\u2013694 (2020).","journal-title":"Minds Mach."},{"key":"85_CR57","doi-asserted-by":"publisher","unstructured":"Achiam, J. et al. GPT-4 technical report. arXiv https:\/\/doi.org\/10.48550\/arXiv.2303.08774 (2023).","DOI":"10.48550\/arXiv.2303.08774"},{"key":"85_CR58","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1038\/s41591-024-03423-7","volume":"31","author":"K Singhal","year":"2025","unstructured":"Singhal, K., Tu, T. & Gottweis, J. Toward expert-level medical question answering with large language models. Nature medicine 31, 943\u2013950 (2025).","journal-title":"Nature medicine"},{"key":"85_CR59","doi-asserted-by":"publisher","first-page":"102938","DOI":"10.1016\/j.artmed.2024.102938","volume":"155","author":"I Alonso","year":"2024","unstructured":"Alonso, I., Oronoz, M. & Agerri, R. MedExpQA: multilingual benchmarking of large language models for medical question answering. Artif. Intell. Med. 155, 102938 (2024).","journal-title":"Artif. Intell. Med."},{"key":"85_CR60","doi-asserted-by":"crossref","unstructured":"Patil, S. & Davies, P. Use of Google translate in medical communication: evaluation of accuracy. BMJ 349 (2014).","DOI":"10.1136\/bmj.g7392"},{"key":"85_CR61","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-019-0158-1","volume":"2","author":"M Lehne","year":"2019","unstructured":"Lehne, M., Sass, J., Essenwanger, A., Schepers, J. & Thun, S. Why digital medicine depends on interoperability. npj Digital Med. 2, 79 (2019).","journal-title":"npj Digital Med."},{"key":"85_CR62","unstructured":"Lovaasen, K. R. & Schwerdtfeger, J. ICD-9-CM Coding: Theory and Practice with ICD-10, 2013\/2014 Edition-E-Book (Elsevier Health Sciences, 2012)."},{"key":"85_CR63","unstructured":"American College of Cardiology, Guideline for the prevention, detection, evaluation, and management of high blood pressure in adults. J. Am. Coll. Cardiol. 71 (2017)."},{"key":"85_CR64","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-0221-y","volume":"3","author":"RT Sutton","year":"2020","unstructured":"Sutton, R. T. et al. An overview of clinical decision support systems: benefits, risks, and strategies for success. npj Digital Med. 3, 17 (2020).","journal-title":"npj Digital Med."},{"key":"85_CR65","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-022-00705-7","volume":"5","author":"H Dong","year":"2022","unstructured":"Dong, H. et al. Automated clinical coding: what, why, and where we are? npj Digital Med. 5, 159 (2022).","journal-title":"npj Digital Med."},{"key":"85_CR66","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00899-4","volume":"6","author":"IJ Borges do Nascimento","year":"2023","unstructured":"Borges do Nascimento, I. J. et al. Barriers and facilitators to utilizing digital health technologies by healthcare professionals. npj Digital Med. 6, 161 (2023).","journal-title":"npj Digital Med."},{"key":"85_CR67","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00768-0","volume":"6","author":"KP Venkatesh","year":"2023","unstructured":"Venkatesh, K. P., Raza, M. M. & Kvedar, J. C. Automating the overburdened clinical coding system: challenges and next steps. npj Digital Med. 6, 16 (2023).","journal-title":"npj Digital Med."},{"key":"85_CR68","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1038\/s44401-024-00008-x","volume":"2","author":"M Kopka","year":"2025","unstructured":"Kopka, M., Wang, S. M., Kunz, S., Schmid, C. & Feufel, M. A. Technology-supported self-triage decision making. npj Health Syst. 2, 3 (2025).","journal-title":"npj Health Syst."},{"key":"85_CR69","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1038\/s43856-025-01052-w","volume":"5","author":"T Liu","year":"2025","unstructured":"Liu, T. et al. A foundational triage system for improving accuracy in moderate acuity level emergency classifications. Commun. Med. 5, 322 (2025).","journal-title":"Commun. Med."},{"key":"85_CR70","doi-asserted-by":"publisher","first-page":"1130","DOI":"10.1097\/01.mlr.0000182534.19832.83","volume":"43","author":"H Quan","year":"2005","unstructured":"Quan, H. et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med. Care 43, 1130\u20131139 (2005).","journal-title":"Med. Care"},{"key":"85_CR71","doi-asserted-by":"publisher","first-page":"1424","DOI":"10.1111\/j.1475-6773.2007.00822.x","volume":"43","author":"H Quan","year":"2008","unstructured":"Quan, H. et al. Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database. Health Serv. Res. 43, 1424\u20131441 (2008).","journal-title":"Health Serv. Res."},{"key":"85_CR72","first-page":"188","volume":"12","author":"WP Glasheen","year":"2019","unstructured":"Glasheen, W. P. et al. Charlson comorbidity index: ICD-9 update and ICD-10 translation. Am. Health Drug Benefits 12, 188 (2019).","journal-title":"Am. Health Drug Benefits"},{"key":"85_CR73","doi-asserted-by":"publisher","unstructured":"Huang, K., Altosaar, J. & Ranganath, R. ClinicalBERT: modeling clinical notes and predicting hospital readmission. arXiv https:\/\/doi.org\/10.48550\/arXiv.1904.05342 (2019).","DOI":"10.48550\/arXiv.1904.05342"},{"key":"85_CR74","unstructured":"NICE: National Institute for Health and Care Excellence. In Department of Health and Social Care. https:\/\/www.nice.org.uk\/."},{"key":"85_CR75","unstructured":"Liu, J. LlamaIndex. https:\/\/github.com\/jerryjliu\/llama_index (2022)."},{"key":"85_CR76","first-page":"24824","volume":"35","author":"J Wei","year":"2022","unstructured":"Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models. Adv. Neural Inf. Process. Syst. 35, 24824\u201324837 (2022).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"85_CR77","first-page":"37","volume":"2","author":"D Powers","year":"2011","unstructured":"Powers, D. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness & correlation. J. Mach. Learn. Technol. 2, 37\u201363 (2011).","journal-title":"J. Mach. Learn. Technol."},{"key":"85_CR78","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1111\/j.1466-8238.2007.00358.x","volume":"17","author":"JM Lobo","year":"2008","unstructured":"Lobo, J. M., Jim\u00e9nez-Valverde, A. & Real, R. AUC: a misleading measure of the performance of predictive distribution models. Glob. Ecol. Biogeogr. 17, 145\u2013151 (2008).","journal-title":"Glob. Ecol. Biogeogr."},{"key":"85_CR79","doi-asserted-by":"crossref","unstructured":"Yerushalmy, J. Statistical problems in assessing methods of medical diagnosis, with special reference to X-ray techniques. Public Health Rep. (1896\u20131970), JSTOR, 62, 1432\u201349 (1947). Accessed 26 Mar 2026.","DOI":"10.2307\/4586294"},{"key":"85_CR80","first-page":"291","volume":"125","author":"A Saah","year":"1998","unstructured":"Saah, A. & Hoover, D. Sensitivity and specificity revisited: significance of the terms in analytic and diagnostic language. Annales de. Dermatologie et de. Venereologie 125, 291\u2013294 (1998).","journal-title":"Annales de. Dermatologie et de. Venereologie"},{"key":"85_CR81","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/BM.2012.031","volume":"22","author":"ML McHugh","year":"2012","unstructured":"McHugh, M. L. Interrater reliability: the kappa statistic. Biochem. Med. 22, 276\u2013282 (2012).","journal-title":"Biochem. Med."},{"key":"85_CR82","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1016\/0005-2795(75)90109-9","volume":"405","author":"BW Matthews","year":"1975","unstructured":"Matthews, B. W. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochim. Biophys Acta Protein Struct. 405, 442\u2013451 (1975).","journal-title":"Biochim. Biophys Acta Protein Struct."},{"key":"85_CR83","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","volume":"15","author":"TK Koo","year":"2016","unstructured":"Koo, T. K. & Li, M. Y. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J. Chiropr. Med. 15, 155\u2013163 (2016).","journal-title":"J. Chiropr. Med."},{"key":"85_CR84","doi-asserted-by":"crossref","unstructured":"Jin, Q., Dhingra, B., Liu, Z., Cohen, W. & Lu, X. PubMedQA: a dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2567\u20132577 (2019).","DOI":"10.18653\/v1\/D19-1259"},{"key":"85_CR85","doi-asserted-by":"crossref","unstructured":"Wang, X. et al. CMB: a comprehensive medical benchmark in Chinese. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 6184\u20136205 (2024).","DOI":"10.18653\/v1\/2024.naacl-long.343"},{"key":"85_CR86","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","volume":"620","author":"K Singhal","year":"2023","unstructured":"Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172\u2013180 (2023).","journal-title":"Nature"},{"key":"85_CR87","unstructured":"Liu, J. et al. Benchmarking large language models on cmexam\u2014a comprehensive chinese medical exam dataset. Adv. Neural Inf. Process. Syst. 36 (2024)."},{"key":"85_CR88","unstructured":"Yan, W. et al. Clinicallab: Aligning agents for multi-departmental clinical diagnostics in the real world. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track (2025)."}],"container-title":["npj Health Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s44401-026-00085-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s44401-026-00085-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s44401-026-00085-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T09:03:01Z","timestamp":1780304581000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s44401-026-00085-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,1]]},"references-count":88,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["85"],"URL":"https:\/\/doi.org\/10.1038\/s44401-026-00085-0","relation":{},"ISSN":["3005-1959"],"issn-type":[{"value":"3005-1959","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,1]]},"assertion":[{"value":"25 March 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 March 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 June 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing financial interests. Y.Y. is an Associate Editor for npj Digital Medicine. Y.Y. was not involved in the review of, or decisions related to, this manuscript in npj Health Systems.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"37"}}