{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T12:44:40Z","timestamp":1782909880436,"version":"3.54.5"},"posted":{"date-parts":[[2026,6,29]]},"group-title":"Health Informatics","reference-count":43,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T00:00:00Z","timestamp":1782691200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Institute of General Medical Sciences of the National Institutes of Health","award":["R35GM151089"],"award-info":[{"award-number":["R35GM151089"]}]},{"name":"National Institute of General Medical Sciences of the National Institutes of Health","award":["K23GM147805"],"award-info":[{"award-number":["K23GM147805"]}]},{"name":"National Library of Medicine of the National Institutes of Health","award":["R01LM013771"],"award-info":[{"award-number":["R01LM013771"]}]},{"name":"National Institute of Health Office of the Director","award":["OT2OD031919"],"award-info":[{"award-number":["OT2OD031919"]}]},{"name":"National Cancer Institute","award":["P30CA082709"],"award-info":[{"award-number":["P30CA082709"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,6,29]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Predicting health outcomes from electronic health records (EHRs) is challenging because traditional models rely on structured data and often ignore external medical knowledge. We propose an approach that integrates structured EHR with text-based clinical evidence to improve prediction and interpretability.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>We introduce PHO-Agents, a multi-agent system powered by large language models (LLMs) for health outcome prediction. Structured EHR sequences are encoded to produce attention-based representations and initial logits, which are converted into patient summaries by a data agent. A retrieval agent gathers relevant clinical guidelines. Research and practical doctor agents independently assess the patient, and a leader agent synthesizes their analyses. Outputs from the EHR-based model and the LLM agents are fused to generate final predictions and explanation reports. PHO-Agents was evaluated on three real-world cohorts: acute kidney injury (AKI) patients (in-hospital mortality), chronic kidney disease patients (AKI onset within two years), and cancer patients receiving immune checkpoint inhibitors (immune-related adverse events within one year).<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>PHO-Agents outperformed single-agent and multi-agent LLM baselines across all cohorts. In the AKI mortality task, it achieved a PR-AUC of 90.20 \u00b1 2.07, compared with 56.46 \u00b1 2.98 for the best single-agent baseline. Similar gains were observed in the ICI and CKD cohorts. Ablation studies showed that both multi-agent reasoning and logit-level fusion contributed to performance improvements, and case analyses demonstrated clinically consistent explanations.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>PHO-Agents integrates longitudinal EHR modeling with collaborative LLM reasoning, improving predictive performance, interpretability, and robustness across diverse clinical tasks. This hybrid approach offers a trustworthy strategy for real-world clinical decision support.<\/jats:p>\n                <\/jats:sec>","DOI":"10.64898\/2026.06.19.26355815","type":"posted-content","created":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T23:45:47Z","timestamp":1782776747000},"source":"Crossref","is-referenced-by-count":0,"title":["PHO-Agents: A Large Language Model\u2013Powered Multi-Agent System for Predicting Health Outcomes"],"prefix":"10.64898","author":[{"given":"Daling","family":"Shi","sequence":"first","affiliation":[{"name":"Department of Health Outcomes and Biomedical Informatics, University of Florida, FL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tyler","family":"Shugg","sequence":"additional","affiliation":[{"name":"Division of Clinical Pharmacology, Department of Medicine, Indiana University School of Medicine, IN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael T.","family":"Eadon","sequence":"additional","affiliation":[{"name":"Department of Medicine, Indiana University School of Medicine, IN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4917-6173","authenticated-orcid":false,"given":"Jing","family":"Su","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Health Data Science, Indiana University School of Medicine, IN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yijiang","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, School of medicine, Stanford university, Palo Alto, CA, USA"},{"name":"Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4455-5302","authenticated-orcid":false,"given":"Qianqian","family":"Song","sequence":"additional","affiliation":[{"name":"Department of Health Outcomes and Biomedical Informatics, University of Florida, FL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"key":"2026070103400807000_2026.06.19.26355815v1.1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2023.104522"},{"key":"2026070103400807000_2026.06.19.26355815v1.2","doi-asserted-by":"publisher","DOI":"10.1038\/srep26094"},{"key":"2026070103400807000_2026.06.19.26355815v1.3","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-018-0029-1"},{"key":"2026070103400807000_2026.06.19.26355815v1.4","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2024\/914"},{"key":"2026070103400807000_2026.06.19.26355815v1.5","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocy068"},{"key":"2026070103400807000_2026.06.19.26355815v1.6","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-018-0677-8"},{"key":"2026070103400807000_2026.06.19.26355815v1.7","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2024.1497397"},{"key":"2026070103400807000_2026.06.19.26355815v1.8","doi-asserted-by":"crossref","unstructured":"Zhang XS , Tang F , Dodge HH , Zhou J , Wang F. Metapred: Meta-learning for clinical risk prediction with limited patient electronic health records. 2019:2487\u20132495.","DOI":"10.1145\/3292500.3330779"},{"key":"2026070103400807000_2026.06.19.26355815v1.9","doi-asserted-by":"publisher","DOI":"10.1016\/j.xcrm.2023.101260"},{"issue":"6","key":"2026070103400807000_2026.06.19.26355815v1.10","doi-asserted-by":"crossref","first-page":"e1549","DOI":"10.1002\/wics.1549","article-title":"Challenges and opportunities beyond structured data in analysis of electronic health records","volume":"13","year":"2021","journal-title":"Wiley Interdisciplinary Reviews: Computational Statistics"},{"key":"2026070103400807000_2026.06.19.26355815v1.11","unstructured":"Choi E , Bahadori MT , Sun J , Kulas J , Schuetz A , Stewart W. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. Advances in neural information processing systems. 2016;29"},{"key":"2026070103400807000_2026.06.19.26355815v1.12","first-page":"191","article-title":"A Review of Challenges and Opportunities in Machine Learning for Health","volume":"2020","year":"2020","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"key":"2026070103400807000_2026.06.19.26355815v1.13","article-title":"No Black Boxes: Interpretable and Interactable Predictive Healthcare with Knowledge-Enhanced Agentic Causal Discovery","year":"2025","journal-title":"arXiv preprint"},{"key":"2026070103400807000_2026.06.19.26355815v1.14","doi-asserted-by":"publisher","DOI":"10.1186\/s13054-024-05005-y"},{"key":"2026070103400807000_2026.06.19.26355815v1.15","doi-asserted-by":"crossref","first-page":"109370","DOI":"10.1016\/j.compeleceng.2024.109370","article-title":"A review of Explainable Artificial Intelligence in healthcare","volume":"118","year":"2024","journal-title":"Computers and Electrical Engineering"},{"key":"2026070103400807000_2026.06.19.26355815v1.16","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-02004-3"},{"key":"2026070103400807000_2026.06.19.26355815v1.17","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.106187"},{"key":"2026070103400807000_2026.06.19.26355815v1.18","doi-asserted-by":"crossref","unstructured":"Chen J , Saha S , Bansal M. Reconcile: Round-table conference improves reasoning via consensus among diverse llms. 2024:7066\u20137085.","DOI":"10.18653\/v1\/2024.acl-long.381"},{"key":"2026070103400807000_2026.06.19.26355815v1.19","doi-asserted-by":"crossref","unstructured":"De Maio C , Fenza G , Furno D , Grauso T , Loia V. A multi-agent architecture for privacy-preserving natural language interaction with fhir-based electronic health records. IEEE; 2024:1\u20136.","DOI":"10.23919\/SoftCOM62040.2024.10721684"},{"key":"2026070103400807000_2026.06.19.26355815v1.20","doi-asserted-by":"crossref","unstructured":"Li H , Cheng X , Zhang X. Accurate Insights, Trustworthy Interactions: Designing a Collaborative AI-Human Multi-Agent System with Knowledge Graph for Diagnosis Prediction. 2025:1\u201315.","DOI":"10.1145\/3706598.3713526"},{"key":"2026070103400807000_2026.06.19.26355815v1.21","doi-asserted-by":"crossref","unstructured":"Liang T , He Z , Jiao W , et al. Encouraging divergent thinking in large language models through multi-agent debate. 2024:17889\u201317904.","DOI":"10.18653\/v1\/2024.emnlp-main.992"},{"key":"2026070103400807000_2026.06.19.26355815v1.22","doi-asserted-by":"crossref","unstructured":"\u00d6\u011fd\u00fc \u00c7U , Arslano\u011flu K , Karak\u00f6se M. An Adaptive Multi-Agent LLM-Based Clinical Decision Support System Integrating Biomedical RAG and Web Intelligence. IEEE Access. 2025;","DOI":"10.1109\/ACCESS.2025.3613340"},{"issue":"3","key":"2026070103400807000_2026.06.19.26355815v1.23","first-page":"1373","article-title":"Integration of multi-source medical data for medical diagnosis question answering","volume":"44","year":"2024","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"2026070103400807000_2026.06.19.26355815v1.24","doi-asserted-by":"crossref","unstructured":"Tang X , Zou A , Zhang Z , et al. Medagents: Large language models as collaborators for zero-shot medical reasoning. 2024:599\u2013621.","DOI":"10.18653\/v1\/2024.findings-acl.33"},{"key":"2026070103400807000_2026.06.19.26355815v1.25","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocaf038"},{"key":"2026070103400807000_2026.06.19.26355815v1.26","article-title":"Medhallu: A comprehensive benchmark for detecting medical hallucinations in large language models","year":"2025","journal-title":"arXiv preprint"},{"key":"2026070103400807000_2026.06.19.26355815v1.27","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01940-4"},{"key":"2026070103400807000_2026.06.19.26355815v1.28","doi-asserted-by":"crossref","unstructured":"Wang Z , Zhu Y , Zhao H , et al. Colacare: Enhancing electronic health record modeling through large language model-driven multi-agent collaboration. 2025:2250\u20132261.","DOI":"10.1145\/3696410.3714877"},{"key":"2026070103400807000_2026.06.19.26355815v1.29","doi-asserted-by":"crossref","unstructured":"Yue L , Xing S , Chen J , Fu T. Clinicalagent: Clinical trial multi-agent system with large language model-based reasoning. 2024:1\u201310.","DOI":"10.1145\/3698587.3701359"},{"key":"2026070103400807000_2026.06.19.26355815v1.30","unstructured":"Johnson A , Bulgarelli L , Pollard T , Horng S , Celi LA , Mark R. Mimic-iv . PhysioNet Available online at: https:\/\/physionet org\/content\/mimiciv\/10\/(accessed August 23, 2021). 2020:49\u201355."},{"key":"2026070103400807000_2026.06.19.26355815v1.31","unstructured":"About Us \u00bb Integrated Data Repository Research Services \u00bb Clinicaland Translational Science Institute \u00bb University of Florida. Accessed April 6, 2022, https:\/\/idr.ufhealth.org\/about-us\/"},{"key":"2026070103400807000_2026.06.19.26355815v1.32","doi-asserted-by":"crossref","unstructured":"Hamer RM , Simpson PM . Last observation carried forward versus mixed models in the analysis of psychiatric clinical trials. American Psychiatric Association; 2009. p. 639\u2013641.","DOI":"10.1176\/appi.ajp.2009.09040458"},{"issue":"1","key":"2026070103400807000_2026.06.19.26355815v1.33","first-page":"2013","article-title":"PubMed: the bibliographic database","volume":"2","year":"2013","journal-title":"The NCBI handbook"},{"key":"2026070103400807000_2026.06.19.26355815v1.34","doi-asserted-by":"publisher","DOI":"10.1159\/000339789"},{"key":"2026070103400807000_2026.06.19.26355815v1.35","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2017.77.6385"},{"key":"2026070103400807000_2026.06.19.26355815v1.36","unstructured":"Porter RS , Kaplan JL . The Merck manual of diagnosis and therapy. Merck Sharp 2 Dohme Corp.; 2011."},{"key":"2026070103400807000_2026.06.19.26355815v1.37","article-title":"The llama 3 herd of models","year":"2024","journal-title":"arXiv preprint"},{"key":"2026070103400807000_2026.06.19.26355815v1.38","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btad651"},{"key":"2026070103400807000_2026.06.19.26355815v1.39","first-page":"22199","article-title":"Large language models are zero-shot reasoners","volume":"35","year":"2022","journal-title":"Advances in neural information processing systems"},{"key":"2026070103400807000_2026.06.19.26355815v1.40","doi-asserted-by":"crossref","unstructured":"Reynolds L , McDonell K. Prompt programming for large language models: Beyond the few-shot paradigm. 2021:1-7.","DOI":"10.1145\/3411763.3451760"},{"key":"2026070103400807000_2026.06.19.26355815v1.41","article-title":"Self-consistency improves chain of thought reasoning in language models","year":"2022","journal-title":"arXiv preprint"},{"key":"2026070103400807000_2026.06.19.26355815v1.42","doi-asserted-by":"publisher","DOI":"10.4103\/tjem.tjem_182_23"},{"issue":"4","key":"2026070103400807000_2026.06.19.26355815v1.43","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1111\/2041-210X.13140","article-title":"The area under the precision-recall curve as a performance metric for rare binary events","volume":"10","year":"2019","journal-title":"Methods in Ecology and Evolution"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.06.19.26355815","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T11:51:08Z","timestamp":1782906668000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.64898\/2026.06.19.26355815"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,29]]},"references-count":43,"URL":"https:\/\/doi.org\/10.64898\/2026.06.19.26355815","relation":{},"subject":[],"published":{"date-parts":[[2026,6,29]]},"subtype":"preprint"}}