{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:43:18Z","timestamp":1787017398010,"version":"build-2736575974"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,10,13]],"date-time":"2023-10-13T00:00:00Z","timestamp":1697155200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Food and Drug Administration"},{"DOI":"10.13039\/100000016","name":"U.S. Department of Health and Human Services","doi-asserted-by":"crossref","award":["U01FD005938"],"award-info":[{"award-number":["U01FD005938"]}],"id":[{"id":"10.13039\/100000016","id-type":"DOI","asserted-by":"crossref"}]},{"name":"American College of Cardiology Foundation\u2019s National Cardiovascular Data Registry"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Healthcare"],"published-print":{"date-parts":[[2023,10,31]]},"abstract":"<jats:p>Observational medical data present unique opportunities for analysis of medical outcomes and treatment decision making. However, because these datasets do not contain the strict pairing of randomized control trials, matching techniques are to draw comparisons among patients. A key limitation to such techniques is verification that the variables used to model treatment decision making are also relevant in identifying the risk of major adverse events. This article explores a deep mixture of experts approach to jointly learn how to match patients and model the risk of major adverse events in patients. Although trained with information regarding treatment and outcomes, after training, the proposed model is decomposable into a network that clusters patients into phenotypes from information available before treatment. This model is validated on a dataset of patients with acute myocardial infarction complicated by cardiogenic shock. The mixture of experts approach can predict the outcome of mortality with an area under the receiver operating characteristic curve of 0.85 \u00b1 0.01 while jointly discovering five potential phenotypes of interest. The technique and interpretation allow for identifying clinically relevant phenotypes that may be used both for outcomes modeling as well as potentially evaluating individualized treatment effects.<\/jats:p>","DOI":"10.1145\/3616021","type":"journal-article","created":{"date-parts":[[2023,8,18]],"date-time":"2023-08-18T08:39:45Z","timestamp":1692347985000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Clinical Phenotyping with an Outcomes-driven Mixture of Experts for Patient Matching and Risk Estimation"],"prefix":"10.1145","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3055-8825","authenticated-orcid":false,"given":"Nathan C.","family":"Hurley","sequence":"first","affiliation":[{"name":"Texas A&amp;M University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0674-2032","authenticated-orcid":false,"given":"Sanket S.","family":"Dhruva","sequence":"additional","affiliation":[{"name":"University of California San Francisco, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2384-2545","authenticated-orcid":false,"given":"Nihar R.","family":"Desai","sequence":"additional","affiliation":[{"name":"Yale University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9218-3320","authenticated-orcid":false,"given":"Joseph R.","family":"Ross","sequence":"additional","affiliation":[{"name":"Yale University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5935-5744","authenticated-orcid":false,"given":"Che G.","family":"Ngufor","sequence":"additional","affiliation":[{"name":"Mayo Clinic, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5681-6709","authenticated-orcid":false,"given":"Frederick","family":"Masoudi","sequence":"additional","affiliation":[{"name":"Ascension Health, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2046-127X","authenticated-orcid":false,"given":"Harlan M.","family":"Krumholz","sequence":"additional","affiliation":[{"name":"Yale University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2655-2095","authenticated-orcid":false,"given":"Bobak J.","family":"Mortazavi","sequence":"additional","affiliation":[{"name":"Texas A&amp;M University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,10,13]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacc.2014.07.979"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2016.10.007"},{"issue":"1","key":"e_1_3_2_4_2","doi-asserted-by":"crossref","first-page":"63","DOI":"10.2217\/cer.13.89","article-title":"Applications of propensity score methods in observational comparative effectiveness and safety research: Where have we come and where should we go?","volume":"3","author":"Borah Bijan J.","year":"2014","unstructured":"Bijan J. Borah, James P. Moriarty, William H. Crown, and Jalpa A. Doshi. 2014. Applications of propensity score methods in observational comparative effectiveness and safety research: Where have we come and where should we go? J. Compar. Effect. Res. 3, 1 (2014), 63\u201378.","journal-title":"J. Compar. Effect. Res."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368555.3384465"},{"issue":"8","key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1001\/jama.2020.0254","article-title":"Association of use of an intravascular microaxial left ventricular assist device vs intra-aortic balloon pump with in-hospital mortality and major bleeding among patients with acute myocardial infarction complicated by cardiogenic shock","volume":"323","author":"Dhruva Sanket S.","year":"2020","unstructured":"Sanket S. Dhruva, Joseph S. Ross, Bobak J. Mortazavi, Nathan C. Hurley, Harlan M. Krumholz, Jeptha P. Curtis, Alyssa Berkowitz, Frederick A. Masoudi, John C. Messenger, Craig S. Parzynski et\u00a0al. 2020. Association of use of an intravascular microaxial left ventricular assist device vs intra-aortic balloon pump with in-hospital mortality and major bleeding among patients with acute myocardial infarction complicated by cardiogenic shock. J. Amer. Med. Assoc. 323, 8 (2020), 734\u2013745.","journal-title":"J. Amer. Med. Assoc."},{"issue":"8","key":"e_1_3_2_7_2","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1001\/jama.2020.0254","article-title":"Association of use of an intravascular microaxial left ventricular assist device vs intra-aortic balloon pump with in-hospital mortality and major bleeding among patients with acute myocardial infarction complicated by cardiogenic shock","volume":"323","author":"Dhruva Sanket S.","year":"2020","unstructured":"Sanket S. Dhruva, Joseph S. Ross, Bobak J. Mortazavi, Nathan C. Hurley, Harlan M. Krumholz, Jeptha P. Curtis, Alyssa Berkowitz, Frederick A. Masoudi, John C. Messenger, Craig S. Parzynski et\u00a0al. 2020. Association of use of an intravascular microaxial left ventricular assist device vs intra-aortic balloon pump with in-hospital mortality and major bleeding among patients with acute myocardial infarction complicated by cardiogenic shock. J. Amer. Med. Assoc. 323, 8 (2020), 734\u2013745.","journal-title":"J. Amer. Med. Assoc."},{"issue":"2","key":"e_1_3_2_8_2","first-page":"e2037748\u2013e20377","article-title":"Use of mechanical circulatory support devices among patients with acute myocardial infarction complicated by cardiogenic shock","volume":"4","author":"Dhruva Sanket S.","year":"2021","unstructured":"Sanket S. Dhruva, Joseph S. Ross, Bobak J. Mortazavi, Nathan C. Hurley, Harlan M. Krumholz, Jeptha P. Curtis, Alyssa P. Berkowitz, Frederick A. Masoudi, John C. Messenger, Craig S. Parzynski et\u00a0al. 2021. Use of mechanical circulatory support devices among patients with acute myocardial infarction complicated by cardiogenic shock. J. Amer. Med. Assoc. Netw. Open 4, 2 (2021), e2037748\u2013e2037748.","journal-title":"J. Amer. Med. Assoc. Netw. Open"},{"key":"e_1_3_2_9_2","first-page":"191","article-title":"A review of challenges and opportunities in machine learning for health","author":"Ghassemi Marzyeh","year":"2020","unstructured":"Marzyeh Ghassemi, Tristan Naumann, Peter Schulam, Andrew L. Beam, Irene Y. Chen, and Rajesh Ranganath. 2020. A review of challenges and opportunities in machine learning for health. In Proceedings of the AMIA Summits on Translational Science. 191.","journal-title":"Proceedings of the AMIA Summits on Translational Science"},{"issue":"4","key":"e_1_3_2_10_2","first-page":"1","article-title":"A survey of learning causality with data: Problems and methods","volume":"53","author":"Guo Ruocheng","year":"2020","unstructured":"Ruocheng Guo, Lu Cheng, Jundong Li, P. Richard Hahn, and Huan Liu. 2020. A survey of learning causality with data: Problems and methods. ACM Comput. Surveys 53, 4 (2020), 1\u201337.","journal-title":"ACM Comput. Surveys"},{"issue":"9","key":"e_1_3_2_11_2","doi-asserted-by":"crossref","first-page":"1016","DOI":"10.1200\/JCO.2015.64.4542","article-title":"Potential Simpson\u2019s paradox in multicenter study of intraperitoneal chemotherapy for ovarian cancer","volume":"34","author":"Holt Gerhard B.","year":"2016","unstructured":"Gerhard B. Holt. 2016. Potential Simpson\u2019s paradox in multicenter study of intraperitoneal chemotherapy for ovarian cancer. J. Clin. Oncol. 34, 9 (2016), 1016\u20131016.","journal-title":"J. Clin. Oncol."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/10.623058"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF01908075"},{"key":"e_1_3_2_14_2","first-page":"3894","volume-title":"Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AIStats\u201920)","author":"Huo Zepeng","year":"2020","unstructured":"Zepeng Huo, Arash PakBin, Xiaohan Chen, Nathan Hurley, Ye Yuan, Xiaoning Qian, Zhangyang Wang, Shuai Huang, and Bobak Mortazavi. 2020. Uncertainty quantification for deep context-aware mobile activity recognition and unknown context discovery. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AIStats\u201920). PMLR, 3894\u20133904."},{"key":"e_1_3_2_15_2","first-page":"101","volume-title":"Proceedings of the Machine Learning for Healthcare Conference","author":"Huo Zepeng","year":"2022","unstructured":"Zepeng Huo, Xiaoning Qian, Shuai Huang, Zhangyang Wang, and Bobak J. Mortazavi. 2022. Density-aware personalized training for risk prediction in imbalanced medical data. In Proceedings of the Machine Learning for Healthcare Conference. PMLR, 101\u2013122."},{"key":"e_1_3_2_16_2","first-page":"1","volume-title":"Proceedings of the IEEE EMBS International Conference on Biomedical and Health Informatics (BHI\u201921)","author":"Huo Zepeng","year":"2021","unstructured":"Zepeng Huo, Lida Zhang, Rohan Khera, Shuai Huang, Xiaoning Qian, Zhangyang Wang, and Bobak J. Mortazavi. 2021. Sparse gated mixture-of-experts to separate and interpret patient heterogeneity in ehr data. In Proceedings of the IEEE EMBS International Conference on Biomedical and Health Informatics (BHI\u201921). IEEE, 1\u20134."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/BHI50953.2021.9508568"},{"key":"e_1_3_2_18_2","article-title":"Visualization of emergency department clinical data for interpretable patient phenotyping","author":"Hurley Nathan C.","year":"2019","unstructured":"Nathan C. Hurley, Adrian D. Haimovich, R Andrew Taylor, and Bobak J. Mortazavi. 2019. Visualization of emergency department clinical data for interpretable patient phenotyping. Retrieved from https:\/\/arXiv:1907.11039","journal-title":"Retrieved from"},{"key":"e_1_3_2_19_2","unstructured":"Mohammad Tariqul Islam and Jason W. Fleischer. 2020. Distinguishing L and H phenotypes of COVID-19 using a single x-ray image. Retrieved from https:\/\/www.medrxiv.org\/content\/10.1101\/2020.04.27.20081984v1"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1991.3.1.79"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0066341"},{"key":"e_1_3_2_22_2","article-title":"Causal effect inference with deep latent-variable models","author":"Louizos Christos","year":"2017","unstructured":"Christos Louizos, Uri Shalit, Joris Mooij, David Sontag, Richard Zemel, and Max Welling. 2017. Causal effect inference with deep latent-variable models. Retrieved from https:\/\/arXiv:1705.08821","journal-title":"Retrieved from"},{"key":"e_1_3_2_23_2","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten Laurens van der","year":"2008","unstructured":"Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov.2008), 2579\u20132605.","journal-title":"J. Mach. Learn. Res."},{"issue":"5","key":"e_1_3_2_24_2","first-page":"121S\u2013126S","article-title":"COPD: epidemiology, prevalence, morbidity and mortality, and disease heterogeneity","volume":"121","author":"Mannino David M.","year":"2002","unstructured":"David M. Mannino. 2002. COPD: epidemiology, prevalence, morbidity and mortality, and disease heterogeneity. Chest 121, 5 (2002), 121S\u2013126S.","journal-title":"Chest"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-012-9338-y"},{"issue":"18","key":"e_1_3_2_26_2","doi-asserted-by":"crossref","first-page":"2833","DOI":"10.1161\/CIRCULATIONAHA105.570093","article-title":"Robustness of late lumen loss in discriminating drug-eluting stents across variable observational and randomized trials","volume":"112","author":"Mauri Laura","year":"2005","unstructured":"Laura Mauri, E. John Orav, Susana C. Candia, Donald E. Cutlip, and Richard E. Kuntz. 2005. Robustness of late lumen loss in discriminating drug-eluting stents across variable observational and randomized trials. Circulation 112, 18 (2005), 2833\u20132839.","journal-title":"Circulation"},{"issue":"13","key":"e_1_3_2_27_2","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1056\/NEJMoa0801485","article-title":"Drug-eluting or bare-metal stents for acute myocardial infarction","volume":"359","author":"Mauri Laura","year":"2008","unstructured":"Laura Mauri, Treacy S. Silbaugh, Pallav Garg, Robert E. Wolf, Katya Zelevinsky, Ann Lovett, Manu R. Varma, Zheng Zhou, and Sharon-Lise T. Normand. 2008. Drug-eluting or bare-metal stents for acute myocardial infarction. New Engl. J. Med. 359, 13 (2008), 1330\u20131342.","journal-title":"New Engl. J. Med."},{"key":"e_1_3_2_28_2","article-title":"Umap: Uniform manifold approximation and projection for dimension reduction","author":"McInnes Leland","year":"2018","unstructured":"Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. Retrieved from https:\/\/arXiv:1802.03426","journal-title":"Retrieved from"},{"issue":"16","key":"e_1_3_2_29_2","doi-asserted-by":"crossref","first-page":"1484","DOI":"10.1016\/j.jacc.2012.07.020","article-title":"The national cardiovascular data registry (NCDR) data quality brief: The NCDR data quality program in 2012","volume":"60","author":"Messenger John C.","year":"2012","unstructured":"John C. Messenger, Kalon K. L. Ho, Christopher H. Young, Lara E. Slattery, Jasmine C. Draoui, Jeptha P. Curtis, Gregory J. Dehmer, Frederick L. Grover, Michael J. Mirro, Matthew R. Reynolds et\u00a0al. 2012. The national cardiovascular data registry (NCDR) data quality brief: The NCDR data quality program in 2012. J. Amer. Coll. Cardiol. 60, 16 (2012), 1484\u20131488.","journal-title":"J. Amer. Coll. Cardiol."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2017.2675340"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368555.3384456"},{"key":"e_1_3_2_32_2","first-page":"774","article-title":"Evaluation of adaptive mixtures of competing experts","volume":"3","author":"Nowlan Steven","year":"1990","unstructured":"Steven Nowlan and Geoffrey E. Hinton. 1990. Evaluation of adaptive mixtures of competing experts. Adv. Neural Info. Process. Syst. 3 (1990), 774\u2013780.","journal-title":"Adv. Neural Info. Process. Syst."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aax2342"},{"issue":"1","key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1186\/s12911-022-01879-6","article-title":"Learning patient-level prediction models across multiple healthcare databases: Evaluation of ensembles for increasing model transportability","volume":"22","author":"Reps Jenna Marie","year":"2022","unstructured":"Jenna Marie Reps, Ross D. Williams, Martijn J. Schuemie, Patrick B. Ryan, and Peter R. Rijnbeek. 2022. Learning patient-level prediction models across multiple healthcare databases: Evaluation of ensembles for increasing model transportability. BMC Med. Info. Decis. Mak. 22, 1 (2022), 142.","journal-title":"BMC Med. Info. Decis. Mak."},{"key":"e_1_3_2_35_2","first-page":"4846","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"33","author":"Schwab Patrick","year":"2019","unstructured":"Patrick Schwab, Djordje Miladinovic, and Walter Karlen. 2019. Granger-causal attentive mixtures of experts: Learning important features with neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 4846\u20134853."},{"key":"e_1_3_2_36_2","volume-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR\u201917)","author":"Shazeer Noam","year":"2017","unstructured":"Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely gated mixture-of-experts layer. In Proceedings of the 5th International Conference on Learning Representations (ICLR\u201917). OpenReview.net."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3302505.3310087"},{"issue":"10","key":"e_1_3_2_38_2","first-page":"e006556","article-title":"Recommendations for reporting machine learning analyses in clinical research","volume":"13","author":"Stevens Laura M.","year":"2020","unstructured":"Laura M. Stevens, Bobak J. Mortazavi, Rahul C. Deo, Lesley Curtis, and David P. Kao. 2020. Recommendations for reporting machine learning analyses in clinical research. Circulation: Cardiovasc. Qual. Outcomes 13, 10 (2020), e006556.","journal-title":"Circulation: Cardiovasc. Qual. Outcomes"},{"key":"e_1_3_2_39_2","first-page":"552","volume-title":"Proceedings of the 35th Uncertainty in Artificial Intelligence Conference","author":"Wang Xin","year":"2020","unstructured":"Xin Wang, Fisher Yu, Lisa Dunlap, Yi-An Ma, Ruth Wang, Azalia Mirhoseini, Trevor Darrell, and Joseph E. Gonzalez. 2020. Deep mixture of experts via shallow embedding. In Proceedings of the 35th Uncertainty in Artificial Intelligence Conference. PMLR, 552\u2013562."},{"key":"e_1_3_2_40_2","article-title":"Patient risk stratification for hospital-associated c. diff as a time-series classification task","volume":"25","author":"Wiens Jenna","year":"2012","unstructured":"Jenna Wiens, Eric Horvitz, and John Guttag. 2012. Patient risk stratification for hospital-associated c. diff as a time-series classification task. Adv. Neural Info. Process. Syst. 25 (2012).","journal-title":"Adv. Neural Info. Process. Syst."},{"key":"e_1_3_2_41_2","unstructured":"Liuyi Yao Zhixuan Chu Sheng Li Yaliang Li Jing Gao and Aidong Zhang. 2020. A survey on causal inference. Retrieved from https:\/\/arXiv:2002.02770"},{"key":"e_1_3_2_42_2","article-title":"Representation learning for treatment effect estimation from observational data","volume":"31","author":"Yao Liuyi","year":"2018","unstructured":"Liuyi Yao, Sheng Li, Yaliang Li, Mengdi Huai, Jing Gao, and Aidong Zhang. 2018. Representation learning for treatment effect estimation from observational data. Adv. Neural Info. Process. Syst. 31 (2018).","journal-title":"Adv. Neural Info. Process. Syst."}],"container-title":["ACM Transactions on Computing for Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3616021","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3616021","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T12:36:30Z","timestamp":1750163790000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3616021"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,13]]},"references-count":41,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,10,31]]}},"alternative-id":["10.1145\/3616021"],"URL":"https:\/\/doi.org\/10.1145\/3616021","relation":{},"ISSN":["2691-1957","2637-8051"],"issn-type":[{"value":"2691-1957","type":"print"},{"value":"2637-8051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,13]]},"assertion":[{"value":"2022-05-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-02","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}