{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,3]],"date-time":"2026-08-03T22:42:20Z","timestamp":1785796940688,"version":"3.56.0"},"reference-count":83,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T00:00:00Z","timestamp":1760572800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01LM011834-05"],"award-info":[{"award-number":["R01LM011834-05"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IS-2107391"],"award-info":[{"award-number":["IS-2107391"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Science Foundation Graduation Research Fellowship","award":["204177"],"award-info":[{"award-number":["204177"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,10,18]]},"abstract":"<jats:p>AI-enabled decision-support systems aim to help medical providers rapidly make decisions with limited information during medical emergencies. A critical challenge in developing these systems is supporting providers in interpreting the system output to make optimal treatment decisions. In this study, we designed and evaluated an AI-enabled decision-support system to aid providers in treating patients with traumatic injuries. We first conducted user research with physicians to identify and design information types and AI outputs for a decision-support display. We then conducted an online experiment with 35 medical providers from six health systems to evaluate two human-AI interaction strategies: (1) AI information synthesis and (2) AI information and recommendations. We found that providers were more likely to make correct decisions when AI information and recommendations were provided compared to receiving no AI support. We also identified two socio-technical barriers to providing AI recommendations during time-critical medical events: (1) an accuracy-time trade-off in providing recommendations and (2) polarizing perceptions of recommendations between providers. We discuss three implications for developing AI-enabled decision support used in time-critical events, contributing to the limited research on human-AI interaction in this context.<\/jats:p>","DOI":"10.1145\/3757512","type":"journal-article","created":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T16:59:10Z","timestamp":1760633950000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8179-0101","authenticated-orcid":false,"given":"Angela","family":"Mastrianni","sequence":"first","affiliation":[{"name":"Drexel University, Philadelphia, Pennsylvania, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-7025-6636","authenticated-orcid":false,"given":"Mary Suhyun","family":"Kim","sequence":"additional","affiliation":[{"name":"Children's National Hospital, Washington D.C., District of Columbia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4399-7037","authenticated-orcid":false,"given":"Travis M","family":"Sullivan","sequence":"additional","affiliation":[{"name":"Children's National Hospital, Washington, District of Columbia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7952-766X","authenticated-orcid":false,"given":"Genevieve Jayne","family":"Sippel","sequence":"additional","affiliation":[{"name":"Children's National Hospital, Washington, District of Columbia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4465-9117","authenticated-orcid":false,"given":"Randall S","family":"Burd","sequence":"additional","affiliation":[{"name":"Children's National Hospital, Washington, District of Columbia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1897-9048","authenticated-orcid":false,"given":"Krzysztof Z.","family":"Gajos","sequence":"additional","affiliation":[{"name":"Harvard University, Allston, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2126-8527","authenticated-orcid":false,"given":"Aleksandra","family":"Sarcevic","sequence":"additional","affiliation":[{"name":"Drexel University, Philadelphia, Pennsylvania, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,10,16]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.3390\/app11115088"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581513"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445717"},{"key":"e_1_2_1_4_1","volume-title":"Henrik Singmann, Bin Dai, Gabor Grothendieck, Peter Green, and M Ben Bolker.","author":"Bates Douglas","year":"2015","unstructured":"Douglas Bates, Martin Maechler, Ben Bolker, Steven Walker, Rune Haubo Bojesen Christensen, Henrik Singmann, Bin Dai, Gabor Grothendieck, Peter Green, and M Ben Bolker. 2015. Package 'lme4'. convergence, Vol. 12, 1 (2015), 2."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3563657.3596110"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1177\/2515245920960351"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377325.3377498"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3449287"},{"key":"e_1_2_1_9_1","volume-title":"Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning. arXiv preprint arXiv:2403.05911","author":"Bu\u00e7inca Zana","year":"2024","unstructured":"Zana Bu\u00e7inca, Siddharth Swaroop, Amanda E Paluch, Susan A Murphy, and Krzysztof Z Gajos. 2024. Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning. arXiv preprint arXiv:2403.05911 (2024)."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581251"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICHI.2015.26"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300234"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359206"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610068"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1553-2712.2000.tb00480.x"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642551"},{"key":"e_1_2_1_17_1","volume-title":"Achieving quality in clinical decision making: cognitive strategies and detection of bias","author":"Croskerry Pat","year":"2002","unstructured":"Pat Croskerry. 2002. Achieving quality in clinical decision making: cognitive strategies and detection of bias. Academic emergency medicine, Vol. 9, 11 (2002), 1184-1204."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1097\/00001888-200308000-00003"},{"key":"e_1_2_1_19_1","volume-title":"Cognitive debiasing 2: impediments to and strategies for change. BMJ quality & safety","author":"Croskerry Pat","year":"2013","unstructured":"Pat Croskerry, Geeta Singhal, and S\u00edlvia Mamede. 2013. Cognitive debiasing 2: impediments to and strategies for change. BMJ quality & safety, Vol. 22, Suppl 2 (2013), ii65-ii72."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1177\/0266382115592536"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580672"},{"key":"e_1_2_1_22_1","volume-title":"Believing in overcoming cognitive biases. AMA journal of ethics","author":"Doherty Tiffany S","year":"2020","unstructured":"Tiffany S Doherty and Aaron E Carroll. 2020. Believing in overcoming cognitive biases. AMA journal of ethics, Vol. 22, 9 (2020), 773-778."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1097\/SLA.0000000000002999"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.4236\/jilsa.2017.91001"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533193"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.5811\/westjem.2016.7.29812"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3490099.3511138"},{"key":"e_1_2_1_28_1","volume-title":"Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine","author":"Gaube Susanne","year":"2021","unstructured":"Susanne Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J Berkowitz, Eva Lermer, Joseph F Coughlin, John V Guttag, Errol Colak, and Marzyeh Ghassemi. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine, Vol. 4, 1 (2021), 31."},{"key":"e_1_2_1_29_1","volume-title":"Melissa Bristow, Dustin R Long, Konstantina Palla, Bala Nair, Christine Fong, Ronald Pauldine, et al.","author":"Ghosh Pratik","year":"2023","unstructured":"Pratik Ghosh, Karen L Posner, Stephanie L Hyland, Wil Van Cleve, Melissa Bristow, Dustin R Long, Konstantina Palla, Bala Nair, Christine Fong, Ronald Pauldine, et al., 2023. Framing Machine Learning Opportunities for Hypotension Prediction in Perioperative Care: A Socio-Technical Perspective. ACM Transactions on Computer-Human Interaction (2023)."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.clsr.2022.105681"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580694"},{"key":"e_1_2_1_32_1","volume-title":"XAI-Explainable artificial intelligence. Science robotics","author":"Gunning David","year":"2019","unstructured":"David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang. 2019. XAI-Explainable artificial intelligence. Science robotics, Vol. 4, 37 (2019), eaay7120."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1186\/cc3780"},{"key":"e_1_2_1_34_1","volume-title":"A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics","author":"Holm Sture","year":"1979","unstructured":"Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics (1979), 65-70."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445385"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3357236.3395436"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517439"},{"key":"e_1_2_1_38_1","volume-title":"A recognition-primed decision (RPD) model of rapid decision making. Decision making in action: Models and methods","author":"Klein Gary A","year":"1993","unstructured":"Gary A Klein. 1993. A recognition-primed decision (RPD) model of rapid decision making. Decision making in action: Models and methods, Vol. 5, 4 (1993), 138-147."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3357236.3395552"},{"key":"e_1_2_1_40_1","volume-title":"Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ quality & safety","author":"Lambe Kathryn Ann","year":"2016","unstructured":"Kathryn Ann Lambe, Gary O'Reilly, Brendan D Kelly, and Sarah Curristan. 2016. Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ quality & safety, Vol. 25, 10 (2016), 808-820."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610218"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445472"},{"key":"e_1_2_1_43_1","unstructured":"Russell V. Lenth. 2024. emmeans: Estimated Marginal Means aka Least-Squares Means. https:\/\/rvlenth.github.io\/emmeans\/ R package version 1.10.2 https:\/\/rvlenth.github.io\/emmeans\/."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3492828"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3593013.3594001"},{"key":"e_1_2_1_46_1","volume-title":"Designing interactions","author":"Moggridge Bill","unstructured":"Bill Moggridge and Bill Atkinson. 2007. Designing interactions. Vol. 17. MIT press Cambridge."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.socscimed.2020.113172"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3641022"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2018.6971"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1002\/bjs.9096"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3502104"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359204"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3512930"},{"key":"e_1_2_1_54_1","volume-title":"When do people use simple heuristics, and how can we tell?","author":"Rieskamp J\u00f6rg","year":"1999","unstructured":"J\u00f6rg Rieskamp and Ulrich Hoffrage. 1999. When do people use simple heuristics, and how can we tell? (1999)."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3649404"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2001.0080527"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2007.09.003"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581075"},{"key":"e_1_2_1_59_1","volume-title":"International ATLS Working Group, et al","author":"Subcommittee ATLS","year":"2013","unstructured":"ATLS Subcommittee, International ATLS Working Group, et al., 2013. Advanced trauma life support (ATLS\u00ae): the ninth edition. The journal of trauma and acute care surgery, Vol. 74, 5 (2013), 1363-1366."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1097\/TA.0000000000003709"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1097\/TA.0000000000003935"},{"key":"e_1_2_1_62_1","volume-title":"An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ digital medicine","author":"Sutton Reed T","year":"2020","unstructured":"Reed T Sutton, David Pincock, Daniel C Baumgart, Daniel C Sadowski, Richard N Fedorak, and Karen I Kroeker. 2020. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ digital medicine, Vol. 3, 1 (2020), 17."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3640543.3645206"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1097\/TA.0b013e31815078ae"},{"key":"e_1_2_1_65_1","volume-title":"Maria Teodora Wetscherek, Cecily Morrison, et al.","author":"Thieme Anja","year":"2024","unstructured":"Anja Thieme, Abhijith Rajamohan, Benjamin Cooper, Heather Groombridge, Robert Simister, Barney Wong, Nicholas Woznitza, Mark Ames Pinnock, Maria Teodora Wetscherek, Cecily Morrison, et al., 2024. Challenges for Responsible AI Design and Workflow Integration in Healthcare: A Case Study of Automatic Feeding Tube Qualification in Radiology. arXiv preprint arXiv:2405.05299 (2024)."},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/1882992.1883101"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3571815"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3579605"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.1276"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445432"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300831"},{"key":"e_1_2_1_72_1","volume-title":"Intelligent decision support in process environments","author":"Woods David D","unstructured":"David D Woods. 1986. Paradigms for intelligent decision support. In Intelligent decision support in process environments. Springer, 153-173."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376807"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581393"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300468"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858373"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642013"},{"key":"e_1_2_1_78_1","volume-title":"Heterogeneity and predictors of the effects of AI assistance on radiologists. Nature Medicine","author":"Yu Feiyang","year":"2024","unstructured":"Feiyang Yu, Alex Moehring, Oishi Banerjee, Tobias Salz, Nikhil Agarwal, and Pranav Rajpurkar. 2024. Heterogeneity and predictors of the effects of AI assistance on radiologists. Nature Medicine (2024), 1-13."},{"key":"e_1_2_1_79_1","volume-title":"Andrew L Beam, and Isaac S Kohane","author":"Yu Kun-Hsing","year":"2018","unstructured":"Kun-Hsing Yu, Andrew L Beam, and Isaac S Kohane. 2018. Artificial intelligence in healthcare. Nature biomedical engineering, Vol. 2, 10 (2018), 719-731."},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642195"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501887"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642343"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/3687024"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3757512","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3757512","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T01:56:48Z","timestamp":1760666208000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3757512"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,16]]},"references-count":83,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,10,18]]}},"alternative-id":["10.1145\/3757512"],"URL":"https:\/\/doi.org\/10.1145\/3757512","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,16]]},"assertion":[{"value":"2025-10-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}