{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T01:15:08Z","timestamp":1792804508086,"version":"4.5.0"},"reference-count":81,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2023,10,4]],"date-time":"2023-10-04T00:00:00Z","timestamp":1696377600000},"content-version":"vor","delay-in-days":6,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["1840088"],"award-info":[{"award-number":["1840088"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2023,9,28]]},"abstract":"<jats:p>Human cognitive and decision-making abilities depreciate under pressure, motivating the emergence of artificial intelligence (AI) systems as decision support tools to assist people in performing tasks under stress. In this work, we study human decision-making behavior and task performance under time pressure---induced from limitedinitial observation time (time to perform the task before providing an initial response without AI input) andfinal decision time (time to weigh an AI's suggestion before reaching a collective human-AI team answer)---for spatial reasoning and count estimation tasks. Our results show that, while the impact of initial observation time on AI-assisted decision-making was dependent on task nature, participants were more likely to follow AI suggestions when they were provided with longer final decision time; moreover, although participants generally tended to adhere to their initial responses, they had more agency when they were more logically engaged in a task. Our results offer a nuanced understanding of human-AI collaboration under time pressure in different phases of the decision-making process.<\/jats:p>","DOI":"10.1145\/3610068","type":"journal-article","created":{"date-parts":[[2023,10,4]],"date-time":"2023-10-04T15:54:10Z","timestamp":1696434850000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":52,"title":["How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4344-5870","authenticated-orcid":false,"given":"Shiye","family":"Cao","sequence":"first","affiliation":[{"name":"Johns Hopkins University, Baltimore, MD, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1536-3421","authenticated-orcid":false,"given":"Catalina","family":"Gomez","sequence":"additional","affiliation":[{"name":"Johns Hopkins University, Baltimore, MD, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6838-3701","authenticated-orcid":false,"given":"Chien-Ming","family":"Huang","sequence":"additional","affiliation":[{"name":"Johns Hopkins University, Baltimore, MD, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,10,4]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.212631"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300233"},{"key":"e_1_2_1_3_1","volume-title":"Counting in the Wild. In European Conference on Computer Vision.","author":"Arteta Carlos","year":"2016","unstructured":"Carlos Arteta and Andrew Lempitsky, Victor andZisserman. 2016. Counting in the Wild. In European Conference on Computer Vision."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v7i1.5285"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33012429"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445717"},{"key":"e_1_2_1_7_1","volume-title":"Workload for radiologists during on-call hours: dramatic increase in the past 15 years. Insights into imaging 11, 1","author":"Bruls RJM","year":"2020","unstructured":"RJM Bruls and RM Kwee. 2020. Workload for radiologists during on-call hours: dramatic increase in the past 15 years. Insights into imaging 11, 1 (2020), 1--7."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3449287"},{"key":"e_1_2_1_9_1","volume-title":"David Keith Williams, and David W Hosmer","author":"Bursac Zoran","year":"2008","unstructured":"Zoran Bursac, C Heath Gauss, David Keith Williams, and David W Hosmer. 2008. Purposeful selection of variables in logistic regression. Source code for biology and medicine 3, 1 (2008), 1--8."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555572"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300789"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447535.3462487"},{"key":"e_1_2_1_13_1","unstructured":"Plainsight Corp. 2021. Use Vision AI for Accurate Livestock Monitoring at Scale. https:\/\/plainsight.ai\/use-vision-ai-for-accurate-livestock-monitoring-at-scale\/"},{"key":"e_1_2_1_14_1","volume-title":"Artificial intelligence: A critical review of applications for lung nodule and lung cancer. Diagnostic and Interventional Imaging","author":"de Margerie-Mellon Constance","year":"2022","unstructured":"Constance de Margerie-Mellon and Guillaume Chassagnon. 2022. Artificial intelligence: A critical review of applications for lung nodule and lung cancer. Diagnostic and Interventional Imaging (2022)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2044-8317.1992.tb00992.x"},{"key":"e_1_2_1_16_1","volume-title":"Spatial abilities of expert clinical anatomists: Comparison of abilities between novices, intermediates, and experts in anatomy. Anatomical sciences education 4, 1","author":"Fernandez Ruth","year":"2011","unstructured":"Ruth Fernandez, Itiel E Dror, and Claire Smith. 2011. Spatial abilities of expert clinical anatomists: Comparison of abilities between novices, intermediates, and experts in anatomy. Anatomical sciences education 4, 1 (2011), 1--8."},{"key":"e_1_2_1_17_1","unstructured":"Howard E Gardner. 2011. Frames of mind: The theory of multiple intelligences. Hachette Uk."},{"key":"e_1_2_1_18_1","volume-title":"Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine 4, 1","author":"Gaube Susanne","year":"2021","unstructured":"Susanne Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J Berkowitz, Eva Lermer, Joseph F Coughlin, John V Guttag, Errol Colak, and Marzyeh Ghassemi. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine 4, 1 (2021), 1--8."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1093\/oxfordhb\/9780198795872.013.22"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2022.102977"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.3200\/GENP.131.1.18-28"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1053\/crad.1999.0379"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1061\/(ASCE)EI.1943-5541.0000266"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1037\/0096-3445.108.3.356"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.lindif.2008.04.006"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1046\/j.1365-2664.2002.00752.x"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445385"},{"key":"e_1_2_1_28_1","volume-title":"How machine-learning recommendations influence clinician treatment selections: the example of antidepressant selection. Translational psychiatry 11, 1","author":"Jacobs Maia","year":"2021","unstructured":"Maia Jacobs, Melanie F Pradier, Thomas H McCoy, Roy H Perlis, Finale Doshi-Velez, and Krzysztof Z Gajos. 2021. How machine-learning recommendations influence clinician treatment selections: the example of antidepressant selection. Translational psychiatry 11, 1 (2021), 1--9."},{"key":"e_1_2_1_29_1","volume-title":"Janis and Leon Mann","author":"Irving","year":"1977","unstructured":"Irving L. Janis and Leon Mann. 1977. Decision Making: A Psychological Analysis of Conflict, Choice, and Commitment. The ANNALS of the American Academy of Political and Social Science 488 (1977)."},{"key":"e_1_2_1_30_1","volume-title":"Attention and effort","author":"Kahneman Daniel","year":"1973","unstructured":"Daniel Kahneman. 1973. Attention and effort. Prentice-Hall series in experimental psychology (1973)."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0887-6185(99)00025-0"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11166-017-9252-4"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jebo.2004.11.013"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2015.06.059"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0933--3657(01)00077-X"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287590"},{"key":"e_1_2_1_37_1","volume-title":"Trust in automation: Designing for appropriate reliance. Human factors 46, 1","author":"Lee John D","year":"2004","unstructured":"John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 (2004), 50--80."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445522"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2016.12.065"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1210\/jcem.82.7.4075"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0166--4328(01)00361--8"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","unstructured":"Mather M. and Lighthall N. R. 2012. Both Risk and Reward are Processed Differently in Decisions Made Under Stress. Current directions in psychological science 21 2 (2012) 36--41. https:\/\/doi.org\/10.1177\/0963721411429452","DOI":"10.1177\/0963721411429452"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.11648\/j.pbs.20160504.15"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.joep.2015.12.002"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v8i1.7469"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1006\/obhd.1997.2717"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359204"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1080\/0144929X.2016.1209242"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.psyneuen.2016.01.007"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.2981\/0909-6396(2007)13[321:ASVHST]2.0.CO;2"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1080\/12460125.2020.1768680"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.biopsych.2007.04.041"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3512930"},{"key":"e_1_2_1_54_1","volume-title":"Automation Dependency Under Time Pressure. In 2009 International Symposium on Aviation Psychology. 611","author":"Rice Stephen","year":"2009","unstructured":"Stephen Rice, David Keller, Gayle Hunt, and David Trafimow. 2009. Automation Dependency Under Time Pressure. In 2009 International Symposium on Aviation Psychology. 611."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020--17419--7"},{"key":"e_1_2_1_56_1","volume-title":"Understanding the impact of time pressure and automation support in a visual search task. Human Factors","author":"Rieger Tobias","year":"2022","unstructured":"Tobias Rieger and Dietrich Manzey. 2022. Understanding the impact of time pressure and automation support in a visual search task. Human Factors (2022), 00187208221111236."},{"key":"e_1_2_1_57_1","volume-title":"Effects of imperfect automation on decision making in a simulated command and control task. Human factors 49, 1","author":"Rovira Ericka","year":"2007","unstructured":"Ericka Rovira, Kathleen McGarry, and Raja Parasuraman. 2007. Effects of imperfect automation on decision making in a simulated command and control task. Human factors 49, 1 (2007), 76--87."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2015.01.007"},{"key":"e_1_2_1_59_1","volume-title":"Supporting decision making and action selection under time pressure and uncertainty: The case of in-flight icing. Human factors 43, 4","author":"Sarter Nadine B","year":"2001","unstructured":"Nadine B Sarter and Beth Schroeder. 2001. Supporting decision making and action selection under time pressure and uncertainty: The case of in-flight icing. Human factors 43, 4 (2001), 573--583."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302308"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1038\/ejhg.2010.131"},{"key":"e_1_2_1_62_1","volume-title":"AI in human teams: effects on technology use, members' interactions, and creative performance under time scarcity. AI & SOCIETY","author":"Shaikh Sonia Jawaid","year":"2022","unstructured":"Sonia Jawaid Shaikh and Ignacio F Cruz. 2022. AI in human teams: effects on technology use, members' interactions, and creative performance under time scarcity. AI & SOCIETY (2022), 1--14."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1037\/0882--7974.21.3.545"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376624"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","unstructured":"Robert S Stawski et al. 2006. Stress-related cognitive interference predicts cognitive function in old age. Psychology and aging 21 3 (2006) 535--544. https:\/\/doi.org\/10.1037\/0882--7974.21.3.535","DOI":"10.1037\/0882--7974.21.3.535"},{"key":"e_1_2_1_66_1","volume-title":"Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making. In 12th ACM Conference on Web Science. 315--324","author":"Suresh Harini","year":"2020","unstructured":"Harini Suresh, Natalie Lao, and Ilaria Liccardi. 2020. Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making. In 12th ACM Conference on Web Science. 315--324."},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467--9450.1987.tb00769.x"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1177\/1541931214581218"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuropsychologia.2016.12.018"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1080\/03610910701723625"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397481.3450650"},{"key":"e_1_2_1_72_1","volume-title":"Do female roe deer in good condition produce more sons than daughters. Behavioral ecology and sociobiology 37, 3","author":"Wauters Luc A","year":"1995","unstructured":"Luc A Wauters, Simon A de Crombrugghe, Nadia Nour, and Erik Matthysen. 1995. Do female roe deer in good condition produce more sons than daughters. Behavioral ecology and sociobiology 37, 3 (1995), 189--193."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0037186"},{"key":"e_1_2_1_74_1","volume-title":"Practical deep reinforcement learning approach for stock trading. arXiv preprint arXiv:1811.07522","author":"Xiong Zhuoran","year":"2018","unstructured":"Zhuoran Xiong, Xiao-Yang Liu, Shan Zhong, Hongyang Yang, and Anwar Walid. 2018. Practical deep reinforcement learning approach for stock trading. arXiv preprint arXiv:1811.07522 (2018)."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0003-2670(01)01271-5"},{"key":"e_1_2_1_76_1","doi-asserted-by":"crossref","unstructured":"Huiqin Yang Carl Thompson and Martin Bland. 2012. The effect of clinical experience judgment task difficulty and time pressure on nurses' confidence calibration in a high fidelity clinical simulation. BMC medical informatics and decision making 12 1 (2012) 1--9.","DOI":"10.1186\/1472-6947-12-113"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376301"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300509"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1081\/SAC-200033363"},{"key":"e_1_2_1_80_1","volume-title":"You Complete Me: Human-AI Teams and Complementary Expertise. In CHI Conference on Human Factors in Computing Systems. 1--28","author":"Zhang Qiaoning","year":"2022","unstructured":"Qiaoning Zhang, Matthew L Lee, and Scott Carter. 2022. You Complete Me: Human-AI Teams and Complementary Expertise. In CHI Conference on Human Factors in Computing Systems. 1--28."},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372852"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610068","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3610068","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3610068","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T04:28:00Z","timestamp":1755750480000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610068"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,28]]},"references-count":81,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2023,9,28]]}},"alternative-id":["10.1145\/3610068"],"URL":"https:\/\/doi.org\/10.1145\/3610068","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,28]]},"assertion":[{"value":"2023-10-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}