{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T10:13:13Z","timestamp":1780567993704,"version":"3.54.1"},"reference-count":37,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW","license":[{"start":{"date-parts":[[2019,11,7]],"date-time":"2019-11-07T00:00:00Z","timestamp":1573084800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["NSF 1564041"],"award-info":[{"award-number":["NSF 1564041"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2019,11,7]]},"abstract":"<jats:p>With computational algorithms making an increasing number of deeply consequential, and often problematic judgments on our behalf, there is a growing interest in slowing down technology to encourage users to reflect on judgments made by algorithms. Prior work in slow technology has established slowness as an agent of reflection and serendipity; however, it has been unclear whether this waiting time actually helps users gain useful insight or any other benefits as they make judgments using an algorithm. To this end, we conducted a series of online and in-person between-subject user studies in which we isolate the impact of an algorithm's speed on how users incorporate the algorithm's advice when making judgments in the context of simple visual recognition tasks. We find that our participants followed good quality algorithms more and bad quality algorithms somewhat less if the response time of the algorithm is slower. Furthermore, qualitative analysis of the in-person study interviews reveals that the waiting was not time wasted, but was often used to reflect on the task and the estimation process of themselves and the algorithm, and to compare and reevaluate the two processes. Based on these findings, we outline design implications of future algorithmic systems.<\/jats:p>","DOI":"10.1145\/3359204","type":"journal-article","created":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T13:40:21Z","timestamp":1573220421000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":60,"title":["A Slow Algorithm Improves Users' Assessments of the Algorithm's Accuracy"],"prefix":"10.1145","volume":"3","author":[{"given":"Joon Sung","family":"Park","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rick","family":"Barber","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alex","family":"Kirlik","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karrie","family":"Karahalios","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,11,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1640233.1640279"},{"key":"e_1_2_1_2_1","volume-title":"Machine bias. ProPublica (May","author":"Angwin Julia","year":"2016","unstructured":"Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016. Machine bias. ProPublica (May 2016 ). https: \/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica (May 2016). https: \/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/182.358464"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300820"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(81)80033-8"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1037\/xge0000033"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of CHI 2013 Workshop on Changing Perspectives of Time in HCI (April","author":"Doark Marian","year":"2013","unstructured":"Marian Doark , Peter Bennett , and Rosamund Davies . 2013 . Taking our sweet time to search . Proceedings of CHI 2013 Workshop on Changing Perspectives of Time in HCI (April 2013). Marian Doark, Peter Bennett, and Rosamund Davies. 2013. Taking our sweet time to search. Proceedings of CHI 2013 Workshop on Changing Perspectives of Time in HCI (April 2013)."},{"key":"e_1_2_1_8_1","volume-title":"Towards a rigorous science of interpretable machine learning. CoRR","author":"Doshi-Velez Finale","year":"2017","unstructured":"Finale Doshi-Velez and Been Kim . 2017. Towards a rigorous science of interpretable machine learning. CoRR ( 2017 ). Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. CoRR (2017)."},{"key":"e_1_2_1_9_1","volume-title":"The real reason tech struggles with algorithmic bias. Wired (February","author":"Eisenstat Yael","year":"2019","unstructured":"Yael Eisenstat . 2019. The real reason tech struggles with algorithmic bias. Wired (February 2019 ). https:\/\/www.wired. com\/story\/the-real-reason-tech-struggles-with-algorithmic-bias\/ Yael Eisenstat. 2019. The real reason tech struggles with algorithmic bias. Wired (February 2019). https:\/\/www.wired. com\/story\/the-real-reason-tech-struggles-with-algorithmic-bias\/"},{"key":"e_1_2_1_10_1","volume-title":"The impact of life coaching on goal attainment, metacognition and mental health. Social Behavior and Personality: an international journal 31, 3","author":"Grant Anthony M","year":"2003","unstructured":"Anthony M Grant . 2003. The impact of life coaching on goal attainment, metacognition and mental health. Social Behavior and Personality: an international journal 31, 3 ( 2003 ), 253--263. Anthony M Grant. 2003. The impact of life coaching on goal attainment, metacognition and mental health. Social Behavior and Personality: an international journal 31, 3 (2003), 253--263."},{"key":"e_1_2_1_11_1","volume-title":"Facebook will put faster loading sites top of news feeds. Small Business Trends (August","author":"Guta Michael","year":"2017","unstructured":"Michael Guta . 2017. Facebook will put faster loading sites top of news feeds. Small Business Trends (August 2017 ). https:\/\/smallbiztrends.com\/2017\/08\/facebook-news-feed-will-favor-faster-sites.html Michael Guta. 2017. Facebook will put faster loading sites top of news feeds. Small Business Trends (August 2017). https:\/\/smallbiztrends.com\/2017\/08\/facebook-news-feed-will-favor-faster-sites.html"},{"key":"e_1_2_1_12_1","volume-title":"Slow technology--designing for reflection. Personal and ubiquitous computing 5, 3","author":"Hallna's Lars","year":"2001","unstructured":"Lars Hallna's and Johan Redstrom . 2001. Slow technology--designing for reflection. Personal and ubiquitous computing 5, 3 ( 2001 ), 201--212. Lars Hallna's and Johan Redstrom. 2001. Slow technology--designing for reflection. Personal and ubiquitous computing 5, 3 (2001), 201--212."},{"key":"e_1_2_1_13_1","volume-title":"The role of discussion in changing opinion regarding a matter of fact. The Journal of Abnormal and Social Psychology","author":"Jenness Arthur","year":"1932","unstructured":"Arthur Jenness . 1932. The role of discussion in changing opinion regarding a matter of fact. The Journal of Abnormal and Social Psychology ( 1932 ). Arthur Jenness. 1932. The role of discussion in changing opinion regarding a matter of fact. The Journal of Abnormal and Social Psychology (1932)."},{"key":"e_1_2_1_14_1","volume-title":"Is it time for a 'slow food' movement for the internet? Retrieved","author":"Johnson Eric","year":"2019","unstructured":"Eric Johnson . 2018. Is it time for a 'slow food' movement for the internet? Retrieved April 3, 2019 from https:\/\/www.recode.net\/2018\/9\/12\/17848368\/nicole-wong-cto-google-twitter-slow-food-tech-internet-congress- regulation-kara-swisher-podcast Eric Johnson. 2018. Is it time for a 'slow food' movement for the internet? Retrieved April 3, 2019 from https:\/\/www.recode.net\/2018\/9\/12\/17848368\/nicole-wong-cto-google-twitter-slow-food-tech-internet-congress- regulation-kara-swisher-podcast"},{"key":"e_1_2_1_15_1","volume-title":"Smart Interfaces for Human-Centered AI. HAI","author":"Landay James","year":"2019","unstructured":"James Landay . 2019. Smart Interfaces for Human-Centered AI. HAI ( 2019 ), March. James Landay. 2019. Smart Interfaces for Human-Centered AI. HAI (2019), March."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951718756684"},{"key":"e_1_2_1_17_1","volume-title":"Pedro Ferreira, and Vygandas Simbelis.","author":"Lindley Sian","year":"2013","unstructured":"Sian Lindley , Robert Corish , Elsa Kosmack Vaara , Pedro Ferreira, and Vygandas Simbelis. 2013 . Changing perspectives of time in HCI. In CHI'13 Extended Abstracts on Human Factors in Computing Systems. ACM , 3211--3214. Sian Lindley, Robert Corish, Elsa Kosmack Vaara, Pedro Ferreira, and Vygandas Simbelis. 2013. Changing perspectives of time in HCI. In CHI'13 Extended Abstracts on Human Factors in Computing Systems. ACM, 3211--3214."},{"key":"e_1_2_1_18_1","volume-title":"The mythos of model interpretability. CoRR","author":"Lipton Zachary C.","year":"2016","unstructured":"Zachary C. Lipton . 2016. The mythos of model interpretability. CoRR ( 2016 ). Zachary C. Lipton. 2016. The mythos of model interpretability. CoRR (2016)."},{"key":"e_1_2_1_19_1","volume-title":"Algorithms behind Tinder, Hinge and other dating apps control your love life. Here's how to navigate them. Chicago Tribune (December","author":"Marotti Ally","year":"2018","unstructured":"Ally Marotti . 2018. Algorithms behind Tinder, Hinge and other dating apps control your love life. Here's how to navigate them. Chicago Tribune (December 2018 ). https:\/\/www.chicagotribune.com\/business\/ct-biz-app-dating- algorithms-20181202-story.html Ally Marotti. 2018. Algorithms behind Tinder, Hinge and other dating apps control your love life. Here's how to navigate them. Chicago Tribune (December 2018). https:\/\/www.chicagotribune.com\/business\/ct-biz-app-dating- algorithms-20181202-story.html"},{"key":"e_1_2_1_20_1","volume-title":"The Potential Pitfalls of Machine Learning Algorithms in Medicine. Pulmonology Advisor (December","author":"Mbadiwe Tafari","year":"2017","unstructured":"Tafari Mbadiwe . 2017. The Potential Pitfalls of Machine Learning Algorithms in Medicine. Pulmonology Advisor (December 2017 ). https:\/\/www.pulmonologyadvisor.com\/home\/topics\/practice-management\/the-potential-pitfalls-of- machine-learning-algorithms-in-medicine\/ Tafari Mbadiwe. 2017. The Potential Pitfalls of Machine Learning Algorithms in Medicine. Pulmonology Advisor (December 2017). https:\/\/www.pulmonologyadvisor.com\/home\/topics\/practice-management\/the-potential-pitfalls-of- machine-learning-algorithms-in-medicine\/"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1476589.1476628"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2317956.2318088"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557178"},{"key":"e_1_2_1_24_1","volume-title":"Jennifer Wortman Vaughan, and Hanna Wallach","author":"Poursabzi-Sangdeh Forough","year":"2018","unstructured":"Forough Poursabzi-Sangdeh , Daniel G. Goldstein , Jake Hofman , Jennifer Wortman Vaughan, and Hanna Wallach . 2018 . Manipulating and Measuring Model Interpretability. CoRR ( 2018). Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and Measuring Model Interpretability. CoRR (2018)."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2514.2517"},{"key":"e_1_2_1_27_1","volume-title":"Performance related changes and their user impact. Velocity 2009","author":"Shurman Eric","year":"2009","unstructured":"Eric Shurman and Jake Brutlag . 2009. Performance related changes and their user impact. Velocity 2009 ( 2009 ). Eric Shurman and Jake Brutlag. 2009. Performance related changes and their user impact. Velocity 2009 (2009)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1080\/00223980.2013.810128"},{"key":"e_1_2_1_29_1","volume-title":"Time is different now ... and so are we. The Verge (November","author":"Stephen Bijan","year":"2018","unstructured":"Bijan Stephen . 2018. Time is different now ... and so are we. The Verge (November 2018 ). https:\/\/www.theverge.com\/2018\/11\/25\/18111179\/facebook-twitter-global-news-engagement-perception-of-time Bijan Stephen. 2018. Time is different now ... and so are we. The Verge (November 2018). https:\/\/www.theverge.com\/2018\/11\/25\/18111179\/facebook-twitter-global-news-engagement-perception-of-time"},{"key":"e_1_2_1_30_1","volume-title":"Economies, Societies and Nations","author":"Surowiecki James","year":"2004","unstructured":"James Surowiecki . 2004. The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business , Economies, Societies and Nations . Doubleday ; Anchor ( 2004 ). James Surowiecki. 2004. The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations. Doubleday; Anchor (2004)."},{"key":"e_1_2_1_31_1","volume-title":"SIGGRAPH '78 Proceedings of the 5th annual conference on Computer graphics and interactive techniques (August","author":"Goodman","year":"1978","unstructured":"Goodman T. and Spence R . 1978. The effects of system response time on users of interactive computer systems . SIGGRAPH '78 Proceedings of the 5th annual conference on Computer graphics and interactive techniques (August 1978 ), 100--104. Goodman T. and Spence R. 1978. The effects of system response time on users of interactive computer systems. SIGGRAPH '78 Proceedings of the 5th annual conference on Computer graphics and interactive techniques (August 1978), 100--104."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2528394.2528395"},{"key":"e_1_2_1_33_1","volume-title":"The soothing promise of our own artisinal internet. Wired (February","author":"Tiku Nitasha","year":"2019","unstructured":"Nitasha Tiku . 2019. The soothing promise of our own artisinal internet. Wired (February 2019 ). https:\/\/www.wired. com\/story\/soothing-promise-our-own-artisanal-internet\/ Nitasha Tiku. 2019. The soothing promise of our own artisinal internet. Wired (February 2019). https:\/\/www.wired. com\/story\/soothing-promise-our-own-artisanal-internet\/"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the International Association of Societies of Design Research Conference (2015)","author":"Tsai W. C.","unstructured":"W. C. Tsai , A. Y. S. Chen , S. Y. Hsu , and R. H. Liang . 2015. Crescendo Message: interacting with slow messaging . Proceedings of the International Association of Societies of Design Research Conference (2015) , 2078--2095. W. C. Tsai, A. Y. S. Chen, S. Y. Hsu, and R. H. Liang. 2015. Crescendo Message: interacting with slow messaging. Proceedings of the International Association of Societies of Design Research Conference (2015), 2078--2095."},{"key":"e_1_2_1_35_1","volume-title":"Fair Work: Crowd Work Minimum Wage with One Line of Code. In Seventh AAAI Conference on Human Computation and Crowdsourcing.","author":"Whiting Mark E.","unstructured":"Mark E. Whiting , Grant Hugh , and Michael S. Bernstein . 2019 . Fair Work: Crowd Work Minimum Wage with One Line of Code. In Seventh AAAI Conference on Human Computation and Crowdsourcing. Mark E. Whiting, Grant Hugh, and Michael S. Bernstein. 2019. Fair Work: Crowd Work Minimum Wage with One Line of Code. In Seventh AAAI Conference on Human Computation and Crowdsourcing."},{"key":"e_1_2_1_36_1","volume-title":"Slow movement (culture). Retrieved","year":"2019","unstructured":"Wikipedia. 2019. Slow movement (culture). Retrieved April 3, 2019 from https:\/\/en.wikipedia.org\/wiki\/Slow_ movement_(culture) Wikipedia. 2019. Slow movement (culture). Retrieved April 3, 2019 from https:\/\/en.wikipedia.org\/wiki\/Slow_ movement_(culture)"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems","author":"Yin Ming","year":"2019","unstructured":"Ming Yin , Jennifer Wortman Vaughan , and Hanna Wallach . 2019 . Understanding the Effect of Accuracy on Trust in Machine Learning Models . Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2019). Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the Effect of Accuracy on Trust in Machine Learning Models. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2019)."}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3359204","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3359204","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3359204","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:12:54Z","timestamp":1750201974000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3359204"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,7]]},"references-count":37,"journal-issue":{"issue":"CSCW","published-print":{"date-parts":[[2019,11,7]]}},"alternative-id":["10.1145\/3359204"],"URL":"https:\/\/doi.org\/10.1145\/3359204","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,7]]},"assertion":[{"value":"2019-11-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}