{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T16:52:15Z","timestamp":1791046335657,"version":"4.1.0"},"reference-count":96,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2022,3,30]],"date-time":"2022-03-30T00:00:00Z","timestamp":1648598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2022,3,30]]},"abstract":"<jats:p>Analyzing usability test videos is arduous. Although recent research showed the promise of AI in assisting with such tasks, it remains largely unknown how AI should be designed to facilitate effective collaboration between user experience (UX) evaluators and AI. Inspired by the concepts of agency and work context in human and AI collaboration literature, we studied two corresponding design factors for AI-assisted UX evaluation: explanations and synchronization. Explanations allow AI to further inform humans how it identifies UX problems from a usability test session; synchronization refers to the two ways humans and AI collaborate: synchronously and asynchronously. We iteratively designed a tool-AI Assistant-with four versions of UIs corresponding to the two levels of explanations (with\/without) and synchronization (sync\/async). By adopting a hybrid wizard-of-oz approach to simulating an AI with reasonable performance, we conducted a mixed-method study with 24 UX evaluators identifying UX problems from usability test videos using AI Assistant. Our quantitative and qualitative results show that AI with explanations, regardless of being presented synchronously or asynchronously, provided better support for UX evaluators' analysis and was perceived more positively; when without explanations, synchronous AI better improved UX evaluators' performance and engagement compared to the asynchronous AI. Lastly, we present the design implications for AI-assisted UX evaluation and facilitating more effective human-AI collaboration.<\/jats:p>","DOI":"10.1145\/3512943","type":"journal-article","created":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T12:54:59Z","timestamp":1649336099000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":81,"title":["Human-AI Collaboration for UX Evaluation: Effects of Explanation and Synchronization"],"prefix":"10.1145","volume":"6","author":[{"given":"Mingming","family":"Fan","sequence":"first","affiliation":[{"name":"The Hong Kong University of Science and Technology (Guangzhou) &amp; The Hong Kong University of Science and Technology, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xianyou","family":"Yang","sequence":"additional","affiliation":[{"name":"Rochester Institute of Technology, Rochester, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"TszTung","family":"Yu","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, ON, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Q. Vera","family":"Liao","sequence":"additional","affiliation":[{"name":"Microsoft Research, Montreal, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Zhao","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, ON, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,4,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2870052"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v35i4.2513"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240838"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.618"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380254"},{"key":"e_1_2_1_6_1","volume-title":"Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q Vera Liao, Ronny Luss, Aleksandra Mojsilovi\u0107, et al.","author":"Arya Vijay","year":"2019","unstructured":"Vijay Arya , Rachel KE Bellamy , Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q Vera Liao, Ronny Luss, Aleksandra Mojsilovi\u0107, et al. 2019 . One explanation does not fit all: A toolkit and taxonomy of ai explainability techniques. arXiv preprint arXiv:1909.03012 (2019). Vijay Arya, Rachel KE Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q Vera Liao, Ronny Luss, Aleksandra Mojsilovi\u0107, et al. 2019. One explanation does not fit all: A toolkit and taxonomy of ai explainability techniques. arXiv preprint arXiv:1909.03012 (2019)."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2669485.2669518"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33012429"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445717"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173951"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145325"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 25th International Conference on Intelligent User Interfaces. 454--464","author":"Zana Bucc","year":"2020","unstructured":"Zana Bucc inca, Phoebe Lin , Krzysztof Z Gajos , and Elena L Glassman . 2020 . Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems . In Proceedings of the 25th International Conference on Intelligent User Interfaces. 454--464 . Zana Bucc inca, Phoebe Lin, Krzysztof Z Gajos, and Elena L Glassman. 2020. Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 454--464."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300234"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359206"},{"key":"e_1_2_1_15_1","first-page":"832","article-title":"Machine Learning Interpretability","volume":"8","author":"Carvalho Diogo V","year":"2019","unstructured":"Diogo V Carvalho , Eduardo M Pereira , and Jaime S Cardoso . 2019 . Machine Learning Interpretability : A Survey on Methods and Metrics. Electronics , Vol. 8 , 8 (2019), 832 . Diogo V Carvalho, Eduardo M Pereira, and Jaime S Cardoso. 2019. Machine Learning Interpretability: A Survey on Methods and Metrics. Electronics, Vol. 8, 8 (2019), 832.","journal-title":"A Survey on Methods and Metrics. Electronics"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/286498.286736"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300789"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951717718855"},{"key":"e_1_2_1_19_1","unstructured":"Kristin A Cook and James J Thomas. 2005. Illuminating the path: The research and development agenda for visual analytics .National Visualization and Analytics Ctr.  Kristin A Cook and James J Thomas. 2005. Illuminating the path: The research and development agenda for visual analytics .National Visualization and Analytics Ctr."},{"key":"e_1_2_1_20_1","volume-title":"The Sage dictionary of statistics: a practical resource for students in the social sciences","author":"Cramer Duncan","unstructured":"Duncan Cramer and Dennis Laurence Howitt . 2004. The Sage dictionary of statistics: a practical resource for students in the social sciences . Sage . Duncan Cramer and Dennis Laurence Howitt. 2004. The Sage dictionary of statistics: a practical resource for students in the social sciences .Sage."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302316"},{"key":"e_1_2_1_22_1","unstructured":"K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data. the MIT Press.  K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data. the MIT Press."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/604045.604056"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3325281"},{"key":"e_1_2_1_25_1","article-title":"a. Practices and Challenges of Using Think-Aloud Protocols in Industry","volume":"15","author":"Fan Mingming","year":"2020","unstructured":"Mingming Fan , Serina Shi , and Khai N Truong . 2020 a. Practices and Challenges of Using Think-Aloud Protocols in Industry . Journal of Usability Studies , Vol. 15 , 4 (2020). Mingming Fan, Serina Shi, and Khai N Truong. 2020 a. Practices and Challenges of Using Think-Aloud Protocols in Industry. Journal of Usability Studies, Vol. 15, 4 (2020).","journal-title":"Journal of Usability Studies"},{"key":"e_1_2_1_26_1","first-page":"343","article-title":"b. VisTA: Integrating Machine Intelligence with Visualization to Support the Investigation of Think-Aloud Sessions","volume":"26","author":"Fan Mingming","year":"2020","unstructured":"Mingming Fan , Ke Wu , Jian Zhao , Yue Li , Winter Wei , and Khai N Truong . 2020 b. VisTA: Integrating Machine Intelligence with Visualization to Support the Investigation of Think-Aloud Sessions . IEEE Transactions on Visualization and Computer Graphics , Vol. 26 , 1 (2020), 343 -- 352 . Mingming Fan, Ke Wu, Jian Zhao, Yue Li, Winter Wei, and Khai N Truong. 2020 b. VisTA: Integrating Machine Intelligence with Visualization to Support the Investigation of Think-Aloud Sessions. IEEE Transactions on Visualization and Computer Graphics, Vol. 26, 1 (2020), 343--352.","journal-title":"IEEE Transactions on Visualization and Computer Graphics"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208365"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447319609526147"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432934"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/DSAA.2018.00018"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359152"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2016.09.009"},{"key":"e_1_2_1_33_1","unstructured":"Nielson Norman Group. [n.d.]. 10 Heuristics for User Interface Design: Article by Jakob Nielsen. https:\/\/www.nngroup.com\/articles\/ten-usability-heuristics\/  Nielson Norman Group. [n.d.]. 10 Heuristics for User Interface Design: Article by Jakob Nielsen. https:\/\/www.nngroup.com\/articles\/ten-usability-heuristics\/"},{"key":"e_1_2_1_34_1","volume-title":"A survey of methods for explaining black box models. ACM computing surveys (CSUR)","author":"Guidotti Riccardo","year":"2018","unstructured":"Riccardo Guidotti , Anna Monreale , Salvatore Ruggieri , Franco Turini , Fosca Giannotti , and Dino Pedreschi . 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR) , Vol. 51 , 5 ( 2018 ), 1--42. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR), Vol. 51, 5 (2018), 1--42."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301423"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240781"},{"key":"e_1_2_1_37_1","unstructured":"Christian Herbst. [n.d.]. Christian's Python Library :: A Python library for voice analysis. https:\/\/homepage.univie.ac.at\/christian.herbst\/python\/namespacepraat_util.html  Christian Herbst. [n.d.]. Christian's Python Library :: A Python library for voice analysis. https:\/\/homepage.univie.ac.at\/christian.herbst\/python\/namespacepraat_util.html"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1207\/S15327590IJHC1304_05"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314273"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v8i1.14550"},{"key":"e_1_2_1_41_1","unstructured":"iMotions. 2021. iMotions. https:\/\/imotions.com\/biosensor\/fea-facial-expression-analysis\/.  iMotions. 2021. iMotions. https:\/\/imotions.com\/biosensor\/fea-facial-expression-analysis\/."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1177\/1473871611412817"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/VAST.2010.5652880"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2019.10.001"},{"key":"e_1_2_1_45_1","volume-title":"Groupware: Computer support for business teams","author":"Johansen Robert","year":"1988","unstructured":"Robert Johansen . 1988 . Groupware: Computer support for business teams . The Free Press . Robert Johansen. 1988. Groupware: Computer support for business teams .The Free Press."},{"key":"e_1_2_1_46_1","volume-title":"International conference on machine learning. PMLR, 2668--2677","author":"Kim Been","year":"2018","unstructured":"Been Kim , Martin Wattenberg , Justin Gilmer , Carrie Cai , James Wexler , Fernanda Viegas , 2018 . Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav) . In International conference on machine learning. PMLR, 2668--2677 . Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning. PMLR, 2668--2677."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300641"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.21437\/Eurospeech.2003-80"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376873"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287590"},{"key":"e_1_2_1_51_1","volume-title":"A Human-in-the-loop Perspective on AutoML: Milestones and the Road Ahead. Data Engineering","author":"Jung-Lin Lee Doris","year":"2019","unstructured":"Doris Jung-Lin Lee , Stephen Macke , Doris Xin , Angela Lee , Silu Huang , and Aditya Parameswaran . 2019. A Human-in-the-loop Perspective on AutoML: Milestones and the Road Ahead. Data Engineering ( 2019 ), 58. Doris Jung-Lin Lee, Stephen Macke, Doris Xin, Angela Lee, Silu Huang, and Aditya Parameswaran. 2019. A Human-in-the-loop Perspective on AutoML: Milestones and the Road Ahead. Data Engineering (2019), 58."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951718756684"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447319509526110"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376590"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2030112.2030168"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359297"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346573"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPC.2011.2182569"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2396636.2396673"},{"key":"e_1_2_1_60_1","volume-title":"Developing and validating trust measures for e-commerce: An integrative typology. Information systems research","author":"McKnight D Harrison","year":"2002","unstructured":"D Harrison McKnight , Vivek Choudhury , and Charles Kacmar . 2002. Developing and validating trust measures for e-commerce: An integrative typology. Information systems research , Vol. 13 , 3 ( 2002 ), 334--359. D Harrison McKnight, Vivek Choudhury, and Charles Kacmar. 2002. Developing and validating trust measures for e-commerce: An integrative typology. Information systems research, Vol. 13, 3 (2002), 334--359."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.2307\/259290"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2018.07.007"},{"key":"e_1_2_1_63_1","unstructured":"Jakob Nielsen. 2005. Ten usability heuristics. https:\/\/www.nngroup.com\/articles\/ten-usability-heuristics  Jakob Nielsen. 2005. Ten usability heuristics. https:\/\/www.nngroup.com\/articles\/ten-usability-heuristics"},{"key":"e_1_2_1_64_1","unstructured":"Don Norman. 2013. The design of everyday things: Revised and expanded edition .Basic books.  Don Norman. 2013. The design of everyday things: Revised and expanded edition .Basic books."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2013.05.003"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/3468.844354"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098279.3098558"},{"key":"e_1_2_1_68_1","volume-title":"Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW '10)","author":"Sharoda","year":"1891","unstructured":"Sharoda A. Paul and Madhu C. Reddy. 2010. Understanding Together: Sensemaking in Collaborative Information Seeking . In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW '10) . Association for Computing Machinery, New York, NY, USA, 321--330. https:\/\/doi.org\/10.1145\/17 1891 8.1718976 10.1145\/1718918.1718976 Sharoda A. Paul and Madhu C. Reddy. 2010. Understanding Together: Sensemaking in Collaborative Information Seeking. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW '10). Association for Computing Machinery, New York, NY, USA, 321--330. https:\/\/doi.org\/10.1145\/1718918.1718976"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0027443"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.106.4.643"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.enbuild.2017.12.043"},{"key":"e_1_2_1_72_1","volume-title":"Jennifer Wortman Vaughan, and Hanna Wallach","author":"Poursabzi-Sangdeh Forough","year":"2018","unstructured":"Forough Poursabzi-Sangdeh , Daniel G Goldstein , Jake M Hofman , Jennifer Wortman Vaughan, and Hanna Wallach . 2018 . Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018)."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/1111449.1111475"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_1_75_1","volume-title":"Proceedings of Graphics Interface Conference. Canadian Information Processing Society, 123--130","author":"Sarvghad Ali","year":"2015","unstructured":"Ali Sarvghad and Melanie Tory . 2015 . Exploiting analysis history to support collaborative data analysis . In Proceedings of Graphics Interface Conference. Canadian Information Processing Society, 123--130 . Ali Sarvghad and Melanie Tory. 2015. Exploiting analysis history to support collaborative data analysis. In Proceedings of Graphics Interface Conference. Canadian Information Processing Society, 123--130."},{"key":"e_1_2_1_76_1","volume-title":"Visualizing dimension coverage to support exploratory analysis","author":"Sarvghad Ali","year":"2016","unstructured":"Ali Sarvghad , Melanie Tory , and Narges Mahyar . 2016. Visualizing dimension coverage to support exploratory analysis . IEEE transactions on visualization and computer graphics, Vol. 23 , 1 ( 2016 ), 21--30. Ali Sarvghad, Melanie Tory, and Narges Mahyar. 2016. Visualizing dimension coverage to support exploratory analysis. IEEE transactions on visualization and computer graphics, Vol. 23, 1 (2016), 21--30."},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0378-7206(99)00015-4"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2020.1741118"},{"key":"e_1_2_1_80_1","volume-title":"CoUX: Collaborative Visual Analysis of Think-Aloud Usability Test Videos for Digital Interfaces","author":"Soure Ehsan Jahangirzadeh","year":"2021","unstructured":"Ehsan Jahangirzadeh Soure , Emily Kuang , Mingming Fan , and Jian Zhao . 2021. CoUX: Collaborative Visual Analysis of Think-Aloud Usability Test Videos for Digital Interfaces . IEEE Transactions on Visualization and Computer Graphics ( 2021 ), 1--1. https:\/\/doi.org\/10.1109\/TVCG.2021.3114822 10.1109\/TVCG.2021.3114822 Ehsan Jahangirzadeh Soure, Emily Kuang, Mingming Fan, and Jian Zhao. 2021. CoUX: Collaborative Visual Analysis of Think-Aloud Usability Test Videos for Digital Interfaces. IEEE Transactions on Visualization and Computer Graphics (2021), 1--1. https:\/\/doi.org\/10.1109\/TVCG.2021.3114822"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1002\/acp.1190"},{"key":"e_1_2_1_82_1","volume-title":"Work and the division of labor. Sociological quarterly","author":"Strauss Anselm","year":"1985","unstructured":"Anselm Strauss . 1985. Work and the division of labor. Sociological quarterly , Vol. 26 , 1 ( 1985 ), 1--19. Anselm Strauss. 1985. Work and the division of labor. Sociological quarterly, Vol. 26, 1 (1985), 1--19."},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314293"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2009.162"},{"key":"e_1_2_1_85_1","volume-title":"Translating videos to natural language using deep recurrent neural networks. arXiv preprint arXiv:1412.4729","author":"Venugopalan Subhashini","year":"2014","unstructured":"Subhashini Venugopalan , Huijuan Xu , Jeff Donahue , Marcus Rohrbach , Raymond Mooney , and Kate Saenko . 2014. Translating videos to natural language using deep recurrent neural networks. arXiv preprint arXiv:1412.4729 ( 2014 ). Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, and Kate Saenko. 2014. Translating videos to natural language using deep recurrent neural networks. arXiv preprint arXiv:1412.4729 (2014)."},{"key":"e_1_2_1_86_1","volume-title":"Jesse Kriss, and Matt McKeon.","author":"Viegas Fernanda B","year":"2007","unstructured":"Fernanda B Viegas , Martin Wattenberg , Frank Van Ham , Jesse Kriss, and Matt McKeon. 2007 . Manyeyes: a site for visualization at internet scale. IEEE transactions on visualization and computer graphics, Vol. 13 , 6 (2007). Fernanda B Viegas, Martin Wattenberg, Frank Van Ham, Jesse Kriss, and Matt McKeon. 2007. Manyeyes: a site for visualization at internet scale. IEEE transactions on visualization and computer graphics, Vol. 13, 6 (2007)."},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359313"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2006.65"},{"key":"e_1_2_1_89_1","volume-title":"Learning to Complement Humans. arXiv preprint arXiv:2005.00582","author":"Wilder Bryan","year":"2020","unstructured":"Bryan Wilder , Eric Horvitz , and Ece Kamar . 2020. Learning to Complement Humans. arXiv preprint arXiv:2005.00582 ( 2020 ). Bryan Wilder, Eric Horvitz, and Ece Kamar. 2020. Learning to Complement Humans. arXiv preprint arXiv:2005.00582 (2020)."},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979407"},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.1145\/3319616"},{"key":"e_1_2_1_92_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13402"},{"key":"e_1_2_1_93_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300468"},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJVD.2007.014908"},{"key":"e_1_2_1_95_1","volume-title":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","author":"Zhang Yunfeng","unstructured":"Yunfeng Zhang , Q. Vera Liao , and Rachel K. E. Bellamy . 2020. Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making . In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency ( Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 295--305. https:\/\/doi.org\/10.1145\/3351095.3372852 10.1145\/3351095.3372852 Yunfeng Zhang, Q. Vera Liao, and Rachel K. E. Bellamy. 2020. Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 295--305. https:\/\/doi.org\/10.1145\/3351095.3372852"},{"key":"e_1_2_1_96_1","volume-title":"Annotation graphs: A graph-based visualization for meta-analysis of data based on user-authored annotations","author":"Zhao Jian","year":"2016","unstructured":"Jian Zhao , Michael Glueck , Simon Breslav , Fanny Chevalier , and Azam Khan . 2016. Annotation graphs: A graph-based visualization for meta-analysis of data based on user-authored annotations . IEEE transactions on visualization and computer graphics, Vol. 23 , 1 ( 2016 ), 261--270. Jian Zhao, Michael Glueck, Simon Breslav, Fanny Chevalier, and Azam Khan. 2016. Annotation graphs: A graph-based visualization for meta-analysis of data based on user-authored annotations. IEEE transactions on visualization and computer graphics, Vol. 23, 1 (2016), 261--270."},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2745279"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512943","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3512943","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T15:31:19Z","timestamp":1750174279000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512943"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,30]]},"references-count":96,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2022,3,30]]}},"alternative-id":["10.1145\/3512943"],"URL":"https:\/\/doi.org\/10.1145\/3512943","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,30]]},"assertion":[{"value":"2022-04-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}