{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T09:01:58Z","timestamp":1775034118120,"version":"3.50.1"},"reference-count":97,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2022,11,7]]},"abstract":"<jats:p>Conversational interfaces require two types of curation: data curation by data science workers and content curation by domain experts. Recent years have seen the possibilities for content curators to instruct conversational machines in the customer service domain (i.e., Machine Teaching). The activities of curating specialized data are time-consuming. These activities have a learning curve for the domain expert, and they rely on collaborators beyond the domain experts, including product owners, technology expert curators, management, marketing, and communication employees. However, recent research has looked at making this task easier for domain experts with a lack of knowledge in the Machine Learning system, and few papers have investigated the work practices and collaborations involved in this role. This paper aims to fill this gap, presenting and unveiling practices extracted from eleven semi-structured interviews and four design workshops with experts in Banking, Technical support, Humans Resources, Telecommunications, and Automotive sectors. First, we investigate the articulation work of the content curators and tech curators in training conversational machines. Second, we inspect the curatorial and collaboration strategies they use, which are not afforded by current conversational platforms. Third, we draw the design implications and possibilities to support individual and collaboration curating practices. We reflect on how those practices rely on self and collaboration with others for curation, trust, and data tracking and ownership.<\/jats:p>","DOI":"10.1145\/3555768","type":"journal-article","created":{"date-parts":[[2022,11,11]],"date-time":"2022-11-11T22:58:54Z","timestamp":1668207534000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Unveiling Practices of Customer Service Content Curators of Conversational Agents"],"prefix":"10.1145","volume":"6","author":[{"given":"Heloisa","family":"Candello","sequence":"first","affiliation":[{"name":"IBM Research, Sao Paulo, Brazil"}]},{"given":"Claudio","family":"Pinhanez","sequence":"additional","affiliation":[{"name":"IBM Research, Sao Paulo, Brazil"}]},{"given":"Michael","family":"Muller","sequence":"additional","affiliation":[{"name":"IBM Research, Cambridge, MA, USA"}]},{"given":"Mairieli","family":"Wessel","sequence":"additional","affiliation":[{"name":"Radboud University, The Netherlands, Netherlands"}]}],"member":"320","published-online":{"date-parts":[[2022,11,11]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces","unstructured":"2020. CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces ( Bilbao, Spain). Association for Computing Machinery, New York, NY, USA. 2020. CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces (Bilbao, Spain). Association for Computing Machinery, New York, NY, USA."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441787"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v35i4.2513"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702509"},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Vijay Arya Rachel K. E. Bellamy Pin-Yu Chen Amit Dhurandhar Michael Hind Samuel C. Hoffman Stephanie Houde Q. Vera Liao Ronny Luss Aleksandra Mojsilovic Sami Mourad Pablo Pedemonte Ramya Raghavendra John Richards Prasanna Sattigeri Karthikeyan Shanmugam Moninder Singh Kush R. Varshney Dennis Wei and Yunfeng Zhang. 2019. One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques. arXiv:1909.03012 [cs.AI]  Vijay Arya Rachel K. E. Bellamy Pin-Yu Chen Amit Dhurandhar Michael Hind Samuel C. Hoffman Stephanie Houde Q. Vera Liao Ronny Luss Aleksandra Mojsilovic Sami Mourad Pablo Pedemonte Ramya Raghavendra John Richards Prasanna Sattigeri Karthikeyan Shanmugam Moninder Singh Kush R. Varshney Dennis Wei and Yunfeng Zhang. 2019. One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques. arXiv:1909.03012 [cs.AI]","DOI":"10.1145\/3351095.3375667"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3449163"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the 2019 CHI Conference on Computer Supported Cooperative Work and Social Media.","author":"Ashktorab Zahra","year":"2020","unstructured":"Zahra Ashktorab , Michael Desmond , Michael Muller , Narendra Nath Joshi , Kristina Brimijoin , Qian Pan , Christine T. Wolf , Aabhas Sharma , Evelyn Duesterward , Casey Dugan , Josh Andres , Werner Geyer , and Darrell Reimer . 2020 . Designing ground truth and the social life of labels . In Proceedings of the 2019 CHI Conference on Computer Supported Cooperative Work and Social Media. in press. Zahra Ashktorab, Michael Desmond, Michael Muller, Narendra Nath Joshi, Kristina Brimijoin, Qian Pan, Christine T. Wolf, Aabhas Sharma, Evelyn Duesterward, Casey Dugan, Josh Andres, Werner Geyer, and Darrell Reimer. 2020. Designing ground truth and the social life of labels. In Proceedings of the 2019 CHI Conference on Computer Supported Cooperative Work and Social Media. in press."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10357"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2007.88"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313602"},{"key":"e_1_2_1_11_1","volume-title":"Datahub: Collaborative data science & dataset version management at scale. arXiv preprint arXiv:1409.0798","author":"Bhardwaj Anant","year":"2014","unstructured":"Anant Bhardwaj , Souvik Bhattacherjee , Amit Chavan , Amol Deshpande , Aaron J Elmore , Samuel Madden , and Aditya G Parameswaran . 2014 . Datahub: Collaborative data science & dataset version management at scale. arXiv preprint arXiv:1409.0798 (2014). Anant Bhardwaj, Souvik Bhattacherjee, Amit Chavan, Amol Deshpande, Aaron J Elmore, Samuel Madden, and Aditya G Parameswaran. 2014. Datahub: Collaborative data science & dataset version management at scale. arXiv preprint arXiv:1409.0798 (2014)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209889.3209891"},{"key":"e_1_2_1_13_1","volume-title":"not a map': Nikki Hayfield in conversation with Virginia Braun and Victoria Clarke about thematic analysis. Qualitative Research in Psychology","author":"Braun Virginia","year":"2019","unstructured":"Virginia Braun , Victoria Clarke , and Nikki Hayfield . 2019. \" A starting point for your journey , not a map': Nikki Hayfield in conversation with Virginia Braun and Victoria Clarke about thematic analysis. Qualitative Research in Psychology ( 2019 ), 1--22. Virginia Braun, Victoria Clarke, and Nikki Hayfield. 2019. \"A starting point for your journey, not a map': Nikki Hayfield in conversation with Virginia Braun and Victoria Clarke about thematic analysis. Qualitative Research in Psychology (2019), 1--22."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3375152"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-91803-7_31"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3405755.3406173"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.10.118"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432924"},{"key":"e_1_2_1_19_1","volume-title":"Cosmin Munteanu, Christine Murad, et al .","author":"Doyle Philip R","year":"2020","unstructured":"Philip R Doyle , Daniel Rough , Justin Edwards , Benjamin R Cowan , Leigh Clark , Martin Porcheron , Stephan Schl\u00f6gl , Mar\u00eda In\u00e9s Torres , Cosmin Munteanu, Christine Murad, et al . 2020 . CUI@ IUI : Theoretical and Methodological Challenges in Intelligent Conversational User Interface Interactions . (2020). Philip R Doyle, Daniel Rough, Justin Edwards, Benjamin R Cowan, Leigh Clark, Martin Porcheron, Stephan Schl\u00f6gl, Mar\u00eda In\u00e9s Torres, Cosmin Munteanu, Christine Murad, et al . 2020. CUI@ IUI: Theoretical and Methodological Challenges in Intelligent Conversational User Interface Interactions. (2020)."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3185517"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/604045.604056"},{"key":"e_1_2_1_22_1","volume-title":"Proc. of ICML'18","author":"Fan Yang","year":"2018","unstructured":"Yang Fan , Fei Tian , Tao Qin , Xiang-Yang Li , and Tie-Yan Liu . 2018 . Learning to Teach . In Proc. of ICML'18 . Yang Fan, Fei Tian, Tao Qin, Xiang-Yang Li, and Tie-Yan Liu. 2018. Learning to Teach. In Proc. of ICML'18."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025837"},{"key":"e_1_2_1_24_1","volume-title":"Material Vision. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 604--617","author":"Feinberg Melanie","year":"2017","unstructured":"Melanie Feinberg . 2017 . Material Vision. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 604--617 . Melanie Feinberg. 2017. Material Vision. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 604--617."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1978965"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357061"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the ACM on Human-Computer Interaction 4, CSCW3","author":"Ghai Bhavya","year":"2021","unstructured":"Bhavya Ghai , Q Vera Liao , Yunfeng Zhang , Rachel Bellamy , and Klaus Mueller . 2021 . Explainable Active Learning (XAL) Toward AI Explanations as Interfaces for Machine Teachers . Proceedings of the ACM on Human-Computer Interaction 4, CSCW3 (2021), 1--28. Bhavya Ghai, Q Vera Liao, Yunfeng Zhang, Rachel Bellamy, and Klaus Mueller. 2021. Explainable Active Learning (XAL) Toward AI Explanations as Interfaces for Machine Teachers. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3 (2021), 1--28."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00136714"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10111-010-0139-2"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1468-005X.2005.00151.x"},{"key":"e_1_2_1_31_1","volume-title":"Training Classifiers with Natural Language Explanations. arXiv preprint arXiv:1805.03818","author":"Hancock Braden","year":"2018","unstructured":"Braden Hancock , Paroma Varma , Stephanie Wang , Martin Bringmann , Percy Liang , and Christopher R\u00e9. 2018. Training Classifiers with Natural Language Explanations. arXiv preprint arXiv:1805.03818 ( 2018 ). Braden Hancock, Paroma Varma, Stephanie Wang, Martin Bringmann, Percy Liang, and Christopher R\u00e9. 2018. Training Classifiers with Natural Language Explanations. arXiv preprint arXiv:1805.03818 (2018)."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2018.2881519"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2696454.2696479"},{"key":"e_1_2_1_34_1","volume-title":"Line Melby, and Pieter Toussaint","author":"Iversen Tobias Buschmann","year":"2013","unstructured":"Tobias Buschmann Iversen , Line Melby, and Pieter Toussaint . 2013 . Instant messaging at the hospital: Supporting articulation work? International journal of medical informatics 82, 9 (2013), 753--761. Tobias Buschmann Iversen, Line Melby, and Pieter Toussaint. 2013. Instant messaging at the hospital: Supporting articulation work? International journal of medical informatics 82, 9 (2013), 753--761."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22362-4_31"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22362-4_31"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/21.31053"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2678025.2701399"},{"key":"e_1_2_1_39_1","volume-title":"Studies in Conversational UX Design","author":"Jia-Jun Li Toby","unstructured":"Toby Jia-Jun Li , Igor Labutov , Brad A Myers , Amos Azaria , Alexander I Rudnicky , and Tom M Mitchell . 2018. Teaching agents when they fail: end user development in goal-oriented conversational agents . In Studies in Conversational UX Design . Springer , 119--137. Toby Jia-Jun Li, Igor Labutov, Brad A Myers, Amos Azaria, Alexander I Rudnicky, and Tom M Mitchell. 2018. Teaching agents when they fail: end user development in goal-oriented conversational agents. In Studies in Conversational UX Design. Springer, 119--137."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2901790.2901842"},{"key":"e_1_2_1_41_1","volume-title":"Proc. of ICML'17","author":"Liu Weiyang","year":"2017","unstructured":"Weiyang Liu , Bo Dai , Ahmad Humayun , Charlene Tay , Chen Yu , Linda B Smith , James M Rehg , and Le Song . 2017 . Iterative machine teaching . In Proc. of ICML'17 . 2149--2158. Weiyang Liu, Bo Dai, Ahmad Humayun, Charlene Tay, Chen Yu, Linda B Smith, James M Rehg, and Le Song. 2017. Iterative machine teaching. In Proc. of ICML'17. 2149--2158."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2016.12.038"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2686612.2686684"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/505282.505285"},{"key":"e_1_2_1_45_1","volume-title":"The Materialization of Articulation Work and its Implications on Nursing Practices","author":"Mellini Barbara","unstructured":"Barbara Mellini . 2013. The Materialization of Articulation Work and its Implications on Nursing Practices .. In CHItaly (Doctoral Consortium) . Citeseer, 74--83. Barbara Mellini. 2013. The Materialization of Articulation Work and its Implications on Nursing Practices.. In CHItaly (Doctoral Consortium). Citeseer, 74--83."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819978"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386389"},{"key":"e_1_2_1_48_1","volume-title":"Studies in Conversational UX Design, Raphael Arar Guang-Jie Ren Robert J","author":"Moore Robert J","unstructured":"Robert J Moore and Raphael Arar . 2018. Conversational UX design: an Introduction . In Studies in Conversational UX Design, Raphael Arar Guang-Jie Ren Robert J . Moore, Margaret H. Szymanski (Ed.). Springer , 1--16. Robert J Moore and Raphael Arar. 2018. Conversational UX design: an Introduction. In Studies in Conversational UX Design, Raphael Arar Guang-Jie Ren Robert J. Moore, Margaret H. Szymanski (Ed.). Springer, 1--16."},{"key":"e_1_2_1_49_1","volume-title":"Kostakos. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems-CHI","volume":"19","author":"Muller Michael","year":"2019","unstructured":"Michael Muller , Ingrid Lange , Dakuo Wang , David Piorkowski , Jason Tsay , Q Vera Liao , Casey Dugan , and Thomas Erickson . 2019 . How Data Science Workers Work with Data. In S. Brewster, G. Fitzpatrick, A. Cox, and V . Kostakos. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems-CHI , Vol. 19 . 2019--09. Michael Muller, Ingrid Lange, Dakuo Wang, David Piorkowski, Jason Tsay, Q Vera Liao, Casey Dugan, and Thomas Erickson. 2019. How Data Science Workers Work with Data. In S. Brewster, G. Fitzpatrick, A. Cox, and V. Kostakos. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems-CHI, Vol. 19. 2019--09."},{"key":"e_1_2_1_50_1","volume-title":"Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems.","author":"Muller Michael","year":"2021","unstructured":"Michael Muller , Christine T. Wolf , Josh Andres , Zahra Ashktorab , Narendra Nath Joshi , Michael Desmond , Aabhas Sharma , Kristina Brimijoin , Qian Pan , Evelyn Duesterward , and Casey Dugan . 2021 . Designing ground truth and the social life of labels . In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. in press. Michael Muller, Christine T. Wolf, Josh Andres, Zahra Ashktorab, Narendra Nath Joshi, Michael Desmond, Aabhas Sharma, Kristina Brimijoin, Qian Pan, Evelyn Duesterward, and Casey Dugan. 2021. Designing ground truth and the social life of labels. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. in press."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379336.3379358"},{"key":"e_1_2_1_52_1","volume-title":"A web on the wind: The structure of invisible work. Computer supported cooperative work 8, 1--2","author":"Nardi Bonnie A","year":"1999","unstructured":"Bonnie A Nardi and Yrj\u00f6 Engestr\u00f6m . 1999. A web on the wind: The structure of invisible work. Computer supported cooperative work 8, 1--2 ( 1999 ), 1--8. Bonnie A Nardi and Yrj\u00f6 Engestr\u00f6m. 1999. A web on the wind: The structure of invisible work. Computer supported cooperative work 8, 1--2 (1999), 1--8."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998331"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702298"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3405755.3406153"},{"key":"e_1_2_1_56_1","volume-title":"Julio Nogima, Maira G de Bayser, Melina Guerra, Henrique Ferreira, et al .","author":"Pinhanez Claudio S","year":"2021","unstructured":"Claudio S Pinhanez , Heloisa C Candello , Paulo Cavalin , Mauro C Pichiliani , Ana P Appel , Victor HA Ribeiro , Julio Nogima, Maira G de Bayser, Melina Guerra, Henrique Ferreira, et al . 2021 . Integrating Machine Learning Data with Symbolic Knowledge from Collaboration Practices of Curators to Improve Conversational Systems . (2021). Claudio S Pinhanez, Heloisa C Candello, Paulo Cavalin, Mauro C Pichiliani, Ana P Appel, Victor HA Ribeiro, Julio Nogima, Maira G de Bayser, Melina Guerra, Henrique Ferreira, et al . 2021. Integrating Machine Learning Data with Symbolic Knowledge from Collaboration Practices of Curators to Improve Conversational Systems. (2021)."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3406865.3418587"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1115\/DETC2020-22377"},{"key":"e_1_2_1_59_1","volume-title":"Principles of data wrangling: Practical techniques for data preparation. \" O'Reilly Media","author":"Rattenbury Tye","unstructured":"Tye Rattenbury , Joseph M Hellerstein , Jeffrey Heer , Sean Kandel , and Connor Carreras . 2017. Principles of data wrangling: Practical techniques for data preparation. \" O'Reilly Media , Inc .\". Tye Rattenbury, Joseph M Hellerstein, Jeffrey Heer, Sean Kandel, and Connor Carreras. 2017. Principles of data wrangling: Practical techniques for data preparation. \" O'Reilly Media, Inc.\"."},{"key":"e_1_2_1_60_1","volume-title":"Malte Ostendorff, Karolina Zaczynska, Armin Berger, Stefan Grill, S\u00f6ren R\u00e4uchle, et al.","author":"Rehm Georg","year":"2020","unstructured":"Georg Rehm , Peter Bourgonje , Stefanie Hegele , Florian Kintzel , Juli\u00e1n Moreno Schneider , Malte Ostendorff, Karolina Zaczynska, Armin Berger, Stefan Grill, S\u00f6ren R\u00e4uchle, et al. 2020 . QURATOR : Innovative Technologies for Content and Data Curation . arXiv preprint arXiv:2004.12195 (2020). Georg Rehm, Peter Bourgonje, Stefanie Hegele, Florian Kintzel, Juli\u00e1n Moreno Schneider, Malte Ostendorff, Karolina Zaczynska, Armin Berger, Stefan Grill, S\u00f6ren R\u00e4uchle, et al. 2020. QURATOR: Innovative Technologies for Content and Data Curation. arXiv preprint arXiv:2004.12195 (2020)."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2883500"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3405755.3406140"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173386.3177074"},{"key":"e_1_2_1_64_1","volume-title":"Proc. of the PPIG'15","author":"Sarkar Advait","year":"2015","unstructured":"Advait Sarkar . 2015 . Confidence, command, complexity: metamodels for structured interaction with machine intelligence . In Proc. of the PPIG'15 . Advait Sarkar. 2015. Confidence, command, complexity: metamodels for structured interaction with machine intelligence. In Proc. of the PPIG'15."},{"key":"e_1_2_1_65_1","volume-title":"Cooperative work and its articulation: requirements for computer support. Le travail humain","author":"Schmidt Kjeld","year":"1994","unstructured":"Kjeld Schmidt . 1994. Cooperative work and its articulation: requirements for computer support. Le travail humain ( 1994 ), 345--366. Kjeld Schmidt. 1994. Cooperative work and its articulation: requirements for computer support. Le travail humain (1994), 345--366."},{"key":"e_1_2_1_66_1","volume-title":"Remarks on the complexity of cooperative work. Revue d'intelligence artificielle 16, 4--5","author":"Schmidt Kjeld","year":"2002","unstructured":"Kjeld Schmidt . 2002. Remarks on the complexity of cooperative work. Revue d'intelligence artificielle 16, 4--5 ( 2002 ), 443--483. Kjeld Schmidt. 2002. Remarks on the complexity of cooperative work. Revue d'intelligence artificielle 16, 4--5 (2002), 443--483."},{"key":"e_1_2_1_67_1","volume-title":"Cooperative work and coordinative practices: Contributions to the conceptual foundations of Computer-Supported Cooperative Work (CSCW)","author":"Schmidt Kjeld","unstructured":"Kjeld Schmidt . 2011. Cooperative work and coordinative practices: Contributions to the conceptual foundations of Computer-Supported Cooperative Work (CSCW) . Springer Science & Business Media . Kjeld Schmidt. 2011. Cooperative work and coordinative practices: Contributions to the conceptual foundations of Computer-Supported Cooperative Work (CSCW). Springer Science & Business Media."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00133655"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.5555\/1045432.1045434"},{"key":"e_1_2_1_70_1","volume-title":"Intent Classification for Dialogue Utterances","author":"Schuurmans Jetze","year":"2019","unstructured":"Jetze Schuurmans and Flavius Frasincar . 2019. Intent Classification for Dialogue Utterances . IEEE Intelligent Systems ( 2019 ). Jetze Schuurmans and Flavius Frasincar. 2019. Intent Classification for Dialogue Utterances. IEEE Intelligent Systems (2019)."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.14236\/ewic\/HCI2018.14"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3029798.3038385"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3319863"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2017.03.009"},{"key":"e_1_2_1_75_1","volume-title":"Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, et al .","author":"Simard Patrice Y","year":"2017","unstructured":"Patrice Y Simard , Saleema Amershi , David M Chickering , Alicia Edelman Pelton , Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, et al . 2017 . Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742 (2017). Patrice Y Simard, Saleema Amershi, David M Chickering, Alicia Edelman Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, et al . 2017. Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742 (2017)."},{"key":"e_1_2_1_76_1","volume-title":"Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742","author":"Simard Patrice Y","year":"2017","unstructured":"Patrice Y Simard , Saleema Amershi , David M Chickering , Alicia E Pelton , Soroush Ghorashi , Christopher Meek , Gonzalo Ramos , Jina Suh , Johan Verwey , Mo Wang , and Joh Wernsing . 2017. Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742 ( 2017 ). Patrice Y Simard, Saleema Amershi, David M Chickering, Alicia E Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, and Joh Wernsing. 2017. Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742 (2017)."},{"key":"e_1_2_1_77_1","volume-title":"Alicia Edelman Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, and John Robert Wernsing.","author":"Simard Patrice Y.","year":"2017","unstructured":"Patrice Y. Simard , Saleema Amershi , David Maxwell Chickering , Alicia Edelman Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, and John Robert Wernsing. 2017 . Machine Teaching : A New Paradigm for Building Machine Learning Systems. CoRR abs\/1707.06742 (2017). Patrice Y. Simard, Saleema Amershi, David Maxwell Chickering, Alicia Edelman Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, and John Robert Wernsing. 2017. Machine Teaching: A New Paradigm for Building Machine Learning Systems. CoRR abs\/1707.06742 (2017)."},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/295666.295671"},{"key":"e_1_2_1_79_1","volume-title":"Layers of silence, arenas of voice: The ecology of visible and invisible work. Computer supported cooperative work (CSCW) 8, 1--2","author":"Star Susan Leigh","year":"1999","unstructured":"Susan Leigh Star and Anselm Strauss . 1999. Layers of silence, arenas of voice: The ecology of visible and invisible work. Computer supported cooperative work (CSCW) 8, 1--2 ( 1999 ), 9--30. Susan Leigh Star and Anselm Strauss. 1999. Layers of silence, arenas of voice: The ecology of visible and invisible work. Computer supported cooperative work (CSCW) 8, 1--2 (1999), 9--30."},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2009.03.004"},{"key":"e_1_2_1_81_1","doi-asserted-by":"crossref","unstructured":"Lucy A Suchman. 1996. Supporting articulation work.  Lucy A Suchman. 1996. Supporting articulation work.","DOI":"10.1016\/B978-0-12-415040-9.50118-4"},{"key":"e_1_2_1_82_1","unstructured":"Alessandra Talamo Barbara Mellini and Barbara Barbieri. 2017. The role of locally-designed organizational artifacts in supporting nurses' work: an ethnographic study on the wards. (2017).  Alessandra Talamo Barbara Mellini and Barbara Barbieri. 2017. The role of locally-designed organizational artifacts in supporting nurses' work: an ethnographic study on the wards. (2017)."},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702558"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.4018\/jkm.2005070102"},{"key":"e_1_2_1_85_1","volume-title":"Thematic analysis. The SAGE handbook of qualitative research in psychology 2","author":"Terry Gareth","year":"2017","unstructured":"Gareth Terry , Nikki Hayfield , Victoria Clarke , and Virginia Braun . 2017. Thematic analysis. The SAGE handbook of qualitative research in psychology 2 ( 2017 ), 17--37. Gareth Terry, Nikki Hayfield, Victoria Clarke, and Virginia Braun. 2017. Thematic analysis. The SAGE handbook of qualitative research in psychology 2 (2017), 17--37."},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2007.09.009"},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274442"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1145\/1810295.1810400"},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806854"},{"key":"e_1_2_1_90_1","volume-title":"Intuitive Dialogue Flows Design for Conversational Interfaces. CUI@CSCW: Collaborating through Conversational User Interfaces workshop at CSCW2020 1, 1","author":"Valtolina Stefano","year":"2020","unstructured":"Stefano Valtolina and Lorenzo Neri . 2020. Intuitive Dialogue Flows Design for Conversational Interfaces. CUI@CSCW: Collaborating through Conversational User Interfaces workshop at CSCW2020 1, 1 ( 2020 ). Stefano Valtolina and Lorenzo Neri. 2020. Intuitive Dialogue Flows Design for Conversational Interfaces. CUI@CSCW: Collaborating through Conversational User Interfaces workshop at CSCW2020 1, 1 (2020)."},{"key":"e_1_2_1_91_1","volume-title":"Attention is all you need. arXiv preprint arXiv:1706.03762","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan N Gomez , Lukasz Kaiser , and Illia Polosukhin . 2017. Attention is all you need. arXiv preprint arXiv:1706.03762 ( 2017 ). Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. arXiv preprint arXiv:1706.03762 (2017)."},{"key":"e_1_2_1_92_1","doi-asserted-by":"publisher","DOI":"10.1145\/3342775.3342804"},{"key":"e_1_2_1_93_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445526"},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359313"},{"key":"e_1_2_1_95_1","volume-title":"Conceptualizing Care in the Everyday Work Practices of Machine Learning Developers. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion","author":"Wolf Christine T.","year":"2019","unstructured":"Christine T. Wolf . 2019 . Conceptualizing Care in the Everyday Work Practices of Machine Learning Developers. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion ( San Diego, CA, USA) (DIS '19 Companion). Association for Computing Machinery, New York, NY, USA, 331--335. https:\/\/doi.org\/10.1145\/3301019.3323879 10.1145\/3301019.3323879 Christine T. Wolf. 2019. Conceptualizing Care in the Everyday Work Practices of Machine Learning Developers. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion (San Diego, CA, USA) (DIS '19 Companion). Association for Computing Machinery, New York, NY, USA, 331--335. https:\/\/doi.org\/10.1145\/3301019.3323879"},{"key":"e_1_2_1_96_1","volume-title":"Data analysis for the critical decision method. Task analysis for human-computer interaction","author":"Wong BLW","year":"2004","unstructured":"BLW Wong . 2004. Data analysis for the critical decision method. Task analysis for human-computer interaction ( 2004 ), 327--346. BLW Wong. 2004. Data analysis for the critical decision method. Task analysis for human-computer interaction (2004), 327--346."},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v29i1.9761"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555768","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3555768","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:19Z","timestamp":1750182559000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555768"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":97,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2022,11,7]]}},"alternative-id":["10.1145\/3555768"],"URL":"https:\/\/doi.org\/10.1145\/3555768","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,7]]},"assertion":[{"value":"2022-11-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}