{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,19]],"date-time":"2026-11-19T00:15:27Z","timestamp":1795047327984,"version":"4.10.2"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,11,29]],"date-time":"2023-11-29T00:00:00Z","timestamp":1701216000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput.-Hum. Interact."],"published-print":{"date-parts":[[2024,2,29]]},"abstract":"<jats:p>\n            While AI-assisted individual qualitative analysis has been substantially studied, AI-assisted\n            <jats:bold>collaborative<\/jats:bold>\n            qualitative analysis (CQA) \u2013 a process that involves multiple researchers working together to interpret data\u2014remains relatively unexplored. After identifying CQA practices and design opportunities through formative interviews, we designed and implemented CoAIcoder, a tool leveraging AI to enhance human-to-human collaboration within CQA through four distinct collaboration methods. With a between-subject design, we evaluated CoAIcoder with 32 pairs of CQA-trained participants across common CQA phases under each collaboration method. Our findings suggest that while using a shared AI model as a mediator among coders could improve CQA efficiency and foster agreement more quickly in the early coding stage, it might affect the final code diversity. We also emphasize the need to consider the independence level when using AI to assist human-to-human collaboration in various CQA scenarios. Lastly, we suggest design implications for future AI-assisted CQA systems.\n          <\/jats:p>","DOI":"10.1145\/3617362","type":"journal-article","created":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T12:03:48Z","timestamp":1692878628000},"page":"1-38","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":72,"title":["CoAIcoder: Examining the Effectiveness of AI-assisted Human-to-Human Collaboration in Qualitative Analysis"],"prefix":"10.1145","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5992-6471","authenticated-orcid":false,"given":"Jie","family":"Gao","sequence":"first","affiliation":[{"name":"Singapore University of Technology and Design, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3845-9143","authenticated-orcid":false,"given":"Kenny Tsu Wei","family":"Choo","sequence":"additional","affiliation":[{"name":"Singapore University of Technology and Design, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-4118-7376","authenticated-orcid":false,"given":"Junming","family":"Cao","sequence":"additional","affiliation":[{"name":"Fudan University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1986-7750","authenticated-orcid":false,"given":"Roy Ka-Wei","family":"Lee","sequence":"additional","affiliation":[{"name":"Singapore University of Technology and Design, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3105-9350","authenticated-orcid":false,"given":"Simon","family":"Perrault","sequence":"additional","affiliation":[{"name":"Singapore University of Technology and Design, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,11,29]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"Roehl Sybing How research AI can enhance your analysis. Retrieved September 16 2023 from https:\/\/atlasti.com\/research-hub\/how-research-ai-can-enhance-your-analysis#how-research-ai-can-enhance-your-analysis"},{"key":"e_1_3_3_3_2","unstructured":"MaxQDA. MAXQDA TeamCloud for Interaction and Data Exchange. Retrieved September 16 2023 from https:\/\/www.maxqda.com\/teamcloud"},{"key":"e_1_3_3_4_2","unstructured":"IBM. What is text mining?. Retrieved September 16 2023 from https:\/\/www.ibm.com\/topics\/text-mining"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sbspro.2009.01.421"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.311.7003.485"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.22034\/ELT.2021.47608.2432"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.56645\/jmde.v12i27.449"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3415167"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/2854946.2854960"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23786"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.53841\/bpsptr.2019.25.2.30"},{"key":"e_1_3_3_13_2","first-page":"5","volume-title":"Proceedings of the Workshop on Designing Technologies to Support Human Problem Solving-VL\/HCC","year":"2018","unstructured":"Alan Blackwell, Luke Church, Ian Hales, Matthew Jones, Richard Jones, Matthew Mahmoudi, Mariana Marasoiu, Sallyanne Meakins, Detlef Nauck, Karl Prince, Ana Semrov, Alexander Simpson, Martin Spott, Alain Vuylsteke, and Xiaomeng Wang. 2018. Computer says \u2018don\u2019t know\u201d-interacting visually with incomplete AI models. In Proceedings of the Workshop on Designing Technologies to Support Human Problem Solving-VL\/HCC. 5\u201314."},{"key":"e_1_3_3_14_2","unstructured":"Tom Bocklisch Joey Faulkner Nick Pawlowski and Alan Nichol. 2017. Rasa: Open Source Language Understanding and Dialogue Management ."},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.4135\/9781848607941"},{"key":"e_1_3_3_17_2","unstructured":"Tanja Bunk Daksh Varshneya Vladimir Vlasov and Alan Nichol. 2020. DIET: Lightweight Language Understanding for Dialogue Systems ."},{"key":"e_1_3_3_18_2","unstructured":"Junming Cao Bihuan Chen Longjie Hu Jie Gao Kaifeng Huang and Xin Peng. 2023. Understanding the Complexity and Its Impact on Testing in ML-Enabled Systems ."},{"key":"e_1_3_3_19_2","unstructured":"Mariano Ceccato Nadzeya Kiyavitskaya Nicola Zeni Luisa Mich Daniel M. Berry. 2004. Ambiguity Identification and Measurement in Natural Language Texts. UNSPECIFIED. (Unpublished)."},{"key":"e_1_3_3_20_2","volume-title":"Constructing Grounded Theory","author":"Charmaz Kathy","year":"2014","unstructured":"Kathy Charmaz. 2014. Constructing Grounded Theory. Sage Publications."},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3185515"},{"key":"e_1_3_3_22_2","volume-title":"Proceedings of the CHI 2016 Workshop on Human Centred Machine Learning","author":"Chen Nan-chen","year":"2016","unstructured":"Nan-chen Chen, Rafal Kocielnik, Margaret Drouhard, Vanessa Pe\u00f1a-Araya, Jina Suh, Keting Cen, Xiangyi Zheng, and Cecilia R. Aragon. 2016. Challenges of applying machine learning to qualitative coding. In Proceedings of the CHI 2016 Workshop on Human Centred Machine Learning."},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300789"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290607.3312879"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.4135\/9781452230153"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.4135\/9781446282243"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1002\/meet.14504701328"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00449"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1177\/1525822X10388468"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/PACIFICVIS.2017.8031598"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3479856"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.4135\/9781446282243"},{"issue":"32","key":"e_1_3_3_33_2","first-page":"97","article-title":"Learn for yourself: The self-learning tools for qualitative analysis software packages","author":"Freitas F\u00e1bio","year":"2017","unstructured":"F\u00e1bio Freitas, Jaime Ribeiro, Catarina Brand\u00e3o, Lu\u00eds Paulo Reis, Francisl\u00ea N. de Souza, and Ant\u00f3nio Pedro Costa. 2017. Learn for yourself: The self-learning tools for qualitative analysis software packages. Digital Education Review32 (2017), 97\u2013117. https:\/\/files.eric.ed.gov\/fulltext\/EJ1166487.pdf","journal-title":"Digital Education Review"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3274401"},{"key":"e_1_3_3_35_2","doi-asserted-by":"crossref","unstructured":"Jie Gao Yuchen Guo Gionnieve Lim Tianqin Zhang Zheng Zhang Toby Jia-Jun Li and Simon Tangi Perrault. 2023. CollabCoder: A GPT-Powered Workflow for Collaborative Qualitative Analysis .","DOI":"10.1145\/3584931.3607500"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581352"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.5555\/1216442.1216444"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.46743\/2160-3715\/2003.1870"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047215"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/IALP.2017.8300601"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2021.113515"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1183\/09031936.00136408"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3526113.3545681"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.46743\/2160-3715\/2021.4639"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/EMR.2008.4490137"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3449168"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.24251\/HICSS.2020.094"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675154"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300641"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1177\/1525822X0001200301"},{"key":"e_1_3_3_51_2","volume-title":"Research Methods in Human-Computer Interaction (2nd Ed.)","author":"Lazar Jonathan","year":"2017","unstructured":"Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research Methods in Human-Computer Interaction (2nd Ed.). Morgan Kaufmann, Cambridge, MA."},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1177\/1609406919887021"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1136\/fmch-2021-001287"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376590"},{"key":"e_1_3_3_55_2","unstructured":"Huiting Liu Avinesh P. V. S. Siddharth Patwardhan Peter Grasch and Sachin Agarwal. 2022. Model Stability with Continuous Data Updates ."},{"key":"e_1_3_3_56_2","unstructured":"Brian Lubars and Chenhao Tan. 2019. Ask not what AI can do but what AI should do: Towards a framework of task delegability. arXiv:1902.03245 Retrieved from http:\/\/arxiv.org\/abs\/1902.03245"},{"issue":"3","key":"e_1_3_3_57_2","article-title":"Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars.","volume":"9","author":"Maguire Moira","year":"2017","unstructured":"Moira Maguire and Brid Delahunt. 2017. Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars. All Ireland Journal of Higher Education 9, 3 (2017), 3351\u201333514.","journal-title":"All Ireland Journal of Higher Education"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173922"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.11613\/BM.2012.031"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2015.7163127"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1177\/0049124117729703"},{"key":"e_1_3_3_62_2","unstructured":"Peter Nielsen. 2012. Collaborative Coding of Qualitative Data (White paper) ."},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.1177\/1473325017744860"},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1177\/1609406919899220"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347957"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1123\/jtpe.2017-0084"},{"key":"e_1_3_3_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445591"},{"key":"e_1_3_3_68_2","first-page":"1","article-title":"Qualitative psychology: A practical guide to research methods","author":"Smith Jonathan A.","year":"2015","unstructured":"Jonathan A. Smith. 2015. Qualitative psychology: A practical guide to research methods. Qualitative Psychology (2015), 1\u2013312.","journal-title":"Qualitative Psychology"},{"key":"e_1_3_3_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3579605"},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-15636-7_4"},{"key":"e_1_3_3_71_2","unstructured":"Jasy Liew Suet Yan Nancy McCracken and Kevin Crowston. 2014. Semi-automatic content analysis of qualitative data. In iConference Berlin Germany."},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173733"},{"key":"e_1_3_3_73_2","doi-asserted-by":"crossref","unstructured":"Zheng Zhang Jie Gao Ranjodh Singh Dhaliwal and Toby Jia-Jun Li. 2023. VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping. arXiv preprint arXiv:2304.07810 (2023).","DOI":"10.1145\/3586183.3606800"}],"container-title":["ACM Transactions on Computer-Human Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3617362","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3617362","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:45:54Z","timestamp":1750178754000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3617362"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,29]]},"references-count":72,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,2,29]]}},"alternative-id":["10.1145\/3617362"],"URL":"https:\/\/doi.org\/10.1145\/3617362","relation":{},"ISSN":["1073-0516","1557-7325"],"issn-type":[{"value":"1073-0516","type":"print"},{"value":"1557-7325","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,29]]},"assertion":[{"value":"2022-04-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-27","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}