{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T19:49:05Z","timestamp":1767383345789,"version":"build-2065373602"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"7","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,10,18]]},"abstract":"<jats:p>This paper explores the potential role of AI, e.g., large language models (LLMs), in supporting theme development in thematic analysis. While prior applications of AI in qualitative data analysis have focused on supporting coding, we investigate whether LLMs can effectively contribute as collaborators in the more abstract and conceptual phases of qualitative analysis, specifically theme development. Despite growing interest in AI as a collaborator in theme development, there is limited empirical evidence on designing AI-assisted tools while supporting user autonomy and understanding researcher interaction with AI-assisted theme development. To address this gap, we designed ThemeViz, an interactive system that uses GPT-4 to generate and visualize multiple versions of themes based on user input while allowing researchers to maintain control through manual coding and theme development. Our study examines the effectiveness of this human-AI collaboration approach in iterative theme development and its implications for future designs.<\/jats:p>","DOI":"10.1145\/3757675","type":"journal-article","created":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T17:06:01Z","timestamp":1760634361000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["ThemeViz: Understanding the Effect of Human-AI Collaboration in Theme Development with an LLM-enhanced Interactive Visual System"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-6062-8481","authenticated-orcid":false,"given":"Daye","family":"Kang","sequence":"first","affiliation":[{"name":"Cornell University, Ithaca, New York, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5123-4110","authenticated-orcid":false,"given":"Zhuolun","family":"Han","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, New York, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4365-0540","authenticated-orcid":false,"given":"Jiahe","family":"Tian","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, New York, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-8160-1176","authenticated-orcid":false,"given":"Muhan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, New York, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4317-9501","authenticated-orcid":false,"given":"Jeffrey M","family":"Rzeszotarski","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, New York, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,10,16]]},"reference":[{"doi-asserted-by":"publisher","key":"e_1_2_1_1_1","DOI":"10.1145\/3616855.3635856"},{"unstructured":"Atlas. 2023. ATLAS.ti in 45 minutes. https:\/\/atlasti.com\/trainings\/atlas-ti-in-45-minutes.","key":"e_1_2_1_2_1"},{"doi-asserted-by":"publisher","key":"e_1_2_1_3_1","DOI":"10.1177\/1046496403256011"},{"doi-asserted-by":"publisher","key":"e_1_2_1_4_1","DOI":"10.1080\/0013188032000133548"},{"unstructured":"Sebastian Becker. 2023. Maximum Token length in GPT-4. https:\/\/community.openai.com\/t\/maximum-token-length-in-gpt-4\/385914. Accessed: 2025-07-01.","key":"e_1_2_1_5_1"},{"doi-asserted-by":"publisher","key":"e_1_2_1_6_1","DOI":"10.1191\/1478088706qp063oa"},{"doi-asserted-by":"publisher","key":"e_1_2_1_7_1","DOI":"10.1037\/13620-004"},{"doi-asserted-by":"publisher","key":"e_1_2_1_8_1","DOI":"10.1080\/2159676X.2019.1628806"},{"doi-asserted-by":"publisher","key":"e_1_2_1_9_1","DOI":"10.1080\/14780887.2020.1769238"},{"doi-asserted-by":"publisher","key":"e_1_2_1_10_1","DOI":"10.1037\/qup0000196"},{"doi-asserted-by":"publisher","key":"e_1_2_1_11_1","DOI":"10.1007\/s11135-021-01182-y"},{"doi-asserted-by":"publisher","key":"e_1_2_1_12_1","DOI":"10.1145\/3491102.3501998"},{"doi-asserted-by":"crossref","unstructured":"Karen A Campbell Elizabeth Orr Pamela Durepos Linda Nguyen Lin Li Carly Whitmore Paige Gehrke Leslie Graham and Susan M Jack. 2021. Reflexive thematic analysis for applied qualitative health research. The Qualitative Report 26 6 (2021) 2011-2028.","key":"e_1_2_1_13_1","DOI":"10.46743\/2160-3715\/2021.5010"},{"volume-title":"Introduction to text visualization","author":"Cao Nan","unstructured":"Nan Cao and Weiwei Cui. 2016. Introduction to text visualization. Vol. 1. Springer.","key":"e_1_2_1_14_1"},{"doi-asserted-by":"publisher","key":"e_1_2_1_15_1","DOI":"10.1016\/j.cptl.2018.03.019"},{"key":"e_1_2_1_16_1","volume-title":"LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding. arXiv preprint arXiv:2306.14924","author":"Chew Robert","year":"2023","unstructured":"Robert Chew, John Bollenbacher, Michael Wenger, Jessica Speer, and Annice Kim. 2023. LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding. arXiv preprint arXiv:2306.14924 (2023). doi:10.48550\/ arXiv.2306.14924"},{"key":"e_1_2_1_17_1","volume-title":"How to Use Artificial Intelligence (AI) as a Resource, Methodological and Analysis Tool in Qualitative Research? The Qualitative Report","author":"Christou Prokopis A.","year":"2023","unstructured":"Prokopis A. Christou. 2023. How to Use Artificial Intelligence (AI) as a Resource, Methodological and Analysis Tool in Qualitative Research? The Qualitative Report (2023)."},{"doi-asserted-by":"publisher","key":"e_1_2_1_18_1","DOI":"10.1177\/16094069231193593"},{"volume-title":"Successful qualitative research: A practical guide for beginners","author":"Clarke Victoria","unstructured":"Victoria Clarke and Virginia Braun. 2013. Successful qualitative research: A practical guide for beginners. Sage publications ltd. http:\/\/digital.casalini.it\/9781446281024","key":"e_1_2_1_19_1"},{"unstructured":"Dovetail. 2023. Analyze data in minutes not hours. https:\/\/dovetail.com\/analysis\/.","key":"e_1_2_1_20_1"},{"doi-asserted-by":"publisher","key":"e_1_2_1_21_1","DOI":"10.48550\/arXiv.2401.15170"},{"doi-asserted-by":"publisher","key":"e_1_2_1_22_1","DOI":"10.1145\/2207676.2207741"},{"doi-asserted-by":"publisher","key":"e_1_2_1_23_1","DOI":"10.1145\/3479856"},{"doi-asserted-by":"publisher","key":"e_1_2_1_24_1","DOI":"10.1098\/rsta.2016.0112"},{"doi-asserted-by":"publisher","key":"e_1_2_1_25_1","DOI":"10.1007\/s11625-019-00701-z"},{"doi-asserted-by":"publisher","key":"e_1_2_1_26_1","DOI":"10.1145\/3617362"},{"doi-asserted-by":"publisher","key":"e_1_2_1_27_1","DOI":"10.1145\/3613904.3642002"},{"doi-asserted-by":"publisher","key":"e_1_2_1_28_1","DOI":"10.1145\/3544548.3581352"},{"doi-asserted-by":"publisher","key":"e_1_2_1_29_1","DOI":"10.1609\/aaai.v38i9.28868"},{"key":"e_1_2_1_30_1","volume-title":"Snowball sampling. The annals of mathematical statistics","author":"Goodman Leo A","year":"1961","unstructured":"Leo A Goodman. 1961. Snowball sampling. The annals of mathematical statistics (1961), 148-170. http:\/\/www.jstor.org\/stable\/2237615"},{"doi-asserted-by":"publisher","key":"e_1_2_1_31_1","DOI":"10.1021\/acs.jchemed.3c00505"},{"doi-asserted-by":"publisher","key":"e_1_2_1_32_1","DOI":"10.1177\/16094069231201504"},{"doi-asserted-by":"publisher","key":"e_1_2_1_33_1","DOI":"10.1109\/INFVIS.2000.885098"},{"doi-asserted-by":"publisher","key":"e_1_2_1_34_1","DOI":"10.1177\/16094069251322346"},{"doi-asserted-by":"publisher","key":"e_1_2_1_35_1","DOI":"10.1007\/s10676-008-9167-5"},{"doi-asserted-by":"publisher","key":"e_1_2_1_36_1","DOI":"10.1080\/07421222.1998.11518185"},{"doi-asserted-by":"publisher","key":"e_1_2_1_37_1","DOI":"10.1145\/3449168"},{"doi-asserted-by":"publisher","key":"e_1_2_1_38_1","DOI":"10.48550\/arXiv.2405.06919"},{"doi-asserted-by":"publisher","key":"e_1_2_1_39_1","DOI":"10.48550\/arXiv.2403.06607"},{"doi-asserted-by":"publisher","key":"e_1_2_1_40_1","DOI":"10.48550\/arXiv.2401.12273"},{"doi-asserted-by":"publisher","key":"e_1_2_1_41_1","DOI":"10.1080\/10447318.2020.1727262"},{"doi-asserted-by":"publisher","unstructured":"Yuanchun Li Hao Wen Weijun Wang Xiangyu Li Yizhen Yuan Guohong Liu Jiacheng Liu Wenxing Xu Xiang Wang Yi Sun et al. 2024. Personal llm agents: Insights and survey about the capability efficiency and security. arXiv preprint arXiv:2401.05459 (2024). doi:10.48550\/arXiv.2401.05459","key":"e_1_2_1_42_1","DOI":"10.48550\/arXiv.2401.05459"},{"doi-asserted-by":"publisher","key":"e_1_2_1_43_1","DOI":"10.1109\/TVCG.2018.2834341"},{"doi-asserted-by":"publisher","key":"e_1_2_1_44_1","DOI":"10.1145\/3173574.3173922"},{"doi-asserted-by":"publisher","key":"e_1_2_1_45_1","DOI":"10.2307\/2392365"},{"doi-asserted-by":"publisher","key":"e_1_2_1_46_1","DOI":"10.18653\/v1\/2023.findings-emnlp.461"},{"doi-asserted-by":"publisher","key":"e_1_2_1_47_1","DOI":"10.1057\/ejis.2016.2"},{"doi-asserted-by":"publisher","key":"e_1_2_1_48_1","DOI":"10.1177\/1609406917733847"},{"doi-asserted-by":"publisher","key":"e_1_2_1_49_1","DOI":"10.48550\/arXiv.2305.13014"},{"key":"e_1_2_1_50_1","first-page":"36","article-title":"Critical thinking: The art of Socratic questioning","volume":"31","author":"Paul Richard","year":"2007","unstructured":"Richard Paul and Linda Elder. 2007. Critical thinking: The art of Socratic questioning. Journal of developmental education 31, 1 (2007), 36. https:\/\/www.proquest.com\/scholarly-journals\/critical-thinking-art-socratic-questioning\/docview\/228487383\/se-2","journal-title":"Journal of developmental education"},{"doi-asserted-by":"publisher","key":"e_1_2_1_51_1","DOI":"10.1108\/TPM-11-2018-0070"},{"doi-asserted-by":"publisher","key":"e_1_2_1_52_1","DOI":"10.1145\/3411763.3451760"},{"doi-asserted-by":"publisher","key":"e_1_2_1_53_1","DOI":"10.1145\/3411764.3445591"},{"doi-asserted-by":"publisher","key":"e_1_2_1_54_1","DOI":"10.48550\/arXiv.2306.01943"},{"doi-asserted-by":"publisher","key":"e_1_2_1_55_1","DOI":"10.4018\/IJSPPC.325475"},{"doi-asserted-by":"publisher","key":"e_1_2_1_56_1","DOI":"10.1007\/s10676-006-9112-4"},{"doi-asserted-by":"publisher","key":"e_1_2_1_57_1","DOI":"10.1145\/3411764.3445458"},{"doi-asserted-by":"publisher","key":"e_1_2_1_58_1","DOI":"10.5255\/UKDA-SN-6226-6"},{"doi-asserted-by":"publisher","key":"e_1_2_1_59_1","DOI":"10.1145\/3491102.3517732"},{"doi-asserted-by":"publisher","key":"e_1_2_1_60_1","DOI":"10.48550\/arXiv.2302.13971"},{"doi-asserted-by":"publisher","key":"e_1_2_1_61_1","DOI":"10.1109\/ICIET56899.2023.10111115"},{"doi-asserted-by":"publisher","key":"e_1_2_1_62_1","DOI":"10.1145\/3581754.3584136"},{"doi-asserted-by":"publisher","key":"e_1_2_1_63_1","DOI":"10.1145\/3544548.3581388"},{"doi-asserted-by":"publisher","key":"e_1_2_1_64_1","DOI":"10.48550\/arXiv.2310.07061"},{"doi-asserted-by":"publisher","key":"e_1_2_1_65_1","DOI":"10.48550\/arXiv.2309.10771"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3757675","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T02:01:59Z","timestamp":1760666519000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3757675"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,16]]},"references-count":65,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,10,18]]}},"alternative-id":["10.1145\/3757675"],"URL":"https:\/\/doi.org\/10.1145\/3757675","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2025,10,16]]},"assertion":[{"value":"2025-10-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}