{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T10:28:23Z","timestamp":1773311303704,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":56,"publisher":"ACM","license":[{"start":{"date-parts":[[2025,4,22]],"date-time":"2025-04-22T00:00:00Z","timestamp":1745280000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/https:\/\/doi.org\/10.13039\/501100003524","name":"Ministry of Business, Innovation and Employment","doi-asserted-by":"publisher","award":["UOAX1912"],"award-info":[{"award-number":["UOAX1912"]}],"id":[{"id":"10.13039\/https:\/\/doi.org\/10.13039\/501100003524","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,4,22]]},"DOI":"10.1145\/3696410.3714595","type":"proceedings-article","created":{"date-parts":[[2025,4,22]],"date-time":"2025-04-22T22:57:28Z","timestamp":1745362648000},"page":"649-658","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Thematic-LM: A LLM-based Multi-agent System for Large-scale Thematic Analysis"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-8396-673X","authenticated-orcid":false,"given":"Tingrui","family":"Qiao","sequence":"first","affiliation":[{"name":"University of Auckland, Auckland, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9210-7651","authenticated-orcid":false,"given":"Caroline","family":"Walker","sequence":"additional","affiliation":[{"name":"University of Auckland, Auckland, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7083-9088","authenticated-orcid":false,"given":"Chris","family":"Cunningham","sequence":"additional","affiliation":[{"name":"Massey University, Wellington, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7256-4049","authenticated-orcid":false,"given":"Yun Sing","family":"Koh","sequence":"additional","affiliation":[{"name":"University of Auckland, Auckland, New Zealand"}]}],"member":"320","published-online":{"date-parts":[[2025,4,22]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2022.102131"},{"key":"e_1_3_2_1_2_1","volume-title":"Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.","author":"Achiam Josh","year":"2023","unstructured":"Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1111\/nyas.14337"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-019-01438-z"},{"key":"e_1_3_2_1_5_1","volume-title":"Using thematic analysis in psychology. Qualitative research in psychology","author":"Braun Virginia","year":"2006","unstructured":"Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology, Vol. 3, 2 (2006), 77--101."},{"key":"e_1_3_2_1_6_1","volume-title":"Thematic analysis","author":"Braun Virginia","unstructured":"Virginia Braun and Victoria Clarke. 2012. Thematic analysis.American Psychological Association."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2019.2956957"},{"key":"e_1_3_2_1_9_1","volume-title":"Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201","author":"Chan Chi-Min","year":"2023","unstructured":"Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201 (2023)."},{"key":"e_1_3_2_1_10_1","volume-title":"AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=EHg5GDnyq1","author":"Chen Weize","year":"2024","unstructured":"Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2024. AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=EHg5GDnyq1"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2019.04.002"},{"key":"e_1_3_2_1_12_1","volume-title":"LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis. In The 2023 Conference on Empirical Methods in Natural Language Processing. https:\/\/openreview.net\/forum?id=NuMemgzPYT","author":"Dai Shih-Chieh","year":"2023","unstructured":"Shih-Chieh Dai, Aiping Xiong, and Lun-Wei Ku. 2023. LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis. In The 2023 Conference on Empirical Methods in Natural Language Processing. https:\/\/openreview.net\/forum?id=NuMemgzPYT"},{"key":"e_1_3_2_1_13_1","volume-title":"Performing an inductive thematic analysis of semi-structured interviews with a large language model: an exploration and provocation on the limits of the approach. Social Science Computer Review","author":"Paoli Stefano De","year":"2023","unstructured":"Stefano De Paoli. 2023. Performing an inductive thematic analysis of semi-structured interviews with a large language model: an exploration and provocation on the limits of the approach. Social Science Computer Review (2023), 08944393231220483."},{"key":"e_1_3_2_1_14_1","volume-title":"Ana Sanchez Chico, and Stefanie Stantcheva","author":"Dechezlepr\u00eatre Antoine","year":"2022","unstructured":"Antoine Dechezlepr\u00eatre, Adrien Fabre, Tobias Kruse, Bluebery Planterose, Ana Sanchez Chico, and Stefanie Stantcheva. 2022. Fighting climate change: International attitudes toward climate policies. Technical Report. National Bureau of Economic Research."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10584-020-02791-6"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Jakub Dr\u00e1pal Hannes Westermann Jaromir Savelka et al. 2023. Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies.. In JURIX. 197--206.","DOI":"10.2139\/ssrn.4617116"},{"key":"e_1_3_2_1_17_1","volume-title":"Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325","author":"Du Yilun","year":"2023","unstructured":"Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325 (2023)."},{"key":"e_1_3_2_1_18_1","first-page":"103","article-title":"Thematic analysis:: the 'good', the 'bad'and the 'ugly","volume":"11","author":"Finlay Linda","year":"2021","unstructured":"Linda Finlay. 2021. Thematic analysis:: the 'good', the 'bad'and the 'ugly'. European Journal for Qualitative Research in Psychotherapy, Vol. 11 (2021), 103--116.","journal-title":"European Journal for Qualitative Research in Psychotherapy"},{"key":"e_1_3_2_1_19_1","volume-title":"Ibrahim Abaker Targio Hashem, and Ejaz Ahmed","author":"Ghani Norjihan Abdul","year":"2019","unstructured":"Norjihan Abdul Ghani, Suraya Hamid, Ibrahim Abaker Targio Hashem, and Ejaz Ahmed. 2019. Social media big data analytics: A survey. Computers in Human behavior, Vol. 101 (2019), 417--428."},{"key":"e_1_3_2_1_20_1","volume-title":"BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794","author":"Grootendorst Maarten","year":"2022","unstructured":"Maarten Grootendorst. 2022. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794 (2022)."},{"key":"e_1_3_2_1_21_1","volume-title":"MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=VtmBAGCN7o","author":"Hong Sirui","year":"2024","unstructured":"Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and J\u00fcrgen Schmidhuber. 2024. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=VtmBAGCN7o"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.825"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_3_2_1_24_1","volume-title":"BERT: a review of applications in natural language processing and understanding. arXiv preprint arXiv:2103.11943","author":"Koroteev Mikhail V","year":"2021","unstructured":"Mikhail V Koroteev. 2021. BERT: a review of applications in natural language processing and understanding. arXiv preprint arXiv:2103.11943 (2021)."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1080\/13814788.2017.1375092"},{"key":"e_1_3_2_1_26_1","volume-title":"Peter Ford Dominey, and Pierre-Yves Oudeyer","author":"Kova\u010d Grgur","year":"2023","unstructured":"Grgur Kova\u010d, R\u00e9my Portelas, Peter Ford Dominey, and Pierre-Yves Oudeyer. 2023. The socialai school: Insights from developmental psychology towards artificial socio-cultural agents. arXiv preprint arXiv:2307.07871 (2023)."},{"key":"e_1_3_2_1_27_1","volume-title":"The trustworthiness of content analysis. The application of content analysis in nursing science research","author":"Kyng\u00e4s Helvi","year":"2020","unstructured":"Helvi Kyng\u00e4s, Maria K\u00e4\u00e4ri\u00e4inen, and Satu Elo. 2020. The trustworthiness of content analysis. The application of content analysis in nursing science research (2020), 41--48."},{"key":"e_1_3_2_1_28_1","volume-title":"Thirty-seventh Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=3IyL2XWDkG","author":"Li Guohao","year":"2023","unstructured":"Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. CAMEL: Communicative Agents for ''Mind'' Exploration of Large Language Model Society. In Thirty-seventh Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=3IyL2XWDkG"},{"key":"e_1_3_2_1_29_1","volume-title":"Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118","author":"Liang Tian","year":"2023","unstructured":"Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118 (2023)."},{"key":"e_1_3_2_1_30_1","volume-title":"ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74--81. https:\/\/aclanthology.org\/W04--1013"},{"key":"e_1_3_2_1_31_1","volume-title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach. arxiv","author":"Liu Yinhan","year":"1907","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arxiv: 1907.11692 [cs.CL] https:\/\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00382-004-0392-2"},{"key":"e_1_3_2_1_33_1","volume-title":"Interrater reliability: the kappa statistic. Biochemia medica","author":"McHugh Mary L","year":"2012","unstructured":"Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, Vol. 22, 3 (2012), 276--282."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00205"},{"key":"e_1_3_2_1_35_1","volume-title":"Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426","author":"McInnes Leland","year":"2018","unstructured":"Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018)."},{"key":"e_1_3_2_1_36_1","volume-title":"Maslow's hierarchy of needs. Simply psychology","author":"McLeod Saul","year":"2007","unstructured":"Saul McLeod. 2007. Maslow's hierarchy of needs. Simply psychology, Vol. 1, 1--18 (2007)."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASONAM.2012.164"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1177\/1609406917733847"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-88708-6_1"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586183.3606763"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.3390\/su12197836"},{"key":"e_1_3_2_1_42_1","volume-title":"Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint arXiv:1908.10084","author":"Reimers N","year":"2019","unstructured":"N Reimers. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint arXiv:1908.10084 (2019)."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2324796.2324824"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1123\/jtpe.2017-0084"},{"key":"e_1_3_2_1_45_1","volume-title":"Leveraging social media networks for classification. Data mining and knowledge discovery","author":"Tang Lei","year":"2011","unstructured":"Lei Tang and Huan Liu. 2011. Leveraging social media networks for classification. Data mining and knowledge discovery, Vol. 23 (2011), 447--478."},{"key":"e_1_3_2_1_46_1","volume-title":"Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971","author":"Touvron Hugo","year":"2023","unstructured":"Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth\u00e9e Lacroix, Baptiste Rozi\u00e8re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)."},{"key":"e_1_3_2_1_47_1","volume-title":"Dreaddit: A reddit dataset for stress analysis in social media. arXiv preprint arXiv:1911.00133","author":"Turcan Elsbeth","year":"2019","unstructured":"Elsbeth Turcan and Kathleen McKeown. 2019. Dreaddit: A reddit dataset for stress analysis in social media. arXiv preprint arXiv:1911.00133 (2019)."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1215\/00138282-55.1-2.153"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1002\/wcc.603"},{"key":"e_1_3_2_1_50_1","volume-title":"Autogen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155","author":"Wu Qingyun","year":"2023","unstructured":"Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155 (2023)."},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.508"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-018-1236-4"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2310.02124"},{"key":"e_1_3_2_1_54_1","volume-title":"CompeteAI: Understanding the Competition Dynamics of Large Language Model-based Agents. In Forty-first International Conference on Machine Learning.","author":"Zhao Qinlin","year":"2024","unstructured":"Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, and Xing Xie. 2024. CompeteAI: Understanding the Competition Dynamics of Large Language Model-based Agents. In Forty-first International Conference on Machine Learning."},{"key":"e_1_3_2_1_55_1","first-page":"46595","article-title":"Judging llm-as-a-judge with mt-bench and chatbot arena","volume":"36","author":"Zheng Lianmin","year":"2023","unstructured":"Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, Vol. 36 (2023), 46595--46623.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106426.3106459"}],"event":{"name":"WWW '25: The ACM Web Conference 2025","location":"Sydney NSW Australia","acronym":"WWW '25","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web"]},"container-title":["Proceedings of the ACM on Web Conference 2025"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696410.3714595","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3696410.3714595","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:33Z","timestamp":1750295913000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696410.3714595"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,22]]},"references-count":56,"alternative-id":["10.1145\/3696410.3714595","10.1145\/3696410"],"URL":"https:\/\/doi.org\/10.1145\/3696410.3714595","relation":{},"subject":[],"published":{"date-parts":[[2025,4,22]]},"assertion":[{"value":"2025-04-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}