{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T02:40:23Z","timestamp":1760668823635,"version":"build-2065373602"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T00:00:00Z","timestamp":1760572800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N00014-21-1-2154"],"award-info":[{"award-number":["N00014-21-1-2154"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,10,18]]},"abstract":"<jats:p>Millions of online communities are governed by volunteer moderators, who shape their communities by setting and enforcing rules, recruiting additional moderators, and participating in the community themselves. These moderators must regularly make decisions about how to govern, yet measuring the 'success' of governance is complex and nuanced, making it challenging to determine what governance strategies are most successful. Furthermore, prior work has shown that communities have differing values, suggesting that 'one-size-fits-all' approaches to governance are unlikely to serve all communities well. In this work, we assess governance practices on reddit by classifying the sentiment of community members' public discussion of their own moderators. We label 1.89 million posts and comments made on reddit over an 18 month period. We relate these perceptions to characteristics of community governance, and to different actions that community moderators take. We identify types of communities where moderators are perceived particularly positively and negatively, and highlight promising strategies for moderator teams. Amongst other findings, we show that strict rule enforcement is linked to more favorable perceptions of moderators of communities dedicated to certain topics, such as news communities, than others. We investigate what kinds of moderators are associated with improved community perceptions upon their addition to a mod team, and find that moderators who are active community members before and during their mod tenures are seen more favorably. We make our models, anonymized datasets, and code public.<\/jats:p>","DOI":"10.1145\/3757644","type":"journal-article","created":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T16:59:10Z","timestamp":1760633950000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Perceptions of Moderators as a Large-Scale Measure of Online Community Governance"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2106-9889","authenticated-orcid":false,"given":"Galen Cassebeer","family":"Weld","sequence":"first","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-9237-3248","authenticated-orcid":false,"given":"Leon","family":"Leibmann","sequence":"additional","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9462-9835","authenticated-orcid":false,"given":"Amy X.","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4793-2289","authenticated-orcid":false,"given":"Tim","family":"Althoff","sequence":"additional","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,10,16]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"[n. d.]. Internet Archive. https:\/\/archive.org\/"},{"key":"e_1_2_1_2_1","volume-title":"Statistical Modeling of Harassment against Reddit Moderators. WWW Companion","author":"Almerekhi Hind","year":"2020","unstructured":"Hind Almerekhi, Haewoon Kwak, and Bernard Jim Jansen. 2020. Statistical Modeling of Harassment against Reddit Moderators. WWW Companion (2020). https:\/\/api.semanticscholar.org\/CorpusID:218522526"},{"key":"e_1_2_1_3_1","volume-title":"Almond and Sidney Verba","author":"Gabriel","year":"1989","unstructured":"Gabriel A. Almond and Sidney Verba. 1989. The civic culture: Political attitudes and democracy in Five nations. Sage Publications."},{"key":"e_1_2_1_4_1","volume-title":"Quantifying dose response relationships between physical activity and health using propensity scores. NIPS ML4H","author":"Althoff Tim","year":"2016","unstructured":"Tim Althoff, R Sosic, JL Hicks, AC King, SL Delp, and J Leskovec. 2016. Quantifying dose response relationships between physical activity and health using propensity scores. NIPS ML4H (2016)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.6607"},{"key":"e_1_2_1_6_1","volume-title":"Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversations. TheWebConf","author":"Bao Jiajun","year":"2021","unstructured":"Jiajun Bao, J. Wu, Yiming Zhang, Eshwar Chandrasekharan, and David Jurgens. 2021. Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversations. TheWebConf (2021). https:\/\/api.semanticscholar.org\/CorpusID:231933671"},{"key":"e_1_2_1_7_1","volume-title":"The Pushshift Reddit Dataset. arXiv:2001.08435","author":"Baumgartner Jason","year":"2020","unstructured":"Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The Pushshift Reddit Dataset. arXiv:2001.08435 (2020)."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jeconom.2020.12.001"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3490499"},{"key":"e_1_2_1_10_1","volume-title":"The Internet's Hidden Rules: An Empirical Study of Reddit Norm Violations at Micro, Meso, and Macro Scales. CSCW","author":"Chandrasekharan Eshwar","year":"2018","unstructured":"Eshwar Chandrasekharan, Mattia Samory, Shagun Jhaver, Hunter Charvat, Amy Bruckman, Cliff Lampe, Jacob Eisenstein, and Eric Gilbert. 2018. The Internet's Hidden Rules: An Empirical Study of Reddit Norm Violations at Micro, Meso, and Macro Scales. CSCW (2018)."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488416"},{"key":"e_1_2_1_12_1","volume-title":"QLoRA: Efficient Finetuning of Quantized LLMs. ArXiv abs\/2305.14314","author":"Dettmers Tim","year":"2023","unstructured":"Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient Finetuning of Quantized LLMs. ArXiv abs\/2305.14314 (2023). https:\/\/api.semanticscholar.org\/CorpusID:258841328"},{"key":"e_1_2_1_13_1","volume-title":"Joshua McCann, Kyle Frye, and Jed R. Brubaker.","author":"Fiesler Casey","year":"2018","unstructured":"Casey Fiesler, Jialun Aaron Jiang, Joshua McCann, Kyle Frye, and Jed R. Brubaker. 2018. Reddit Rules! Characterizing an Ecosystem of Governance. ICWSM (2018). https:\/\/api.semanticscholar.org\/CorpusID:49404607"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1177\/2056305118763366"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0216335"},{"key":"e_1_2_1_16_1","volume-title":"Hutto and Eric Gilbert","author":"Clayton","year":"2014","unstructured":"Clayton J. Hutto and Eric Gilbert. 2014. VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. ICWSM (2014). https:\/\/api.semanticscholar.org\/CorpusID:12233345"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3479606"},{"key":"e_1_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Sohyeon Hwang and Aaron Shaw. 2022. Rules and Rule-Making in the Five Largest Wikipedias. In ICWSM. https:\/\/api.semanticscholar.org\/CorpusID:249892728","DOI":"10.1609\/icwsm.v17i1.27319"},{"key":"e_1_2_1_19_1","first-page":"1","article-title":"Does Transparency in Moderation Really Matter","volume":"3","author":"Jhaver Shagun","year":"2019","unstructured":"Shagun Jhaver, Amy Bruckman, and Eric Gilbert. 2019. Does Transparency in Moderation Really Matter? CSCW 3 (2019), 1 - 27. https:\/\/api.semanticscholar.org\/CorpusID:203558216","journal-title":"CSCW"},{"key":"e_1_2_1_20_1","doi-asserted-by":"crossref","unstructured":"Shagun Jhaver Seth Frey and Amy Zhang. 2023. Decentralizing Platform Power: A Design Space of Multi-level Governance in Online Social Platforms. arXiv:2108.12529 [cs.HC]","DOI":"10.1177\/20563051231207857"},{"key":"e_1_2_1_21_1","doi-asserted-by":"crossref","unstructured":"Shagun Jhaver Himanshu Rathi and Koustuv Saha. 2023. Bystanders of Online Moderation: Examining the Effects of Witnessing Post-Removal Explanations. arXiv:2309.08361 [cs.HC]","DOI":"10.1145\/3613904.3642204"},{"key":"e_1_2_1_22_1","volume-title":"SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning. ArXiv abs\/2307.10234","author":"Kheiri Kiana","year":"2023","unstructured":"Kiana Kheiri and Hamid Karimi. 2023. SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning. ArXiv abs\/2307.10234 (2023). https:\/\/api.semanticscholar.org\/CorpusID:259991148"},{"key":"e_1_2_1_23_1","first-page":"1","article-title":"Measuring User-Moderator Alignment on r\/ChangeMyView","volume":"7","author":"Koshy Vinay","year":"2023","unstructured":"Vinay Koshy, Tanvi Bajpai, Eshwar Chandrasekharan, Hari Sundaram, and Karrie Karahalios. 2023. Measuring User-Moderator Alignment on r\/ChangeMyView. CSCW 7 (2023), 1 - 36. https:\/\/api.semanticscholar.org\/CorpusID:263621128","journal-title":"CSCW"},{"volume-title":"Building successful online communities: Evidence-based social design","author":"Kraut Robert E","key":"e_1_2_1_24_1","unstructured":"Robert E Kraut and Paul Resnick. 2012. Building successful online communities: Evidence-based social design. Mit Press."},{"key":"e_1_2_1_25_1","volume-title":"Watch Your Language: Large Language Models and Content Moderation. In International Conference on Web and Social Media. https:\/\/api.semanticscholar.org\/CorpusID:262825535","author":"Kumar Deepak","year":"2023","unstructured":"Deepak Kumar, Yousef Anees AbuHashem, and Zakir Durumeric. 2023. Watch Your Language: Large Language Models and Content Moderation. In International Conference on Web and Social Media. https:\/\/api.semanticscholar.org\/CorpusID:262825535"},{"key":"e_1_2_1_26_1","volume-title":"Koch","author":"Richard Landis J.","year":"1977","unstructured":"J. Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics (1977)."},{"key":"e_1_2_1_27_1","volume-title":"All That's Happening behind the Scenes: Putting the Spotlight on Volunteer Moderator Labor in Reddit. ICWSM","author":"Li Hanlin","year":"2022","unstructured":"Hanlin Li, Brent J. Hecht, and Stevie Chancellor. 2022. All That's Happening behind the Scenes: Putting the Spotlight on Volunteer Moderator Labor in Reddit. ICWSM (2022). https:\/\/api.semanticscholar.org\/CorpusID:249191685"},{"key":"e_1_2_1_28_1","volume-title":"Measuring the Monetary Value of Online Volunteer Work. ICWSM","author":"Li Hanlin","year":"2022","unstructured":"Hanlin Li, Brent J. Hecht, and Stevie Chancellor. 2022. Measuring the Monetary Value of Online Volunteer Work. ICWSM (2022)."},{"key":"e_1_2_1_29_1","volume-title":"Bernstein","author":"Lin Zhiyuan Jerry","year":"2017","unstructured":"Zhiyuan Jerry Lin, Niloufar Salehi, Bowen Yao, Yiqi Chen, and Michael S. Bernstein. 2017. Better When ItWas Smaller? Community Content and Behavior After Massive Growth. In ICWSM. https:\/\/api.semanticscholar.org\/CorpusID:2524208"},{"key":"e_1_2_1_30_1","volume-title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv abs\/1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv abs\/1907.11692 (2019). https:\/\/api.semanticscholar.org\/CorpusID:198953378"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1177\/2056305119836778"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1813486116"},{"key":"e_1_2_1_33_1","volume-title":"CivilServant: Community-Led Experiments in Platform Governance. CHI","author":"Nathan Matias J.","year":"2018","unstructured":"J. Nathan Matias and Merry Ember Mou. 2018. CivilServant: Community-Led Experiments in Platform Governance. CHI (2018). https:\/\/api.semanticscholar.org\/CorpusID:4809150"},{"key":"e_1_2_1_34_1","unstructured":"OpenAI et al. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]"},{"key":"e_1_2_1_35_1","volume-title":"Terveen","author":"Panciera Katherine A.","year":"2009","unstructured":"Katherine A. Panciera, Aaron L Halfaker, and Loren G. Terveen. 2009. Wikipedians are born, not made: a study of power editors on Wikipedia. GROUP (2009). https:\/\/api.semanticscholar.org\/CorpusID:6286454"},{"key":"e_1_2_1_36_1","volume-title":"Studying Reddit: A Systematic Overview of Disciplines, Approaches, Methods, and Ethics","author":"Proferes Nicholas","year":"2021","unstructured":"Nicholas Proferes, Naiyan Jones, Sarah A. Gilbert, Casey Fiesler, and Michael Zimmer. 2021. Studying Reddit: A Systematic Overview of Disciplines, Approaches, Methods, and Ethics. Social Media Society 7 (2021). https:\/\/api.semanticscholar.org\/CorpusID:235724916"},{"key":"e_1_2_1_37_1","unstructured":"Inc. Reddit. 2025. Moderator Reserves Overview. https:\/\/support.reddithelp.com\/hc\/en-us\/articles\/15484270707092-Moderator-Reserves-Overview"},{"key":"e_1_2_1_38_1","volume-title":"Evolution of Rules in Reddit Communities. CSCW","author":"Reddy Harita","year":"2023","unstructured":"Harita Reddy and Eshwar Chandrasekharan. 2023. Evolution of Rules in Reddit Communities. CSCW (2023). https:\/\/api.semanticscholar.org\/CorpusID:264039135"},{"key":"e_1_2_1_39_1","volume-title":"Gianluca Stringhini, and Robert West.","author":"Ribeiro Manoel Horta","year":"2020","unstructured":"Manoel Horta Ribeiro, Shagun Jhaver, Savvas Zannettou, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, and Robert West. 2020. Do Platform Migrations Compromise Content Moderation? Evidence from r\/The_Donald and r\/Incels. CSCW (2020). https:\/\/api.semanticscholar.org\/CorpusID:263873469"},{"key":"e_1_2_1_40_1","first-page":"1","article-title":"Who Moderates on Twitch and What Do They Do","volume":"7","author":"Seering Joseph","year":"2022","unstructured":"Joseph Seering and Sanjay Ram Kairam. 2022. Who Moderates on Twitch and What Do They Do? GROUP 7 (2022), 1 - 18. https:\/\/api.semanticscholar.org\/CorpusID:255226737","journal-title":"GROUP"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444818821316"},{"key":"e_1_2_1_42_1","volume-title":"Content Removal as a Moderation Strategy. CSCW","author":"Srinivasan Kumar Bhargav","year":"2019","unstructured":"Kumar Bhargav Srinivasan, Cristian Danescu-Niculescu-Mizil, Lillian Lee, and Chenhao Tan. 2019. Content Removal as a Moderation Strategy. CSCW (2019). https:\/\/api.semanticscholar.org\/CorpusID:203617040"},{"key":"e_1_2_1_43_1","first-page":"809","article-title":"Behavior Change in Response to Subreddit Bans and External Events","volume":"8","author":"Thomas Pamela Bilo","year":"2021","unstructured":"Pamela Bilo Thomas, Daniel Riehm, Maria Glenski, and TimWeninger. 2021. Behavior Change in Response to Subreddit Bans and External Events. IEEE TCSS 8 (2021), 809-818. https:\/\/api.semanticscholar.org\/CorpusID:230770317","journal-title":"IEEE TCSS"},{"key":"e_1_2_1_44_1","unstructured":"Hugo Touvron et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288 [cs.CL]"},{"key":"e_1_2_1_45_1","volume-title":"Quantifying social organization and political polarization in online platforms. Nature","author":"Ashton Anderson IsaacWaller","year":"2020","unstructured":"IsaacWaller and Ashton Anderson. 2020. Quantifying social organization and political polarization in online platforms. Nature (2020). https:\/\/api.semanticscholar.org\/CorpusID:236469369"},{"key":"e_1_2_1_46_1","volume-title":"Political Bias and Factualness in News Sharing Across more then 100, 000 Online Communities. ICWSM","author":"Weld Galen Cassebeer","year":"2021","unstructured":"Galen Cassebeer Weld, Maria Glenski, and Tim Althoff. 2021. Political Bias and Factualness in News Sharing Across more then 100, 000 Online Communities. ICWSM (2021). https:\/\/api.semanticscholar.org\/CorpusID:231942492"},{"key":"e_1_2_1_47_1","volume-title":"What Makes Online Communities 'Better'? Measuring Values, Consensus, and Conflict across Thousands of Subreddits. ICWSM","author":"Weld Galen Cassebeer","year":"2022","unstructured":"Galen Cassebeer Weld, Amy X. Zhang, and Tim Althoff. 2022. What Makes Online Communities 'Better'? Measuring Values, Consensus, and Conflict across Thousands of Subreddits. ICWSM (2022)."},{"key":"e_1_2_1_48_1","volume-title":"Making Online Communities 'Better': A Taxonomy of Community Values on Reddit. ICWSM","author":"Weld Galen Cassebeer","year":"2024","unstructured":"Galen Cassebeer Weld, Amy X. Zhang, and Tim Althoff. 2024. Making Online Communities 'Better': A Taxonomy of Community Values on Reddit. ICWSM (2024)."},{"key":"e_1_2_1_49_1","volume-title":"Bernstein","author":"Zhang Amy X.","year":"2020","unstructured":"Amy X. Zhang, Grant Hugh, and Michael S. Bernstein. 2020. PolicyKit: Building Governance in Online Communities. UIST (2020)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557213"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3757644","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3757644","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T01:59:11Z","timestamp":1760666351000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3757644"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,16]]},"references-count":50,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,10,18]]}},"alternative-id":["10.1145\/3757644"],"URL":"https:\/\/doi.org\/10.1145\/3757644","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2025,10,16]]},"assertion":[{"value":"2025-10-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}