{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T23:54:24Z","timestamp":1775087664593,"version":"3.50.1"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW","license":[{"start":{"date-parts":[[2019,11,7]],"date-time":"2019-11-07T00:00:00Z","timestamp":1573084800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["SES-1741441, IIS-1750615"],"award-info":[{"award-number":["SES-1741441, IIS-1750615"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2019,11,7]]},"abstract":"<jats:p>Moderators of online communities often employ comment deletion as a tool. We ask here whether, beyond the positive effects of shielding a community from undesirable content, does comment removal actually cause the behavior of the comment's author to improve? We examine this question in a particularly well-moderated community, the ChangeMyView subreddit.<\/jats:p>\n          <jats:p>The standard analytic approach of interrupted time-series analysis unfortunately cannot answer this question of causality because it fails to distinguish the effect of having made a non-compliant comment from the effect of being subjected to moderator removal of that comment. We therefore leverage a \"delayed feedback\" approach based on the observation that some users may remain active between the time when they posted the non-compliant comment and the time when that comment is deleted. Applying this approach to such users, we reveal the causal role of comment deletion in reducing immediate noncompliance rates, although we do not find evidence of it having a causal role in inducing other behavior improvements. Our work thus empirically demonstrates both the promise and some potential limits of content removal as a positive moderation strategy, and points to future directions for identifying causal effects from observational data.<\/jats:p>","DOI":"10.1145\/3359265","type":"journal-article","created":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T13:40:21Z","timestamp":1573220421000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":61,"title":["Content Removal as a Moderation Strategy"],"prefix":"10.1145","volume":"3","author":[{"given":"Kumar Bhargav","family":"Srinivasan","sequence":"first","affiliation":[{"name":"University of Colorado Boulder, Boulder, CO, USA"}]},{"given":"Cristian","family":"Danescu-Niculescu-Mizil","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}]},{"given":"Lillian","family":"Lee","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}]},{"given":"Chenhao","family":"Tan","sequence":"additional","affiliation":[{"name":"University of Colorado Boulder, Boulder, CO, USA"}]}],"member":"320","published-online":{"date-parts":[[2019,11,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/2655780.2655784"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819963"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134666"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274301"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of WWW. 184--195","author":"Jonathan","unstructured":"Jonathan P. Chang and Cristian Danescu-Niculescu-Mizil. 2019 a. Trajectories of Blocked Community Members: Redemption, Recidivism and Departure . In Proceedings of WWW. 184--195 . https:\/\/doi.org\/10.1145\/3308558.3313638 Jonathan P. Chang and Cristian Danescu-Niculescu-Mizil. 2019 a. Trajectories of Blocked Community Members: Redemption, Recidivism and Departure. In Proceedings of WWW. 184--195. https:\/\/doi.org\/10.1145\/3308558.3313638"},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of EMNLP .","author":"Jonathan","unstructured":"Jonathan P. Chang and Cristian Danescu-Niculescu-Mizil. 2019 b. Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop . In Proceedings of EMNLP . Jonathan P. Chang and Cristian Danescu-Niculescu-Mizil. 2019 b. Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop. In Proceedings of EMNLP ."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531608"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of ICWSM . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM14\/paper\/view\/8066","author":"Cheng Justin","year":"2014","unstructured":"Justin Cheng , Cristian Danescu-Niculescu-Mizil , and Jure Leskovec . 2014 . How community feedback shapes user behavior . In Proceedings of ICWSM . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM14\/paper\/view\/8066 Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2014. How community feedback shapes user behavior. In Proceedings of ICWSM . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM14\/paper\/view\/8066"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of International AAAI Conference on Web and Social Media(ICWSM) . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM15\/paper\/view\/10469\/10489","author":"Cheng Justin","year":"2015","unstructured":"Justin Cheng , Cristian Danescu-Niculescu-Mizil , and Jure Leskovec . 2015 . Antisocial Behavior in Online Discussion Communities . In Proceedings of International AAAI Conference on Web and Social Media(ICWSM) . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM15\/paper\/view\/10469\/10489 Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2015. Antisocial Behavior in Online Discussion Communities. In Proceedings of International AAAI Conference on Web and Social Media(ICWSM) . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM15\/paper\/view\/10469\/10489"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2817946.2817959"},{"key":"e_1_2_1_11_1","unstructured":"Robert B. Cialdini Carl A. Kallgren and Raymond R Reno. [n. d.]. A focus theory of normative conduct: A theoretical refinement and reevaluation of the role of norms in human behavior.  Robert B. Cialdini Carl A. Kallgren and Raymond R Reno. [n. d.]. A focus theory of normative conduct: A theoretical refinement and reevaluation of the role of norms in human behavior."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3055131"},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of International AAAI Conference on Web and Social Media(ICWSM) . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM18\/paper\/view\/17898","author":"Fiesler Casey","year":"2018","unstructured":"Casey Fiesler , Jialun Jiang , Joshua McCann , Kyle Frye , and Jed Brubaker . 2018 . Reddit rules! Characterizing an Ecosystem of Governance . In Proceedings of International AAAI Conference on Web and Social Media(ICWSM) . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM18\/paper\/view\/17898 Casey Fiesler, Jialun Jiang, Joshua McCann, Kyle Frye, and Jed Brubaker. 2018. Reddit rules! Characterizing an Ecosystem of Governance. In Proceedings of International AAAI Conference on Web and Social Media(ICWSM) . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM18\/paper\/view\/17898"},{"key":"e_1_2_1_14_1","article-title":"Interrupted time series regression for the evaluation of public health interventions: a tutorial","volume":"46","author":"Gasparrini Antonio","year":"2016","unstructured":"Antonio Gasparrini , Steven Cummins , and James Lopez Bernal . 2016 . Interrupted time series regression for the evaluation of public health interventions: a tutorial . International Journal of Epidemiology , Vol. 46 , 1 (06 2016), 348--355. https:\/\/doi.org\/10.1093\/ije\/dyw098 Antonio Gasparrini, Steven Cummins, and James Lopez Bernal. 2016. Interrupted time series regression for the evaluation of public health interventions: a tutorial . International Journal of Epidemiology , Vol. 46, 1 (06 2016), 348--355. https:\/\/doi.org\/10.1093\/ije\/dyw098","journal-title":"International Journal of Epidemiology"},{"key":"e_1_2_1_15_1","volume-title":"https:\/\/www.hatebase.org\/. [Online","author":"Hatebase","year":"2018","unstructured":"Hatebase. 2018. Hatebase . https:\/\/www.hatebase.org\/. [Online ; accessed 3- Nov- 2018 ]. Hatebase. 2018. Hatebase . https:\/\/www.hatebase.org\/. [Online; accessed 3-Nov-2018]."},{"key":"e_1_2_1_16_1","volume-title":"Our best hope for civil discourse online is on ... Reddit . Wired","author":"Heffernan Virgina","year":"2018","unstructured":"Virgina Heffernan . 2018. Our best hope for civil discourse online is on ... Reddit . Wired ( 2018 ). https:\/\/www.wired.com\/story\/free-speech-issue-reddit-change-my-view\/ Virgina Heffernan. 2018. Our best hope for civil discourse online is on ... Reddit . Wired (2018). https:\/\/www.wired.com\/story\/free-speech-issue-reddit-change-my-view\/"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274351"},{"key":"e_1_2_1_19_1","volume-title":"Ayman Moghnieh, Rodrigo Meza, Josep Blat, and Vicente L\u00f3 pez.","author":"Kaltenbrunner Andreas","year":"2007","unstructured":"Andreas Kaltenbrunner , Vicencc G\u00f3 mez , Ayman Moghnieh, Rodrigo Meza, Josep Blat, and Vicente L\u00f3 pez. 2007 . Homogeneous temporal activity patterns in a large online communication space. CoRR , Vol. abs\/ 0708 .1579 (2007). http:\/\/arxiv.org\/abs\/0708.1579 Andreas Kaltenbrunner, Vicencc G\u00f3 mez, Ayman Moghnieh, Rodrigo Meza, Josep Blat, and Vicente L\u00f3 pez. 2007. Homogeneous temporal activity patterns in a large online communication space. CoRR , Vol. abs\/0708.1579 (2007). http:\/\/arxiv.org\/abs\/0708.1579"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.31235\/osf.io\/28sgr"},{"key":"e_1_2_1_21_1","volume-title":"Regulating behavior in online communities","author":"Kiesler Sara","unstructured":"Sara Kiesler , Robert E. Kraut , Paul Resnick , and Aniket Kittur . 2011. Regulating behavior in online communities . MIT Press , Cambridge , 125--178. http:\/\/www.jstor.org\/stable\/j.ctt5hhgvw.7 Sara Kiesler, Robert E. Kraut, Paul Resnick, and Aniket Kittur. 2011. Regulating behavior in online communities. MIT Press, Cambridge, 125--178. http:\/\/www.jstor.org\/stable\/j.ctt5hhgvw.7"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.h2750"},{"key":"e_1_2_1_23_1","volume-title":"Clark","author":"Leavitt Alex","year":"2014","unstructured":"Alex Leavitt and Joshua A . Clark . 2014 . Upvoting hurricane Sandy: event-based news production processes on a social news site. In Proceedings of CHI. ACM , 1495--1504. Alex Leavitt and Joshua A. Clark. 2014. Upvoting hurricane Sandy: event-based news production processes on a social news site. In Proceedings of CHI. ACM, 1495--1504."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v12i1.15022"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1813486116"},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the ACM on Human-Computer Interaction","volume":"2","author":"Jonathan","year":"2018","unstructured":"Jonathan T. Morgan and Anna Filippova. 2018. `Welcome' Changes?: Descriptive and Injunctive Norms in a Wikipedia Sub-Community . Proceedings of the ACM on Human-Computer Interaction , Vol. 2 , CSCW ( 2018 ). https:\/\/doi.org\/10.1145\/3274321 Jonathan T. Morgan and Anna Filippova. 2018. `Welcome' Changes?: Descriptive and Injunctive Norms in a Wikipedia Sub-Community. Proceedings of the ACM on Human-Computer Interaction , Vol. 2, CSCW (2018). https:\/\/doi.org\/10.1145\/3274321"},{"key":"e_1_2_1_27_1","volume-title":"Science","volume":"341","author":"Muchnik Lev","year":"2013","unstructured":"Lev Muchnik , Sinan Aral , and Sean J Taylor . 2013 . Social influence bias: A randomized experiment . Science , Vol. 341 , 6146 (2013), 647--651. http:\/\/science.sciencemag.org\/content\/341\/6146\/647 Lev Muchnik, Sinan Aral, and Sean J Taylor. 2013. Social influence bias: A randomized experiment. Science , Vol. 341, 6146 (2013), 647--651. http:\/\/science.sciencemag.org\/content\/341\/6146\/647"},{"key":"e_1_2_1_28_1","unstructured":"Daniel S. Nagin Francis T. Cullen and Cheryl Lero Jonson (Eds.). 2018. Deterrence choice and crime: Contemporary perspectives .Routledge an imprint of the Taylor & Francis Group New York NY.  Daniel S. Nagin Francis T. Cullen and Cheryl Lero Jonson (Eds.). 2018. Deterrence choice and crime: Contemporary perspectives .Routledge an imprint of the Taylor & Francis Group New York NY."},{"key":"e_1_2_1_29_1","volume-title":"Measuring Offensive Speech in Online Political Discourse. CoRR","author":"Nithyanand Rishab","year":"1875","unstructured":"Rishab Nithyanand , Brian Schaffner , and Phillipa Gill . 2017. Measuring Offensive Speech in Online Political Discourse. CoRR , Vol. abs\/ 1706 .0 1875 (2017). http:\/\/arxiv.org\/abs\/1706.01875 Rishab Nithyanand, Brian Schaffner, and Phillipa Gill. 2017. Measuring Offensive Speech in Online Political Discourse. CoRR , Vol. abs\/1706.01875 (2017). http:\/\/arxiv.org\/abs\/1706.01875"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the First Workshop on Social Media Analytics. ACM, 1--9.","author":"Oktay H\u00fcseyin","unstructured":"H\u00fcseyin Oktay , Brian J. Taylor , and David D. Jensen . 2010. Causal discovery in social media using quasi-experimental designs . In Proceedings of the First Workshop on Social Media Analytics. ACM, 1--9. H\u00fcseyin Oktay, Brian J. Taylor, and David D. Jensen. 2010. Causal discovery in social media using quasi-experimental designs. In Proceedings of the First Workshop on Social Media Analytics. ACM, 1--9."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1302"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274406"},{"key":"e_1_2_1_33_1","volume-title":"Francis","author":"Pennebaker James W.","year":"2007","unstructured":"James W. Pennebaker , Roger J. Booth , and Martha E . Francis . 2007 . Linguistic Inquiry and Word Count (LIWC) : A computerized text analysis program. http:\/\/www.liwc.net\/. James W. Pennebaker, Roger J. Booth, and Martha E. Francis. 2007. Linguistic Inquiry and Word Count (LIWC): A computerized text analysis program. http:\/\/www.liwc.net\/."},{"key":"e_1_2_1_34_1","volume-title":"Madensen","author":"Pratt Travis C.","year":"2006","unstructured":"Travis C. Pratt , Francis T. Cullen , Kristie R. Blevins , Leah E. Daigle , and Tamara D . Madensen . 2006 . The Empirical Status of Deterrence Theory: A Meta-Analysis. In Taking stock: The status of criminological theory. Transaction Publishers , Piscataway, NJ, US, 367--395. Travis C. Pratt, Francis T. Cullen, Kristie R. Blevins, Leah E. Daigle, and Tamara D. Madensen. 2006. The Empirical Status of Deterrence Theory: A Meta-Analysis. In Taking stock: The status of criminological theory. Transaction Publishers, Piscataway, NJ, US, 367--395."},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of ICWSM . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM18\/paper\/view\/17907","author":"Samory Mattia","year":"2018","unstructured":"Mattia Samory and Tanushree Mitra . 2018 . Conspiracies Online: User Discussions in a Conspiracy Community Following Dramatic Events . In Proceedings of ICWSM . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM18\/paper\/view\/17907 Mattia Samory and Tanushree Mitra. 2018. Conspiracies Online: User Discussions in a Conspiracy Community Following Dramatic Events. In Proceedings of ICWSM . https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM18\/paper\/view\/17907"},{"key":"e_1_2_1_36_1","volume-title":"Smith","author":"Sap Maarten","year":"2019","unstructured":"Maarten Sap , Dallas Card , Saadia Gabriel , Yejin Choi , and Noah A . Smith . 2019 . The Risk of Racial Bias in Hate Speech Detection. In Proceedings of ACL . Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The Risk of Racial Bias in Hate Speech Detection. In Proceedings of ACL ."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998277"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1177\/0022427893030004006"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2736277.2741661"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883081"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2492517.2492646"},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of NAACL .","author":"Wiegand Michael","year":"2019","unstructured":"Michael Wiegand , Josef Ruppenhofer , and Thomas Kleinbauer . 2019 . Detection of Abusive Language: the Problem of Biased Datasets . In Proceedings of NAACL . Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of Abusive Language: the Problem of Biased Datasets. In Proceedings of NAACL ."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1125"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3359265","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3359265","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3359265","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:23:53Z","timestamp":1750202633000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3359265"}},"subtitle":["Compliance and Other Outcomes in the ChangeMyView Community"],"short-title":[],"issued":{"date-parts":[[2019,11,7]]},"references-count":42,"journal-issue":{"issue":"CSCW","published-print":{"date-parts":[[2019,11,7]]}},"alternative-id":["10.1145\/3359265"],"URL":"https:\/\/doi.org\/10.1145\/3359265","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,7]]},"assertion":[{"value":"2019-11-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}