{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T23:18:49Z","timestamp":1776122329344,"version":"3.50.1"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2021,4,13]],"date-time":"2021-04-13T00:00:00Z","timestamp":1618272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["IIS-1563816, CNS-1704701"],"award-info":[{"award-number":["IIS-1563816, CNS-1704701"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000185","name":"Defense Advanced Research Projects Agency","doi-asserted-by":"publisher","award":["HR00112030001"],"award-info":[{"award-number":["HR00112030001"]}],"id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,4,13]]},"abstract":"<jats:p>With the widespread use of toxic language online, platforms are increasingly using automated systems that leverage advances in natural language processing to automatically flag and remove toxic comments. However, most automated systems, when detecting and moderating toxic language, do not provide feedback to their users, let alone provide an avenue of recourse for these users to make actionable changes. We present our work, RECAST, an interactive, open-sourced web tool for visualizing these models' toxic predictions, while providing alternative suggestions for flagged toxic language. Our work also provides users with a new path of recourse when using these automated moderation tools. RECAST highlights text responsible for classifying toxicity, and allows users to interactively substitute potentially toxic phrases with neutral alternatives. We examined the effect of RECAST via two large-scale user evaluations, and found that RECAST was highly effective at helping users reduce toxicity as detected through the model. Users also gained a stronger understanding of the underlying toxicity criterion used by black-box models, enabling transparency and recourse. In addition, we found that when users focus on optimizing language for these models instead of their own judgement (which is the implied incentive and goal of deploying automated models), these models cease to be effective classifiers of toxicity compared to human annotations. This opens a discussion for how toxicity detection models work and should work, and their effect on the future of online discourse.<\/jats:p>","DOI":"10.1145\/3449280","type":"journal-article","created":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T17:49:24Z","timestamp":1619113764000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["RECAST"],"prefix":"10.1145","volume":"5","author":[{"given":"Austin P.","family":"Wright","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Omar","family":"Shaikh","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Haekyu","family":"Park","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Will","family":"Epperson","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Muhammed","family":"Ahmed","sequence":"additional","affiliation":[{"name":"Mailchimp, Atlanta, GA, USA"}]},{"given":"Stephane","family":"Pinel","sequence":"additional","affiliation":[{"name":"Mailchimp, Atlanta, GA, USA"}]},{"given":"Duen Horng (Polo)","family":"Chau","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Diyi","family":"Yang","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}]}],"member":"320","published-online":{"date-parts":[[2021,4,22]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Tolga Bolukbasi Kai-Wei Chang James Zou Venkatesh Saligrama and Adam Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. arxiv: cs.CL\/1607.06520"},{"key":"e_1_2_1_2_1","unstructured":"Catherine Buni and Soraya Chemaly. 2016. The secret rules of the internet. https:\/\/www.theverge.com\/2016\/4\/13\/11387934\/internet-moderator-history-youtube-facebook-reddit-censorship-free-speech"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1002\/poi3.85"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3311957.3359478"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819963"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359276"},{"key":"e_1_2_1_7_1","volume-title":"Gianluca Stringhini, and Athena Vakali.","author":"Chatzakou Despoina","year":"2017","unstructured":"Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, and Athena Vakali. 2017. Mean Birds: Detecting Aggression and Bullying on Twitter. arxiv: cs.CY\/1702.06877"},{"key":"e_1_2_1_8_1","first-page":"1435","article-title":"Intermediaries and hate speech: Fostering digital citizenship for our information age","volume":"91","author":"Citron Danielle Keats","year":"2011","unstructured":"Danielle Keats Citron and Helen Norton. 2011. Intermediaries and hate speech: Fostering digital citizenship for our information age. BUL Rev., Vol. 91 (2011), 1435.","journal-title":"BUL Rev."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_2_1_10_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)."},{"key":"e_1_2_1_11_1","unstructured":"Allyson Ettinger. 2019. What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. arxiv: cs.CL\/1907.13528"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2005.10.010"},{"key":"e_1_2_1_13_1","unstructured":"Menczer Filippo R. Fulper E. Ferrara Y. Ahn A. Flammini B. Lewis and K. Rowe. 2015. Misogynistic Language on Twitter and Sexual Violence."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718918.1718941"},{"key":"e_1_2_1_15_1","volume-title":"Custodians of the Internet: platforms, content moderation, and the hidden decisions that shape social media","author":"Gillespie Tarleton","unstructured":"Tarleton Gillespie. 2018. Custodians of the Internet: platforms, content moderation, and the hidden decisions that shape social media. Yale University Press."},{"key":"e_1_2_1_16_1","unstructured":"Fred Hohman Minsuk Kahng Robert Pienta and Duen Horng Chau. 2018. Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers. arxiv: cs.HC\/1801.06889"},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Benjamin Hoover Hendrik Strobelt and Sebastian Gehrmann. 2019. exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models. arxiv: cs.CL\/1910.05276","DOI":"10.18653\/v1\/2020.acl-demos.22"},{"key":"e_1_2_1_18_1","unstructured":"Hossein Hosseini Sreeram Kannan Baosen Zhang and Radha Poovendran. 2017. Deceiving Google's Perspective API Built for Detecting Toxic Comments. arxiv: cs.LG\/1702.08138"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3291018"},{"key":"e_1_2_1_20_1","volume-title":"Conference on Empirical Methods in Natural Language Processing. Citeseer.","author":"JeffreyPennington RichardSocher","year":"2014","unstructured":"RichardSocher JeffreyPennington and ChristopherD Manning. 2014. Glove: Global vectors for word representation. In Conference on Empirical Methods in Natural Language Processing. Citeseer."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359294"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338243"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359252"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3185593"},{"key":"e_1_2_1_25_1","unstructured":"Kaggle. 2017. Jigsaw Toxicity Dataset. https:\/\/tinyurl.com\/y3fbco5b"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376873"},{"key":"e_1_2_1_28_1","doi-asserted-by":"crossref","unstructured":"Brandon Laughlin Christopher Collins Karthik Sankaranarayanan and Khalil El-Khatib. 2019. A Visual Analytics Framework for Adversarial Text Generation. arxiv: cs.HC\/1909.11202","DOI":"10.1109\/VizSec48167.2019.9161563"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2019.23138"},{"key":"e_1_2_1_30_1","unstructured":"Google LLC. 2017. Perspective API. https:\/\/www.perspectiveapi.com\/."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025830"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174160"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v13i01.3237"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173583"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the 26th International Conference on Neural Information Processing Systems -","volume":"2","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and Their Compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 (Lake Tahoe, Nevada) (NIPS'13). Curran Associates Inc., Red Hook, NY, USA, 3111--3119."},{"key":"e_1_2_1_37_1","volume-title":"Jian Zhao, Hyungjong Noh, Yeonsoo Lee, and Jaegul Choo.","author":"Park Cheonbok","year":"2019","unstructured":"Cheonbok Park, Inyoup Na, Yongjang Jo, Sungbok Shin, Jaehyo Yoo, Bum Chul Kwon, Jian Zhao, Hyungjong Noh, Yeonsoo Lee, and Jaegul Choo. 2019. SANVis: Visual Analytics for Understanding Self-Attention Networks. arxiv: cs.CL\/1909.09595"},{"key":"e_1_2_1_38_1","volume-title":"PyTorch: An Imperative Style","author":"Paszke Adam","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. dtextquotesingle Alch\u00e9-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024--8035."},{"key":"e_1_2_1_39_1","volume-title":"Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang.","author":"Pryzant Reid","year":"2019","unstructured":"Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2019. Automatically Neutralizing Subjective Bias in Text. arxiv: cs.CL\/1911.09709"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3415178"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300836"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_2_1_44_1","doi-asserted-by":"crossref","unstructured":"Dylan Slack Sophie Hilgard Emily Jia Sameer Singh and Himabindu Lakkaraju. 2019. How can we fool LIME and SHAP? Adversarial Attacks on Post hoc Explanation Methods. arxiv: cs.LG\/1911.02508","DOI":"10.1145\/3375627.3375830"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376783"},{"key":"e_1_2_1_46_1","unstructured":"Kit Smith. 2019. 53 Incredible Facebook Statistics and Facts. https:\/\/www.brandwatch.com\/blog\/facebook-statistics\/"},{"key":"e_1_2_1_47_1","volume-title":"GSDRC Applied Research Services","volume":"23","author":"Strachan Anna Louise","year":"2014","unstructured":"Anna Louise Strachan. 2014. Interventions to counter hate speech. GSDRC Applied Research Services, Vol. 23 (2014)."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1017\/S1062798700002660"},{"key":"e_1_2_1_49_1","unstructured":"Mukund Sundararajan Ankur Taly and Qiqi Yan. 2017. Axiomatic Attribution for Deep Networks. arxiv: cs.LG\/1703.01365"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1177\/1748048518757142"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359220"},{"key":"e_1_2_1_52_1","volume-title":"Nudge: Improving decisions about health, wealth, and happiness","author":"Thaler Richard H","year":"2009","unstructured":"Richard H Thaler and Cass R Sunstein. 2009. Nudge: Improving decisions about health, wealth, and happiness. Penguin."},{"key":"e_1_2_1_53_1","volume-title":"The visual display of quantitative information","author":"Tufte Edward R.","unstructured":"Edward R. Tufte. 2018. The visual display of quantitative information. Graphics Press."},{"key":"e_1_2_1_54_1","unstructured":"Hernan Valdivieso Denis Parra Andres Carvallo Gabriel Rada Katrien Verbert and Tobias Schreck. 2019. Analyzing the Design Space for Visualizing Neural Attention in Text Classification. https:\/\/tinyurl.com\/yzt866vb"},{"key":"e_1_2_1_55_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008."},{"key":"e_1_2_1_56_1","doi-asserted-by":"crossref","unstructured":"Suresh Venkatasubramanian and Mark Alfano. 2020. The philosophical basis of algorithmic recourse. (2020).","DOI":"10.1145\/3351095.3372876"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/tvcg.2019.2934619"},{"key":"e_1_2_1_58_1","doi-asserted-by":"crossref","unstructured":"Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not Explanation. arxiv: cs.CL\/1908.04626","DOI":"10.18653\/v1\/D19-1002"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_2_1_60_1","unstructured":"Wei Emma Zhang Quan Z. Sheng Ahoud Alhazmi and Chenliang Li. 2019. Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey. arxiv: cs.CL\/1901.06796"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449280","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3449280","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3449280","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:22Z","timestamp":1750195702000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449280"}},"subtitle":["Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization"],"short-title":[],"issued":{"date-parts":[[2021,4,13]]},"references-count":60,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2021,4,13]]}},"alternative-id":["10.1145\/3449280"],"URL":"https:\/\/doi.org\/10.1145\/3449280","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,13]]},"assertion":[{"value":"2021-04-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}