{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T06:02:23Z","timestamp":1775023343903,"version":"3.50.1"},"reference-count":82,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2021,10,13]],"date-time":"2021-10-13T00:00:00Z","timestamp":1634083200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,10,13]]},"abstract":"<jats:p>The proliferation of harmful content on online social media platforms has necessitated empirical understandings of experiences of harm online and the development of practices for harm mitigation. Both understandings of harm and approaches to mitigating that harm, often through content moderation, have implicitly embedded frameworks of prioritization-what forms of harm should be researched, how policy on harmful content should be implemented, and how harmful content should be moderated. To aid efforts of better understanding the variety of online harms, how they relate to one another, and how to prioritize harms relevant to research, policy, and practice, we present a theoretical framework of severity for harmful online content. By employing a grounded theory approach, we developed a framework of severity based on interviews and card-sorting activities conducted with 52 participants over the course of ten months. Through our analysis, we identified four Types of Harm (physical, emotional, relational, and financial) and eight Dimensions along which the severity of harm can be understood (perspectives, intent, agency, experience, scale, urgency, vulnerability, sphere). We describe how our framework can be applied to both research and policy settings towards deeper understandings of specific forms of harm (e.g., harassment) and prioritization frameworks when implementing policies encompassing many forms of harm.<\/jats:p>","DOI":"10.1145\/3479512","type":"journal-article","created":{"date-parts":[[2021,10,19]],"date-time":"2021-10-19T02:32:07Z","timestamp":1634610727000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":82,"title":["A Framework of Severity for Harmful Content Online"],"prefix":"10.1145","volume":"5","author":[{"given":"Morgan Klaus","family":"Scheuerman","sequence":"first","affiliation":[{"name":"University of Colorado Boulder, Boulder, CO, USA"}]},{"given":"Jialun Aaron","family":"Jiang","sequence":"additional","affiliation":[{"name":"University of Colorado Boulder, Boulder, CO, USA"}]},{"given":"Casey","family":"Fiesler","sequence":"additional","affiliation":[{"name":"University of Colorado, Boulder, CO, USA"}]},{"given":"Jed R.","family":"Brubaker","sequence":"additional","affiliation":[{"name":"University of Colorado Boulder, Boulder, CO, USA"}]}],"member":"320","published-online":{"date-parts":[[2021,10,18]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Optimal Workshop | User Experience (UX) Research Platform. https:\/\/www.optimalworkshop.com\/","unstructured":"[n.d.]. Optimal Workshop | User Experience (UX) Research Platform. https:\/\/www.optimalworkshop.com\/"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--3--319--78583--7_8"},{"key":"e_1_2_1_3_1","unstructured":"Harriet Alexander. 2020. Woman jailed for shooting her rapist dead after he threatened to kill her. https:\/\/www.independent.co.uk\/news\/world\/americas\/alabama-rape-shooting-murder-brittany-smith-todd-jail-sentence-b936467.html"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2905055.2905362"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998243"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998243"},{"key":"e_1_2_1_7_1","unstructured":"Jeff Asher and Ben Horwitz. 2020. How Do the Police Actually Spend Their Time? https:\/\/www.nytimes.com\/2020\/06\/19\/upshot\/unrest-police-time-violent-crime.html"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858548"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3327525"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/1555412015592473"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134659"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","unstructured":"Lindsay Blackwell Nicole Ellison Natasha Elliott-Deflo and Raz Schwartz. 2019. Harassment in social virtual reality: Challenges for platform governance. 25 pages. https:\/\/doi.org\/10.1145\/3359202","DOI":"10.1145\/3359202"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3170427.3170610"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1177\/2056305116672484"},{"key":"e_1_2_1_15_1","volume-title":"Content or context moderation? Artisanal, community-reliant, and industrial approaches","author":"Caplan Robyn","year":"2018","unstructured":"Robyn Caplan. 2018. Content or context moderation? Artisanal, community-reliant, and industrial approaches. Data & Society (2018). https:\/\/datasociety.net\/library\/content-or-context-moderation\/"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819963"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359276"},{"key":"e_1_2_1_18_1","unstructured":"Kathy Charmaz. 2006. Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. 388 pages. http:\/\/www.sxf.uevora.pt\/wp-content\/uploads\/2013\/03\/Charmaz_2006.pdf"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.26522\/brocked.v12i2.38"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3053890"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5210\/fm.v20i2.5450"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.3389\/fhumd.2021.626409"},{"key":"e_1_2_1_23_1","volume-title":"Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory","author":"Corbin Juliet","unstructured":"Juliet Corbin and Anselm Strauss. 2014. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE Publications Inc. https:\/\/us.sagepub.com\/en-us\/nam\/basics-of-qualitative-research\/book235578http:\/\/us.sagepub.com\/en-us\/nam\/basics-of-qualitative-research\/book235578"},{"key":"e_1_2_1_24_1","volume-title":"But Who Protects the Moderators? The Case of Crowdsourced Image Moderation. (apr","author":"Dang Brandon","year":"2018","unstructured":"Brandon Dang, Martin J. Riedl, and Matthew Lease. 2018. But Who Protects the Moderators? The Case of Crowdsourced Image Moderation. (apr 2018). arXiv:1804.10999 http:\/\/arxiv.org\/abs\/1804.10999"},{"key":"e_1_2_1_25_1","unstructured":"City of Eugene Police Department. 2009. Police Call Priority Definitions. Technical Report. www.eugene-or.gov\/police"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--981--13--3582--2_12"},{"key":"e_1_2_1_27_1","unstructured":"Facebook. [n.d.]. Community Standards. ([n. d.]). https:\/\/www.facebook.com\/communitystandards\/violence_criminal_behavior"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359328"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3184558.3188730"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3392845"},{"key":"e_1_2_1_31_1","unstructured":"Maggie Fick and Dave Paresh. 2019. INSIGHT-Facebook's flood of languages leave it struggling to monitor content. https:\/\/www.reuters.com\/article\/facebook-languages\/insight-facebooks-flood-of-languages-leave-it-struggling-to-monitor-content-idUSL1N20U01Ehttps:\/\/www.reuters.com\/article\/us-facebook-languages-insight-idUSKCN1RZ0DW"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v12i1.15033"},{"key":"e_1_2_1_33_1","unstructured":"Ed Gilgor. [n.d.]. Is Your 911 Call a Priority? Technical Report. https:\/\/grantpark.org\/info\/16029"},{"key":"e_1_2_1_34_1","volume-title":"Custodians of the Internet","author":"Gillespie Tarleton","unstructured":"Tarleton Gillespie. 2018. Custodians of the Internet. Yale University Press. https:\/\/yalebooks.yale.edu\/book\/9780300173130\/custodians-internet"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444810363910"},{"key":"e_1_2_1_36_1","first-page":"172","article-title":"Hate Crimes in Cyberspace","volume":"8","author":"Halder Debarati","year":"2014","unstructured":"Debarati Halder. 2014. Hate Crimes in Cyberspace. International Journal of Cyber Criminology 8, 2 (2014), 172--175. http:\/\/www.jstor.org\/stable\/j.ctt7zsws7http:\/\/search.ebscohost.com\/login.aspx?direct=true&db=i3h&AN=101788663&site=ehost-live","journal-title":"International Journal of Cyber Criminology"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CBMI.2019.8877435"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1093\/joc\/jqz006"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338243"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359157"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3406865.3418312"},{"key":"e_1_2_1_42_1","first-page":"1599","article-title":"The new governors: The people, rules, and processes governing online speech","volume":"131","author":"Klonick Kate","year":"2018","unstructured":"Kate Klonick. 2018. The new governors: The people, rules, and processes governing online speech. Harvard Law Review 131, 6 (2018), 1599--1670. https:\/\/harvardlawreview.org\/2018\/04\/the-new-governors-the-people-rules-and-processes-governing-online-speech\/","journal-title":"Harvard Law Review"},{"key":"e_1_2_1_43_1","unstructured":"Jason Koebler and Joseph Cox. 2018. The Impossible Job: Inside Facebook's Struggle to Moderate Two Billion People. Aug 23 pages. https:\/\/www.vice.com\/en\/article\/xwk9zd\/how-facebook-content-moderation-workshttps:\/\/www.vice.com\/en_us\/article\/xwk9zd\/how-facebook-content-moderation-works"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376396"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240760"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702529"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.5210\/SPIR.V5I0.8490"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1177\/0267323114521045"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1002\/9781119429128.iegmc014"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025875"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444818773059"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272973.3274057"},{"key":"e_1_2_1_53_1","first-page":"21","article-title":"Men Are Scum\": Self-Regulation, Hate Speech, and Gender-Based Censorship on Facebook","volume":"13","author":"Nurik Chlo\u00e9","year":"2019","unstructured":"Chlo\u00e9 Nurik. 2019. \"Men Are Scum\": Self-Regulation, Hate Speech, and Gender-Based Censorship on Facebook. International Journal of Communication 13 (2019), 21. http:\/\/ijoc.org.","journal-title":"International Journal of Communication"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2957276.2957297"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","unstructured":"Lynne Peeples. 2020. What the data say about police brutality and racial bias - and which reforms might work. 22--24 pages. https:\/\/doi.org\/10.1038\/d41586-020-01846-z","DOI":"10.1038\/d41586-020-01846-z"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272973.3274055"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.2307\/j.ctt1fq9wdp"},{"key":"e_1_2_1_58_1","unstructured":"Dallas Police. 2016. Communications Operations Center (Handling Calls for Service). Technical Report."},{"key":"e_1_2_1_59_1","volume-title":"Crash Override: How Gamergate (Nearly) Destroyed My Life, and How We Can Win the Fight Against Online Hate. 242 pages. https:\/\/books.google.com\/books\/about\/Crash_Override.html?id=gpvUDQAAQBAJ","author":"Quinn Zoe","year":"2017","unstructured":"Zoe Quinn. 2017. Crash Override: How Gamergate (Nearly) Destroyed My Life, and How We Can Win the Fight Against Online Hate. 242 pages. https:\/\/books.google.com\/books\/about\/Crash_Override.html?id=gpvUDQAAQBAJ"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v13i01.3356"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","unstructured":"Kathryn E Ringland Christine T Wolf Lynn Dombrowski and Gillian R Hayes. 2015. Making \"Safe\". In CSCW. 1788--1800. https:\/\/doi.org\/10.1145\/2675133.2675216","DOI":"10.1145\/2675133.2675216"},{"key":"e_1_2_1_62_1","volume-title":"Behind the Screen","author":"Roberts Sarah T.","unstructured":"Sarah T. Roberts. 2019. Behind the Screen. Yale University Press. https:\/\/yalebooks.yale.edu\/book\/9780300235883\/behind-screen"},{"key":"e_1_2_1_63_1","unstructured":"Adi Robertson. 2014. 'Massacre' threat forces Anita Sarkeesian to cancel university talk - The Verge. https:\/\/www.theverge.com\/2014\/10\/14\/6978809\/utah-state-university-receives-shooting-threat-for-anita-sarkeesian-visit"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444819875990"},{"key":"e_1_2_1_65_1","unstructured":"Valeriya Safronova and Rebecca Halleck. 2019. These Rape Victims Had to Sue to Get the Police to Investigate. https:\/\/www.nytimes.com\/2019\/05\/23\/us\/rape-victims-kits-police-departments.html"},{"key":"e_1_2_1_66_1","volume-title":"The Coding Manual for Qualitative Researchers Instruction","author":"Saldana Johnny","year":"2013","unstructured":"Johnny Saldana. 2013. The Coding Manual for Qualitative Researchers Instruction. Sage (2013), 299. www.sagepublications.com"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274424"},{"key":"e_1_2_1_68_1","unstructured":"S Schrock and D Boyd. 2009. Online threats to youth: Solicitation harassment and problematic content. In Enhancing Child Safety and Online Technologies. http:\/\/cyber.law.harvard.edu\/research\/isttf"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444818821316"},{"key":"e_1_2_1_70_1","volume-title":"Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences","author":"Seidman Irving","unstructured":"Irving Seidman. 1998. Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences. Teachers College Press. https:\/\/books.google.com\/books\/about\/Interviewing_as_Qualitative_Research.html?id=DhpjQgAACAAJ&source=kp_book_descriptionhttps:\/\/books.google.co.uk\/books?hl=en&lr=&id=pk1Rmq-Y15QC&oi=fnd&pg=PR9&ots=4pl7SRNPJR&sig=rRd9I_WNECoUTA7KzGwTti_R3g0&redir_esc=y#v"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--3--319--32699--3_11"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998279"},{"key":"e_1_2_1_73_1","unstructured":"Donna Spencer and Jesse James Garrett. 2009. Card Sorting: Designing Usable Categories - Donna Spencer - Google Books. Rosenfeld Media. 162 pages. https:\/\/books.google.com\/books\/about\/Card_Sorting.html?id=_h4D9gqi5tsC&source=kp_book_descriptionhttps:\/\/books.google.com.cy\/books?hl=en&lr=&id=-nk3DwAAQBAJ&oi=fnd&pg=PP1&dq=card+sorting+web+design&ots=2__cH6MatE&sig=i2cJh6KKSmAjc_zKoaS8HxnFaiU&redir_esc"},{"key":"e_1_2_1_74_1","unstructured":"Alice Speri Alleen Brown and Mara Hvistendahl. [n.d.]. The George Floyd Killing Exposes Failures of Police Reform. https:\/\/theintercept.com\/2020\/05\/29\/george-floyd-minneapolis-police-reform\/"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3392828"},{"key":"e_1_2_1_76_1","unstructured":"Twitter. [n.d.]. Twitter Terms of Service. https:\/\/twitter.com\/en\/tos"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jadohealth.2019.03.006"},{"key":"e_1_2_1_78_1","unstructured":"James Vincent. 2019. AI won't relieve the misery of Facebook's human moderators. https:\/\/www.theverge.com\/2019\/2\/27\/18242724\/facebook-moderation-ai-artificial-intelligence-platforms"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998337"},{"key":"e_1_2_1_80_1","unstructured":"Nick Wingfield. 2014. Feminist Critics of Video Games Facing Threats in 'GamerGate' Campaign. https:\/\/www.nytimes.com\/2014\/10\/16\/technology\/gamergate-women-video-game-threats-anita-sarkeesian.htmlhttps:\/\/www.nytimes.com\/2014\/10\/16\/technology\/gamergate-women-video-game-threats-anita-sarkeesian.html?_r=0"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300390"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/2554850.2554894"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479512","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3479512","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:58:16Z","timestamp":1752469096000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479512"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,13]]},"references-count":82,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2021,10,13]]}},"alternative-id":["10.1145\/3479512"],"URL":"https:\/\/doi.org\/10.1145\/3479512","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,13]]},"assertion":[{"value":"2021-10-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}