{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T22:45:32Z","timestamp":1776293132763,"version":"3.50.1"},"reference-count":81,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2023,4,14]],"date-time":"2023-04-14T00:00:00Z","timestamp":1681430400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100001143","name":"William T. Grant Foundation","doi-asserted-by":"publisher","award":["#187941"],"award-info":[{"award-number":["#187941"]}],"id":[{"id":"10.13039\/100001143","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIP-1827700 and IIS-1844881"],"award-info":[{"award-number":["IIP-1827700 and IIS-1844881"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2023,4,14]]},"abstract":"<jats:p>We collected Instagram data from 150 adolescents (ages 13-21) that included 15,547 private message conversations of which 326 conversations were flagged as sexually risky by participants. Based on this data, we leveraged a human-centered machine learning approach to create sexual risk detection classifiers for youth social media conversations. Our Convolutional Neural Network (CNN) and Random Forest models outperformed in identifying sexual risks at the conversation-level (AUC=0.88), and CNN outperformed at the message-level (AUC=0.85). We also trained classifiers to detect the severity risk level (i.e., safe, low, medium-high) of a given message with CNN outperforming other models (AUC=0.88). A feature analysis yielded deeper insights into patterns found within sexually safe versus unsafe conversations. We found that contextual features (e.g., age, gender, and relationship type) and Linguistic Inquiry and Word Count (LIWC) contributed the most for accurately detecting sexual conversations that made youth feel uncomfortable or unsafe. Our analysis provides insights into the important factors and contextual features that enhance automated detection of sexual risks within youths' private conversations. As such, we make valuable contributions to the computational risk detection and adolescent online safety literature through our human-centered approach of collecting and ground truth coding private social media conversations of youth for the purpose of risk classification.<\/jats:p>","DOI":"10.1145\/3579522","type":"journal-article","created":{"date-parts":[[2023,4,16]],"date-time":"2023-04-16T17:23:07Z","timestamp":1681665787000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":48,"title":["Sliding into My DMs: Detecting Uncomfortable or Unsafe Sexual Risk Experiences within Instagram Direct Messages Grounded in the Perspective of Youth"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5829-8004","authenticated-orcid":false,"given":"Afsaneh","family":"Razi","sequence":"first","affiliation":[{"name":"Drexel University, Philadelphia, PA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1569-9662","authenticated-orcid":false,"given":"Ashwaq","family":"Alsoubai","sequence":"additional","affiliation":[{"name":"Vanderbilt University, Nashville, TN, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2481-2292","authenticated-orcid":false,"given":"Seunghyun","family":"Kim","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7520-5279","authenticated-orcid":false,"given":"Shiza","family":"Ali","sequence":"additional","affiliation":[{"name":"Boston University, Boston, MA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6162-578X","authenticated-orcid":false,"given":"Gianluca","family":"Stringhini","sequence":"additional","affiliation":[{"name":"Boston University, Boston, MA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8939-264X","authenticated-orcid":false,"given":"Munmun","family":"De Choudhury","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6223-1029","authenticated-orcid":false,"given":"Pamela J.","family":"Wisniewski","sequence":"additional","affiliation":[{"name":"Vanderbilt University, Nashville, TN, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,4,16]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2020. National Center for Missing Exploited Children. https:\/\/www.missingkids.org\/footer\/media\/keyfacts"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3383073"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501969"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3500868.3559710"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555136"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858096"},{"key":"e_1_2_1_7_1","unstructured":"Monica Anderson and Jingjing Jiang. 2018. Teens Social Media & Technology 2018 | Pew Research Center. http:\/\/www.pewinternet.org\/2018\/05\/31\/teens-social-media-technology-2018\/"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/EISIC.2015.41"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951717718854"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.appdev.2010.07.005"},{"key":"e_1_2_1_12_1","volume-title":"A survey on data augmentation for text classification. arXiv preprint arXiv:2107.03158","author":"Bayer Markus","year":"2021","unstructured":"Markus Bayer, Marc-Andr\u00e9 Kaufhold, and Christian Reuter. 2021. A survey on data augmentation for text classification. arXiv preprint arXiv:2107.03158 (2021)."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/WIFS47025.2019.9035090"},{"key":"e_1_2_1_14_1","first-page":"267","article-title":"Constitutional Dimensions of Predictive Algorithms in Criminal Justice","volume":"55","author":"Brenner Michael","year":"2020","unstructured":"Michael Brenner, Jeannie Suk Gersen, Michael Haley, Matthew Lin, Amil Merchant, Richard Jagdishwar Millett, Suproteem K Sarkar, and Drew Wegner. 2020. Constitutional Dimensions of Predictive Algorithms in Criminal Justice. Harv. CR-CLL Rev. 55 (2020), 267.","journal-title":"Harv. CR-CLL Rev."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chiabu.2020.104647"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1021316409277"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3462204.3481731"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"key":"e_1_2_1_19_1","volume-title":"Perception Over Time: Temporal Dynamics for Robust Image Understanding. arXiv preprint arXiv:2203.06254","author":"Daniali Maryam","year":"2022","unstructured":"Maryam Daniali and Edward Kim. 2022. Perception Over Time: Temporal Dynamics for Robust Image Understanding. arXiv preprint arXiv:2203.06254 (2022)."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v8i1.14526"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1080\/00224499.2017.1386763"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3500868.3559469"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1089\/cyber.2012.0545"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1089\/cyber.2012.0545"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2013.11.001"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/IWBF49977.2020.9107945"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2020.3031769"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2014.11.013"},{"key":"e_1_2_1_30_1","unstructured":"Eszter Hargittai Jason Schultz John Palfrey et al. 2011. Why parents help their children lie to Facebook about age: Unintended consequences of the ?Children's Online Privacy Protection Act'. First Monday (2011)."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3449116"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3462204.3481739"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v14i1.7296"},{"key":"e_1_2_1_34_1","doi-asserted-by":"crossref","unstructured":"Laurie Collier Hillstrom. 2018. The# metoo movement. ABC-CLIO.","DOI":"10.5040\/9798400605062"},{"key":"e_1_2_1_35_1","volume-title":"Alexandra Olteanu, and Kush R Varshney.","author":"Hind Michael","year":"2018","unstructured":"Michael Hind, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Alexandra Olteanu, and Kush R Varshney. 2018. Increasing trust in ai services through supplier's declarations of conformity. arXiv preprint arXiv:1808.07261 18 (2018), 2813--2869."},{"key":"e_1_2_1_36_1","volume-title":"Three approaches to qualitative content analysis. Qualitative health research 15, 9","author":"Hsieh Hsiu-Fang","year":"2005","unstructured":"Hsiu-Fang Hsieh and Sarah E Shannon. 2005. Three approaches to qualitative content analysis. Qualitative health research 15, 9 (2005), 1277--1288."},{"key":"e_1_2_1_37_1","volume-title":"CLEF (Online working notes\/labs\/workshop)","author":"Inches Giacomo","unstructured":"Giacomo Inches and Fabio Crestani. 2012. Overview of the International Sexual Predator Identification Competition at PAN-2012.. In CLEF (Online working notes\/labs\/workshop), Vol. 30."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675287"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v15i1.18061"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3476066"},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. In EMNLP.","DOI":"10.3115\/v1\/D14-1181"},{"key":"e_1_2_1_43_1","volume-title":"Contextual augmentation: Data augmentation by words with paradigmatic relations. arXiv preprint arXiv:1805.06201","author":"Kobayashi Sosuke","year":"2018","unstructured":"Sosuke Kobayashi. 2018. Contextual augmentation: Data augmentation by words with paradigmatic relations. arXiv preprint arXiv:1805.06201 (2018)."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CRISIS.2012.6378951"},{"key":"e_1_2_1_45_1","unstructured":"Amanda Lenhart Monica Anderson and Aaron Smith. 2015. Teens Technology and Romantic Relationships | Pew Research Center. http:\/\/www.pewinternet.org\/2015\/10\/01\/teens-technology-and-romantic-relationships\/"},{"key":"e_1_2_1_46_1","first-page":"53","article-title":"3lc: Lightweight and effective traffic compression for distributed machine learning","volume":"1","author":"Lim Hyeontaek","year":"2019","unstructured":"Hyeontaek Lim, David G Andersen, and Michael Kaminsky. 2019. 3lc: Lightweight and effective traffic compression for distributed machine learning. Proceedings of Machine Learning and Systems 1 (2019), 53--64.","journal-title":"Proceedings of Machine Learning and Systems"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3358146"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pragma.2019.09.010"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-5010"},{"key":"e_1_2_1_50_1","volume-title":"A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635","author":"Mehrabi Ninareh","year":"2019","unstructured":"Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635 (2019)."},{"key":"e_1_2_1_51_1","volume-title":"A typology of child cybersexploitation and online grooming practices. Cyberspace Research Unit","author":"O'Connell Rachel","unstructured":"Rachel O'Connell. 2003. A typology of child cybersexploitation and online grooming practices. Cyberspace Research Unit, University of Central Lancashire (2003)."},{"key":"e_1_2_1_52_1","first-page":"2001","article-title":"Linguistic inquiry and word count: LIWC 2001. Mahway","volume":"71","author":"Pennebaker James W","year":"2001","unstructured":"James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates 71, 2001 (2001), 2001.","journal-title":"Lawrence Erlbaum Associates"},{"key":"e_1_2_1_53_1","volume-title":"Manning","author":"Pennington Jeffrey","year":"2014","unstructured":"Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe Global Vectors for Word Representation, 1532--1543 pages. http:\/\/www.aclweb.org\/anthology\/D14--1162"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1080\/10807039609383615"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11747-019-00710-5"},{"key":"e_1_2_1_56_1","volume-title":"Emerging Perspectives in Human-Centered Machine Learning. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1--8.","author":"Ramos Gonzalo","year":"2019","unstructured":"Gonzalo Ramos, Jina Suh, Soroush Ghorashi, Christopher Meek, Richard Banks, Saleema Amershi, Rebecca Fiebrink, Alison Smith-Renner, and Gagan Bansal. 2019. Emerging Perspectives in Human-Centered Machine Learning. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1--8."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3587558"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3503569"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376400"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3479609"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-022-09663-w"},{"key":"e_1_2_1_62_1","volume-title":"Spinning words as disguise: Shady services for ethical research? First Monday","author":"Reagle Joseph","year":"2022","unstructured":"Joseph Reagle and Manas Gaur. 2022. Spinning words as disguise: Shady services for ethical research? First Monday (2022)."},{"key":"e_1_2_1_63_1","volume-title":"How do adolescents experience sexting in dating relationships? Motivations to sext and responses to sexting requests from dating partners. Children & Youth Services Rev 109","author":"Reed Lauren","year":"2020","unstructured":"Lauren Reed, Margaret Boyer, Haley Meskunas, Richard Tolman, and L Ward. 2020. How do adolescents experience sexting in dating relationships? Motivations to sext and responses to sexting requests from dating partners. Children & Youth Services Rev 109 (2020)."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/SMC.2019.8914528"},{"key":"e_1_2_1_65_1","volume-title":"Daniel Weiskopf, Stephen North, and Daniel Keim.","author":"Sacha Dominik","year":"2016","unstructured":"Dominik Sacha, Michael Sedlmair, Leishi Zhang, John Aldo Lee, Daniel Weiskopf, Stephen North, and Daniel Keim. 2016. Human-centered machine learning through interactive visualization. ESANN, Bruges, Belgium, 641--646. https:\/\/www.elen.ucl.ac.be\/Proceedings\/esann\/esannpdf\/es2016--166.pdf"},{"key":"e_1_2_1_66_1","volume-title":"Introduction to information retrieval","author":"Sch\u00fctze Hinrich","unstructured":"Hinrich Sch\u00fctze, Christopher D Manning, and Prabhakar Raghavan. 2008. Introduction to information retrieval. Cambridge University Press."},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.forsciint.2019.02.028"},{"key":"e_1_2_1_68_1","first-page":"8","article-title":"Chi-square test is statistically significant: Now what","volume":"20","author":"Sharpe Donald","year":"2015","unstructured":"Donald Sharpe. 2015. Chi-square test is statistically significant: Now what? Practical Assessment, Research, and Evaluation 20, 1 (2015), 8.","journal-title":"Practical Assessment, Research, and Evaluation"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1080\/03069885.2016.1255717"},{"key":"e_1_2_1_70_1","unstructured":"Svetlana Smirnova Sonia Livingstone and Mariya Stoilova. 2021. Understanding of user needs and problems: a rapid evidence review of age assurance and parental controls. (2021)."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3400806.3400825"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--3-030--22636--7_6"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1177\/0261927X09351676"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-020-09488-3"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1177\/1059840514541964"},{"key":"e_1_2_1_76_1","volume-title":"A human-centered agenda for intelligible machine learning. Machines We Trust: Getting Along with Artificial Intelligence","author":"Vaughan Jennifer Wortman","year":"2020","unstructured":"Jennifer Wortman Vaughan and Hanna Wallach. 2020. A human-centered agenda for intelligible machine learning. Machines We Trust: Getting Along with Artificial Intelligence (2020)."},{"key":"e_1_2_1_77_1","volume-title":"In Proc 17th ACM CSCW. 461--474","author":"Vitak Jessica","year":"2014","unstructured":"Jessica Vitak and Jinyoung Kim. 2014. \"You can't block people offline\" examining how facebook's affordances shape the disclosure process. In In Proc 17th ACM CSCW. 461--474."},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.avb.2012.09.003"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858317"},{"key":"e_1_2_1_80_1","volume-title":"How risky are social networking sites? A comparison of places online where youth sexual solicitation and harassment occurs. Pediatrics 121, 2","author":"Ybarra Michele L","year":"2008","unstructured":"Michele L Ybarra and Kimberly J Mitchell. 2008. How risky are social networking sites? A comparison of places online where youth sexual solicitation and harassment occurs. Pediatrics 121, 2 (2008)."},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.11345"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3579522","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3579522","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3579522","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:24Z","timestamp":1750182684000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3579522"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,14]]},"references-count":81,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2023,4,14]]}},"alternative-id":["10.1145\/3579522"],"URL":"https:\/\/doi.org\/10.1145\/3579522","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,14]]},"assertion":[{"value":"2023-04-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}