{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T21:06:58Z","timestamp":1781212018594,"version":"3.54.1"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation","award":["IIS-1845460"],"award-info":[{"award-number":["IIS-1845460"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2022,11,7]]},"abstract":"<jats:p>The scale of scientific publishing continues to grow, creating overload on science journalists who are inundated with choices for what would be most interesting, important, and newsworthy to cover in their reporting. Our work addresses this problem by considering the viability of creating a predictive model of newsworthiness of scientific articles that is trained using crowdsourced evaluations of newsworthiness. We proceed by first evaluating the potential of crowd-sourced evaluations of newsworthiness by assessing their alignment with expert ratings of newsworthiness, analyzing both quantitative correlations and qualitative rating rationale to understand limitations. We then demonstrate and evaluate a predictive model trained on these crowd ratings together with arXiv article metadata, text, and other computed features. Based on the crowdsourcing protocol we developed, we find that while crowdsourced ratings of newsworthiness often align moderately with expert ratings, there are also notable differences and divergences which limit the approach. Yet despite these limitations we also find that the predictive model we built provides a reasonably precise set of rankings when validated against expert evaluations (P@10 = 0.8, P@15 = 0.67), suggesting that a viable signal can be learned from crowdsourced evaluations of newsworthiness. Based on these findings we discuss opportunities for future work to leverage crowdsourcing and predictive approaches to support journalistic work in discovering and filtering newsworthy information.<\/jats:p>","DOI":"10.1145\/3555542","type":"journal-article","created":{"date-parts":[[2022,11,11]],"date-time":"2022-11-11T22:58:54Z","timestamp":1668207534000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["From Crowd Ratings to Predictive Models of Newsworthiness to Support Science Journalism"],"prefix":"10.1145","volume":"6","author":[{"given":"Sachita","family":"Nishal","sequence":"first","affiliation":[{"name":"Northwestern University, Evanston, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas","family":"Diakopoulos","sequence":"additional","affiliation":[{"name":"Northwestern University, Evanston, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,11,11]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1080\/21670811.2015.1034807"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1177\/1464884911412688"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1515\/nor-2017-0327"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1177\/1464884911402451"},{"key":"e_1_2_1_5_1","volume-title":"How to write a good abstract for a scientific paper or conference presentation. Indian journal of psychiatry","author":"Andrade Chittaranjan","year":"2011","unstructured":"Chittaranjan Andrade . 2011. How to write a good abstract for a scientific paper or conference presentation. Indian journal of psychiatry , Vol. 53 , 2 ( April 2011 ), 172--175. https:\/\/doi.org\/10.4103\/0019--5545.82558 Publisher : Medknow Publications . 10.4103\/0019--5545.82558 Chittaranjan Andrade. 2011. How to write a good abstract for a scientific paper or conference presentation. Indian journal of psychiatry, Vol. 53, 2 (April 2011), 172--175. https:\/\/doi.org\/10.4103\/0019--5545.82558 Publisher: Medknow Publications."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli.07-034-R2"},{"key":"e_1_2_1_7_1","volume-title":"The basics of social research","author":"Babbie Earl R.","unstructured":"Earl R. Babbie . 2008. The basics of social research 4 th ed ed.). Thomson\/Wadsworth , Belmont, CA . Earl R. Babbie. 2008. The basics of social research 4th ed ed.). Thomson\/Wadsworth, Belmont, CA.","edition":"4"},{"key":"e_1_2_1_8_1","volume-title":"Simone R\u00f6dder, Martina Franzen, and Peter Weingart (Eds.).","author":"Badenschier Franziska","year":"2085","unstructured":"Franziska Badenschier and Holger Wormer . 2012. Issue Selection in Science Journalism: Towards a Special Theory of News Values for Science News? In The Sciences' Media Connection --Public Communication and its Repercussions , Simone R\u00f6dder, Martina Franzen, and Peter Weingart (Eds.). Vol. 28 . Springer Netherlands , Dordrecht , 59--85. https:\/\/doi.org\/10.1007\/978--94-007-- 2085 --5_4 Series Title : Sociology of the Sciences Yearbook. 10.1007\/978--94-007--2085--5_4 Franziska Badenschier and Holger Wormer. 2012. Issue Selection in Science Journalism: Towards a Special Theory of News Values for Science News? In The Sciences' Media Connection --Public Communication and its Repercussions, Simone R\u00f6dder, Martina Franzen, and Peter Weingart (Eds.). Vol. 28. Springer Netherlands, Dordrecht, 59--85. https:\/\/doi.org\/10.1007\/978--94-007--2085--5_4 Series Title: Sociology of the Sciences Yearbook."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1080\/21670811.2021.1912622"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1080\/17512780701768568"},{"key":"e_1_2_1_11_1","volume-title":"Science journalism grows up. Science","author":"Blum Deborah","year":"2021","unstructured":"Deborah Blum . 2021. Science journalism grows up. Science , Vol. 372 , 6540 ( April 2021 ), 323--323. https:\/\/doi.org\/10.1126\/science.abj0434 Publisher : American Association for the Advancement of Science. 10.1126\/science.abj0434 Deborah Blum. 2021. Science journalism grows up. Science, Vol. 372, 6540 (April 2021), 323--323. https:\/\/doi.org\/10.1126\/science.abj0434 Publisher: American Association for the Advancement of Science."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1057\/s41599-021-00903-w"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_2_1_15_1","volume-title":"Taylor Owen, Derek Ruths, Lisa Teichmann, and Oleg Zhilin.","author":"Bridgman Aengus","year":"2020","unstructured":"Aengus Bridgman , Eric Merkley , Peter John Loewen , Taylor Owen, Derek Ruths, Lisa Teichmann, and Oleg Zhilin. 2020 . The causes and consequences of COVID-19 misperceptions: Understanding the role of news and social media. Harvard Kennedy School Misinformation Review (June 2020). https:\/\/doi.org\/10.37016\/mr-2020-028 10.37016\/mr-2020-028 Aengus Bridgman, Eric Merkley, Peter John Loewen, Taylor Owen, Derek Ruths, Lisa Teichmann, and Oleg Zhilin. 2020. The causes and consequences of COVID-19 misperceptions: Understanding the role of news and social media. Harvard Kennedy School Misinformation Review (June 2020). https:\/\/doi.org\/10.37016\/mr-2020-028"},{"key":"e_1_2_1_16_1","volume-title":"News values","author":"Brighton Paul","unstructured":"Paul Brighton and Dennis Foy . 2007. News values . SAGE Publications, London ; Thousand Oaks, Calif. OCLC : ocn153557904. Paul Brighton and Dennis Foy. 2007. News values. SAGE Publications, London ; Thousand Oaks, Calif. OCLC: ocn153557904."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1232329"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444812440159"},{"key":"e_1_2_1_19_1","volume-title":"Handbook of Human Computation","author":"Chandler Jesse","unstructured":"Jesse Chandler , Gabriele Paolacci , and Pam Mueller . 2013. Risks and Rewards of Crowdsourcing Marketplaces . In Handbook of Human Computation , Pietro Michelucci (Ed.). Springer New York , New York, NY , 377--392. https:\/\/doi.org\/10.1007\/978--1--4614--8806--4_30 10.1007\/978--1--4614--8806--4_30 Jesse Chandler, Gabriele Paolacci, and Pam Mueller. 2013. Risks and Rewards of Crowdsourcing Marketplaces. In Handbook of Human Computation, Pietro Michelucci (Ed.). Springer New York, New York, NY, 377--392. https:\/\/doi.org\/10.1007\/978--1--4614--8806--4_30"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053168015622072"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444818815684"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025659"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1080\/21670811.2020.1736946"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208409"},{"key":"e_1_2_1_25_1","volume-title":"Proc. Computation Journalism Symposium.","author":"Diakopoulos Nicholas","year":"2020","unstructured":"Nicholas Diakopoulos , Madison Dong , Leonard Bronner , and Jeremy Bowers . 2020 . Generating Location-Based News Leads for National Politics Reporting . In Proc. Computation Journalism Symposium. Nicholas Diakopoulos, Madison Dong, Leonard Bronner, and Jeremy Bowers. 2020. Generating Location-Based News Leads for National Politics Reporting. In Proc. Computation Journalism Symposium."},{"key":"e_1_2_1_26_1","article-title":"Algorithmic Transparency in the News Media","volume":"5","author":"Diakopoulos Nicholas","year":"2016","unstructured":"Nicholas Diakopoulos and Michael Koliska . 2016 . Algorithmic Transparency in the News Media . Digital Journalism , Vol. 5 , 7 (08 2016), 1--20. https:\/\/doi.org\/10.1080\/21670811.2016.1208053 10.1080\/21670811.2016.1208053 Nicholas Diakopoulos and Michael Koliska. 2016. Algorithmic Transparency in the News Media. Digital Journalism, Vol. 5, 7 (08 2016), 1--20. https:\/\/doi.org\/10.1080\/21670811.2016.1208053","journal-title":"Digital Journalism"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3479550"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1080\/21670811.2013.841368"},{"key":"e_1_2_1_29_1","volume-title":"Science journalism","author":"Dunwoody Sharon","year":"2034","unstructured":"Sharon Dunwoody . 2012. Science journalism . In Routledge Handbook of Public Communication of Science and Technology. Routledge . https:\/\/doi.org\/10.4324\/9780 2034 83794.ch3 10.4324\/9780203483794.ch3 Sharon Dunwoody. 2012. Science journalism. In Routledge Handbook of Public Communication of Science and Technology. Routledge. https:\/\/doi.org\/10.4324\/9780203483794.ch3"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1177\/1464884911412697"},{"key":"e_1_2_1_31_1","article-title":"From \"Public Journalism\" to \"Engaged Journalism\": Imagined Audiences and Denigrating Discourse","volume":"14","author":"Ferrucci Patrick","year":"2020","unstructured":"Patrick Ferrucci , Jacob L Nelson , and Miles P Davis . 2020 . From \"Public Journalism\" to \"Engaged Journalism\": Imagined Audiences and Denigrating Discourse . International Journal of Communication , Vol. 14 , 0 (02 2020), 19. Patrick Ferrucci, Jacob L Nelson, and Miles P Davis. 2020. From \"Public Journalism\" to \"Engaged Journalism\": Imagined Audiences and Denigrating Discourse. International Journal of Communication, Vol. 14, 0 (02 2020), 19.","journal-title":"International Journal of Communication"},{"key":"e_1_2_1_32_1","volume-title":"Building the ?Truthmeter': Training algorithms to help journalists assess the credibility of social media sources. Convergence","author":"Fletcher Richard","year":"2020","unstructured":"Richard Fletcher , Steve Schifferes , and Neil Thurman . 2020. Building the ?Truthmeter': Training algorithms to help journalists assess the credibility of social media sources. Convergence , Vol. 26 , 1 ( Feb. 2020 ), 19--34. https:\/\/doi.org\/10.1177\/1354856517714955 Publisher : SAGE Publications Ltd . 10.1177\/1354856517714955 Richard Fletcher, Steve Schifferes, and Neil Thurman. 2020. Building the ?Truthmeter': Training algorithms to help journalists assess the credibility of social media sources. Convergence, Vol. 26, 1 (Feb. 2020), 19--34. https:\/\/doi.org\/10.1177\/1354856517714955 Publisher: SAGE Publications Ltd."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.17645\/mac.v8i3.3228"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1177\/002234336500200104"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557228"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-006-6226-1"},{"key":"e_1_2_1_37_1","volume-title":"Making the news","author":"Golding Peter","unstructured":"Peter Golding and Philip Ross Courtney Elliott . 1979. Making the news 1 st ed ed.). Longman, London ; New York . Peter Golding and Philip Ross Courtney Elliott. 1979. Making the news 1st ed ed.). Longman, London ; New York.","edition":"1"},{"key":"#cr-split#-e_1_2_1_38_1.1","unstructured":"Andreas Graefe. 2016. Guide to Automated Journalism. (2016). https:\/\/doi.org\/10.7916\/D80G3XDJ Publisher: Columbia University. 10.7916\/D80G3XDJ"},{"key":"#cr-split#-e_1_2_1_38_1.2","unstructured":"Andreas Graefe. 2016. Guide to Automated Journalism. (2016). https:\/\/doi.org\/10.7916\/D80G3XDJ Publisher: Columbia University."},{"key":"e_1_2_1_39_1","volume-title":"A Question of Design: Strategies for Embedding AI-Driven Tools into Journalistic Work Routines. Digital Journalism (March","author":"Lopez Marisela Gutierrez","year":"2022","unstructured":"Marisela Gutierrez Lopez , Colin Porlezza , Glenda Cooper , Stephann Makri , Andrew MacFarlane , and Sondess Missaoui . 2022. A Question of Design: Strategies for Embedding AI-Driven Tools into Journalistic Work Routines. Digital Journalism (March 2022 ), 1--20. https:\/\/doi.org\/10.1080\/21670811.2022.2043759 10.1080\/21670811.2022.2043759 Marisela Gutierrez Lopez, Colin Porlezza, Glenda Cooper, Stephann Makri, Andrew MacFarlane, and Sondess Missaoui. 2022. A Question of Design: Strategies for Embedding AI-Driven Tools into Journalistic Work Routines. Digital Journalism (March 2022), 1--20. https:\/\/doi.org\/10.1080\/21670811.2022.2043759"},{"key":"e_1_2_1_40_1","volume-title":"All the News That's Fit to Sell","author":"Hamilton James","unstructured":"James Hamilton . 2004. All the News That's Fit to Sell . Princeton University Press . https:\/\/doi.org\/10.2307\/j.ctt7smgs 10.2307\/j.ctt7smgs James Hamilton. 2004. All the News That's Fit to Sell. Princeton University Press. https:\/\/doi.org\/10.2307\/j.ctt7smgs"},{"key":"e_1_2_1_41_1","volume-title":"Accountability Through Algorithm: Developing the Field of Computational Journalism","author":"Hamilton James","year":"2009","unstructured":"James Hamilton and Fred Turner . 2009. Accountability Through Algorithm: Developing the Field of Computational Journalism . Center For Advanced Study in the Behavioral Sciences Summer Workshop ( 2009 ). http:\/\/web.stanford.edu\/ fturner\/Hamilton%20Turner%20Acc%20by%20Alg%20Final.pdf James Hamilton and Fred Turner. 2009. Accountability Through Algorithm: Developing the Field of Computational Journalism. Center For Advanced Study in the Behavioral Sciences Summer Workshop (2009). http:\/\/web.stanford.edu\/ fturner\/Hamilton%20Turner%20Acc%20by%20Alg%20Final.pdf"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1080\/14616700118449"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1080\/1461670X.2016.1150193"},{"key":"e_1_2_1_44_1","volume-title":"The engaged scholar : expanding the impact of academic research in today's world","author":"Hoffman Andrew J.","unstructured":"Andrew J. Hoffman . 2021. The engaged scholar : expanding the impact of academic research in today's world . Stanford University Press . Andrew J. Hoffman. 2021. The engaged scholar : expanding the impact of academic research in today's world. Stanford University Press."},{"key":"e_1_2_1_45_1","unstructured":"Alok Jha. 2010. What is a story tracker? https:\/\/www.theguardian.com\/science\/blog\/2010\/jun\/09\/science-story-trackers  Alok Jha. 2010. What is a story tracker? https:\/\/www.theguardian.com\/science\/blog\/2010\/jun\/09\/science-story-trackers"},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the 24th annual ACM symposium on User interface software and technology - UIST '11. ACM Press","author":"Kittur Aniket","year":"2047","unstructured":"Aniket Kittur , Boris Smus , Susheel Khamkar , and Robert E. Kraut . 2011. CrowdForge: crowdsourcing complex work . In Proceedings of the 24th annual ACM symposium on User interface software and technology - UIST '11. ACM Press , Santa Barbara, California, USA, 43. https:\/\/doi.org\/10.1145\/ 2047 196.2047202 10.1145\/2047196.2047202 Aniket Kittur, Boris Smus, Susheel Khamkar, and Robert E. Kraut. 2011. CrowdForge: crowdsourcing complex work. In Proceedings of the 24th annual ACM symposium on User interface software and technology - UIST '11. ACM Press, Santa Barbara, California, USA, 43. https:\/\/doi.org\/10.1145\/2047196.2047202"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-010-0202-z"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1080\/19312458.2017.1317338"},{"key":"e_1_2_1_49_1","volume-title":"Reuters Tracer: Toward Automated News Production Using Large Scale Social Media Data. arXiv:1711.04068 [cs] (Nov.","author":"Liu Xiaomo","year":"2017","unstructured":"Xiaomo Liu , Armineh Nourbakhsh , Quanzhi Li , Sameena Shah , Robert Martin , and John Duprey . 2017 . Reuters Tracer: Toward Automated News Production Using Large Scale Social Media Data. arXiv:1711.04068 [cs] (Nov. 2017). http:\/\/arxiv.org\/abs\/1711.04068 arXiv: 1711.04068. Xiaomo Liu, Armineh Nourbakhsh, Quanzhi Li, Sameena Shah, Robert Martin, and John Duprey. 2017. Reuters Tracer: Toward Automated News Production Using Large Scale Social Media Data. arXiv:1711.04068 [cs] (Nov. 2017). http:\/\/arxiv.org\/abs\/1711.04068 arXiv: 1711.04068."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1080\/21670811.2015.1081822"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v12i1.14999"},{"key":"e_1_2_1_52_1","unstructured":"M\u00e5ns Magnusson Jens Finn\u00e4s and Leonard Wallentin. 2016. Finding the news lead in the data haystack: Automated local data journalism using crime data. (2016) 4.  M\u00e5ns Magnusson Jens Finn\u00e4s and Leonard Wallentin. 2016. Finding the news lead in the data haystack: Automated local data journalism using crime data. (2016) 4."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-011-0124-6"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3415204"},{"key":"e_1_2_1_55_1","volume-title":"Inioluwa Deborah Raji, and Timnit Gebru","author":"Mitchell Margaret","year":"2019","unstructured":"Margaret Mitchell , Simone Wu , Andrew Zaldivar , Parker Barnes , Lucy Vasserman , Ben Hutchinson , Elena Spitzer , Inioluwa Deborah Raji, and Timnit Gebru . 2019 . Model cards for model reporting. (2019), 220--229. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. (2019), 220--229."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3005513"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3461702.3462608"},{"key":"e_1_2_1_58_1","unstructured":"Nature. 2019. How to Write a Nature Summary. https:\/\/www.nature.com\/documents\/nature-summary-paragraph.pdf  Nature. 2019. How to Write a Nature Summary. https:\/\/www.nature.com\/documents\/nature-summary-paragraph.pdf"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.17645\/mac.v8i3.3001"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0181742"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1080\/14616700600757928"},{"key":"e_1_2_1_62_1","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Reimers Nils","year":"1865","unstructured":"Nils Reimers and Iryna Gurevych . 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Association for Computational Linguistics , Hong Kong , China, 3980--3990. https:\/\/doi.org\/10. 1865 3\/v1\/D19--1410 10.18653\/v1 Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 3980--3990. https:\/\/doi.org\/10.18653\/v1\/D19--1410"},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the Twentieth International Conference on Machine Learning (ICML 2003), Washington, D.C. proceedings of the twentieth international conference on machine learning (icml","author":"Rennie Jason D. M.","year":"2003","unstructured":"Jason D. M. Rennie , Lawrence Shih , Jaime Teevan , and David R. Karger . 2003. Tackling the Poor Assumptions of Naive Bayes Text Classifiers . In Proceedings of the Twentieth International Conference on Machine Learning (ICML 2003), Washington, D.C. proceedings of the twentieth international conference on machine learning (icml 2003 ), washington, d.c. ed.). https:\/\/www.microsoft.com\/en-us\/research\/publication\/tackling-poor-assumptions-naive-bayes-text-classifiers\/ Jason D. M. Rennie, Lawrence Shih, Jaime Teevan, and David R. Karger. 2003. Tackling the Poor Assumptions of Naive Bayes Text Classifiers. In Proceedings of the Twentieth International Conference on Machine Learning (ICML 2003), Washington, D.C. proceedings of the twentieth international conference on machine learning (icml 2003), washington, d.c. ed.). https:\/\/www.microsoft.com\/en-us\/research\/publication\/tackling-poor-assumptions-naive-bayes-text-classifiers\/"},{"key":"e_1_2_1_64_1","volume-title":"Writing and reporting news: a coaching method","author":"Rich Carole","unstructured":"Carole Rich . 2016. Writing and reporting news: a coaching method 8 th ed ed.). Cengage Learning, Boston, MA . OCLC : ocn908085863. Carole Rich. 2016. Writing and reporting news: a coaching method 8th ed ed.). Cengage Learning, Boston, MA. OCLC: ocn908085863.","edition":"8"},{"key":"e_1_2_1_65_1","volume-title":"Analyzing Media Messages: Using Quantitative Content Analysis in Research","author":"Riffe Daniel","unstructured":"Daniel Riffe , Stephen Lacy , Brendan R. Watson , and Frederick Fico . 2019. Analyzing Media Messages: Using Quantitative Content Analysis in Research 4 th ed.). Lawrence Erlbaum, Mahwah, NJ , USA. Daniel Riffe, Stephen Lacy, Brendan R. Watson, and Frederick Fico. 2019. Analyzing Media Messages: Using Quantitative Content Analysis in Research 4th ed.). Lawrence Erlbaum, Mahwah, NJ, USA.","edition":"4"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1080\/17512780701275507"},{"key":"e_1_2_1_67_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media","volume":"9","author":"Schwartz Raz","year":"2015","unstructured":"Raz Schwartz , Mor Naaman , and Rannie Teodoro . 2015 . Editorial Algorithms: Using Social Media to Discover and Report Local News . Proceedings of the International AAAI Conference on Web and Social Media , Vol. 9 , 1 (April 2015). https:\/\/ojs.aaai.org\/index.php\/ICWSM\/article\/view\/14633 Number : 1. Raz Schwartz, Mor Naaman, and Rannie Teodoro. 2015. Editorial Algorithms: Using Social Media to Discover and Report Local News. Proceedings of the International AAAI Conference on Web and Social Media, Vol. 9, 1 (April 2015). https:\/\/ojs.aaai.org\/index.php\/ICWSM\/article\/view\/14633 Number: 1."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173955"},{"key":"e_1_2_1_69_1","volume-title":"Modeling \"Newsworthiness\" for Lead-Generation Across Corpora. arXiv:2104.09653 [cs] (April","author":"Spangher Alexander","year":"2021","unstructured":"Alexander Spangher , Nanyun Peng , Jonathan May , and Emilio Ferrara . 2021. Modeling \"Newsworthiness\" for Lead-Generation Across Corpora. arXiv:2104.09653 [cs] (April 2021 ). http:\/\/arxiv.org\/abs\/2104.09653 arXiv: 2104.09653. Alexander Spangher, Nanyun Peng, Jonathan May, and Emilio Ferrara. 2021. Modeling \"Newsworthiness\" for Lead-Generation Across Corpora. arXiv:2104.09653 [cs] (April 2021). http:\/\/arxiv.org\/abs\/2104.09653 arXiv: 2104.09653."},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1177\/0963662508096776"},{"key":"e_1_2_1_71_1","volume-title":"Thomas and Scott Clifford","author":"Kyle","year":"2017","unstructured":"Kyle A. Thomas and Scott Clifford . 2017 . Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments. Computers in Human Behavior , Vol. 77 (Dec. 2017), 184--197. https:\/\/doi.org\/10.1016\/j.chb.2017.08.038 10.1016\/j.chb.2017.08.038 Kyle A. Thomas and Scott Clifford. 2017. Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments. Computers in Human Behavior, Vol. 77 (Dec. 2017), 184--197. https:\/\/doi.org\/10.1016\/j.chb.2017.08.038"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1177\/1077699016654682"},{"key":"e_1_2_1_73_1","volume-title":"Crowdsourcing in Investigative Journalism","author":"Vehkoo Johanna","unstructured":"Johanna Vehkoo . 2013. Crowdsourcing in Investigative Journalism . Reuters Institute for the Study of Journalism, Vol. Oxford: Report (2013). Johanna Vehkoo. 2013. Crowdsourcing in Investigative Journalism. Reuters Institute for the Study of Journalism, Vol. Oxford: Report (2013)."},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v5i1.13317"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555542","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3555542","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:02Z","timestamp":1750182542000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555542"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":75,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2022,11,7]]}},"alternative-id":["10.1145\/3555542"],"URL":"https:\/\/doi.org\/10.1145\/3555542","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,7]]},"assertion":[{"value":"2022-11-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}