{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T21:46:23Z","timestamp":1780609583687,"version":"3.54.1"},"reference-count":50,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,5,21]],"date-time":"2024-05-21T00:00:00Z","timestamp":1716249600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003246","name":"Nederlandse Organisatie voor Wetenschappelijk Onderzoek","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Recently, content moderators on news platforms face the challenging task to select high-quality comments to feature on the webpage, a manual and time-consuming task exacerbated by platform growth. This paper introduces a group recommender system based on classifiers to aid moderators in this selection process.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Utilizing data from a Dutch news platform, we demonstrate that integrating comment data with user history and contextual relevance yields high ranking scores. To evaluate our models, we created realistic evaluation scenarios based on unseen online discussions from both 2020 and 2023, replicating changing news cycles and platform growth.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We demonstrate that our best-performing models maintain their ranking performance even when article topics change, achieving an optimum mean NDCG@5 of 0.89.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>The expert evaluation by platform-employed moderators underscores the subjectivity inherent in moderation practices, emphasizing the value of recommending comments over classification. Our research contributes to the advancement of (semi-)automated content moderation and the understanding of deliberation quality assessment in online discourse.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fdata.2024.1399739","type":"journal-article","created":{"date-parts":[[2024,5,21]],"date-time":"2024-05-21T04:42:01Z","timestamp":1716266521000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["A time-robust group recommender for featured comments on news platforms"],"prefix":"10.3389","volume":"7","author":[{"given":"Cedric","family":"Waterschoot","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antal","family":"van den Bosch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,5,21]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1016\/j.ins.2022.03.085","article-title":"Multi-view approach to suggest moderation actions in community question answering sites","volume":"600","author":"Annamoradnejad","year":"2022","journal-title":"Inf. Sci"},{"key":"B2","unstructured":"AQuA\u2013combining experts' and non-experts' views to assess deliberation quality in online discussions using LLMs\n            BehrendtM.\n            WagnerS. S.\n            ZiegeleM.\n            WilmsL.\n            StollA.\n            HeinbachD.\n          arXiv [Preprint]. arXiv2024"},{"key":"B3","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1109\/CBI.2019.00056","article-title":"\u201cEffect of transparency and trust on acceptance of automatic online comment moderation systems,\u201d","author":"Brunk","year":"2019","journal-title":"Proceedings"},{"key":"B4","doi-asserted-by":"crossref","first-page":"3255","DOI":"10.18653\/v1\/2020.findings-emnlp.292","article-title":"\u201cRobBERT: a Dutch RoBERTa-based language model,\u201d","volume-title":"Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020","author":"Delobelle","year":"2020"},{"key":"B5","first-page":"147","article-title":"Picking the NYT picks: editorial criteria and automation in the curation of online news comments","volume":"5","author":"Diakopoulos","year":"","journal-title":"ISOJ J"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1153","DOI":"10.1145\/2675133.2675160","article-title":"\u201cThe editor's eye: curation and comment relevance on the New York Times,\u201d","author":"Diakopoulos","year":"","journal-title":"CSCW 2015"},{"key":"B7","doi-asserted-by":"publisher","first-page":"2469","DOI":"10.18653\/v1\/2023.findings-eacl.187","article-title":"\u201cBridging argument quality and deliberative quality annotations with adapters,\u201d","author":"Falk","year":"2023","journal-title":"Findings of the Association for Computational Linguistics: EACL 2023"},{"key":"B8","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1002\/poi3.95","article-title":"A systematic review of online deliberation research","volume":"7","author":"Friess","year":"2015","journal-title":"Policy Internet"},{"key":"B9","volume-title":"Custodians of the Internet: Platforms, Content Moderation, and The hidden Decisions that Shape Social Media","author":"Gillespie","year":"2018"},{"key":"B10","doi-asserted-by":"publisher","first-page":"205395172094323","DOI":"10.1177\/2053951720943234","article-title":"Content moderation, AI, and the question of scale","volume":"7","author":"Gillespie","year":"2020","journal-title":"Big Data Soc"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.14763\/2020.4.1512","article-title":"Expanding the debate about content moderation: scholarly research agendas for the coming policy debates","volume":"9","author":"Gillespie","year":"2020","journal-title":"Internet Policy Rev"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.1342324","article-title":"Removals of online hate speech in numbers","author":"Gollatz","year":"2018","journal-title":"HIIG Science Blog"},{"key":"B13","doi-asserted-by":"publisher","first-page":"2053951719897945","DOI":"10.1177\/2053951719897945","article-title":"Algorithmic content moderation: technical and political challenges in the automation of platform governance","volume":"7","author":"Gorwa","year":"2020","journal-title":"Big Data Soc"},{"key":"B14","article-title":"\u201cEffects of empowerment moderation in online discussions: a field experiment with four news outlets,\u201d","volume-title":"72nd Annual Conference of the International Communication Association (ICA), 26-30 May 2022","author":"Heinbach","year":"2022"},{"key":"B15","first-page":"41","article-title":"\u201cIR evaluation methods for retrieving highly relevant documents,\u201d","volume-title":"SIGIR Forum (ACM Special Interest Group on Information Retrieval)","author":"Jarvelin","year":"2000"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3534929","article-title":"A trade-off-centered framework of content moderation","volume":"30","author":"Jiang","year":"2023","journal-title":"ACM Trans. Comput. Hum. Interact"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/ijep.2014010101","article-title":"The challenges for online deliberation research","volume":"5","author":"Jonsson","year":"2014","journal-title":"Int. J. E-Politics"},{"key":"B18","first-page":"11","article-title":"\u201cConstructive language in news comments,\u201d","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics","author":"Kolhatkar","year":"2017"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.5210\/fm.v28i4.13163","article-title":"Classifying constructive comments","author":"Kolhatkar","year":"2023","journal-title":"First Monday"},{"key":"B20","doi-asserted-by":"crossref","DOI":"10.1145\/3491102.3501999","article-title":"\u201cHuman-AI collaboration via conditional delegation: a case study of content moderation,\u201d","volume-title":"Conference on Human Factors in Computing Systems Proceedings","author":"Lai","year":"2022"},{"key":"B21","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1162\/coli_a_00364","article-title":"Argument mining: a survey","volume":"45","author":"Lawrence","year":"2020","journal-title":"Comput. Linguist"},{"key":"B22","article-title":"\u201cHuman-AI collaboration in content moderation: the effects of information cues and time constraints,\u201d","volume-title":"European Conference on Information Systems (ECIS) 2023","author":"Li","year":"2023"},{"key":"B23","unstructured":"\u201cDecoupled weight decay regularization,\u201d\n            LoshchilovI.\n            HutterF.\n          38536692New Orleans, LA7th International Conference on Learning Representations, ICLR 2019, May 6\u20139 20192019"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1052","DOI":"10.1080\/21670811.2018.1498295","article-title":"Audience engagement in a post-truth age: what it means and how to learn the activities connected with it","volume":"6","author":"Meier","year":"2018","journal-title":"Digit. Journal"},{"key":"B25","doi-asserted-by":"publisher","first-page":"zmac010","DOI":"10.1093\/jcmc\/zmac010","article-title":"When AI moderates online content: effects of human collaboration and interactive transparency on user trust","volume":"27","author":"Molina","year":"2022","journal-title":"J. Comput.-Mediat. Commun"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1365","DOI":"10.1002\/asi.24637","article-title":"The emerging science of content labeling: contextualizing social media content moderation","volume":"73","author":"Morrow","year":"2022","journal-title":"J. Assoc. Inf. Sci. Technol"},{"key":"B27","doi-asserted-by":"publisher","first-page":"13","DOI":"10.18653\/v1\/W17-0802","article-title":"\u201cFinding good conversations online: the yahoo news annotated comments corpus,\u201d","author":"Napoles","year":"2017","journal-title":"LAW 2017"},{"key":"B28","unstructured":"NUjij laat met expertlabels de kennis en expertise van gebruikers zien2020"},{"key":"B29","unstructured":"NUjij2018"},{"key":"B30","doi-asserted-by":"publisher","first-page":"224","DOI":"10.1080\/1461670X.2021.2017793","article-title":"\u201cThe boundaries are blurry\u2026\u201d: how comment moderators in Germany see and respond to hate comments","volume":"3","author":"Paasch-Colberg","year":"2022","journal-title":"Journal. Stud"},{"key":"B31","first-page":"1114","article-title":"\u201cSupporting comment moderators in identifying high quality online news comments,\u201d","volume-title":"Conference on Human Factors in Computing Systems Proceedings","author":"Park","year":"2016"},{"key":"B32","doi-asserted-by":"publisher","first-page":"36","DOI":"10.17645\/mac.v6i4.1519","article-title":"Dark participation","volume":"6","author":"Quandt","year":"2018","journal-title":"Media Commun"},{"key":"B33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/14648849231192789","article-title":"Euphoria, disillusionment and fear: twenty-five years of digital journalism (research)","volume":"25","author":"Quandt","year":"2023","journal-title":"Journalism"},{"key":"B34","doi-asserted-by":"publisher","first-page":"101284","DOI":"10.1016\/j.copsyc.2021.11.004","article-title":"Dark social media participation and well-being","volume":"45","author":"Quandt","year":"2022","journal-title":"Curr. Opin. Psychol"},{"key":"B35","doi-asserted-by":"publisher","first-page":"749","DOI":"10.1007\/s10462-021-10043-x","article-title":"News recommender system: a review of recent progress, challenges, and opportunities","volume":"55","author":"Raza","year":"2022","journal-title":"Artif. Intell. Rev"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.121542","article-title":"NBIAS: a natural language processing framework for BIAS identification in text","author":"Raza","year":"2024","journal-title":"Expert Syst. Appl"},{"key":"B37","doi-asserted-by":"publisher","first-page":"3982","DOI":"10.18653\/v1\/D19-1410","article-title":"\u201cSentence-BERT: sentence embeddings using siamese BERT-networks,\u201d","author":"Reimers","year":"2020","journal-title":"EMNLP-IJCNLP 2019"},{"key":"B38","doi-asserted-by":"publisher","first-page":"567","DOI":"10.18653\/v1\/P19-1054","article-title":"\u201cClassification and clustering of arguments with contextualized word embeddings,\u201d","author":"Reimers","year":"2020","journal-title":"ACL 2019"},{"key":"B39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-3-319-32001-4_44-1","author":"Roberts","year":"2017","journal-title":"Content Moderation. Encyclopedia of Big Data"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1026","DOI":"10.1177\/1461444819875990","article-title":"Re-humanizing the platform: content moderators and the logic of care","volume":"22","author":"Ruckenstein","year":"2020","journal-title":"New Media Soc"},{"key":"B41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/su13031187","article-title":"Assessment of online deliberative quality: new indicators using network analysis and time-series analysis","volume":"13","author":"Shin","year":"2021","journal-title":"Sustainability"},{"key":"B42","unstructured":"Van HoekC.\n          Hoe NU.nl beter wordt van een robot2020"},{"key":"B43","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1080\/21670811.2020.1851279","article-title":"Moderating uncivil user comments by humans or machines? The effects of moderation agent on perceptions of bias and credibility in news content","volume":"9","author":"Wang","year":"2021","journal-title":"Digit. Journal"},{"key":"B44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3484245","article-title":"Highlighting high-quality content as a moderation strategy: the role of new york times picks in comment quality","volume":"4","author":"Wang","year":"2022","journal-title":"ACM Trans. Soc. Comput"},{"key":"B45","first-page":"25","article-title":"\u201cA theoretical analysis of ndcg type ranking measures,\u201d","author":"Wang","year":"2013","journal-title":"Proceedings of the 26th Annual Conference on Learning Theory, volume 30 of Proceedings of Machine Learning Research"},{"key":"B46","first-page":"6715","article-title":"\u201cDetecting minority arguments for mutual understanding: a moderation tool for the online climate change debate,\u201d","volume-title":"Proceedings of the 29th International Conference on Computational Linguistics","author":"Waterschoot","year":"2022"},{"key":"B47","doi-asserted-by":"publisher","first-page":"904","DOI":"10.1080\/21670811.2020.1797519","article-title":"How to cope with dark participation: moderation practices in German newsrooms","volume":"8","author":"Wintterlin","year":"2020","journal-title":"Digit. Journal"},{"key":"B48","doi-asserted-by":"crossref","first-page":"38","DOI":"10.18653\/v1\/2020.emnlp-demos.6","article-title":"\u201cTransformers: state-of-the-art natural language processing,\u201d","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations","author":"Wolf","year":"2020"},{"key":"B49","doi-asserted-by":"publisher","first-page":"103094","DOI":"10.1016\/j.jnca.2021.103094","article-title":"Minimizing the spread of misinformation in online social networks: a survey","volume":"186","author":"Zareie","year":"2021","journal-title":"J. Netw. Comput. Appl"},{"key":"B50","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1177\/0093650218797884","article-title":"Linking news value theory with online deliberation: how news factors and illustration factors in news articles affect the deliberative quality of user discussions in SNS' comment sections","volume":"47","author":"Ziegele","year":"2020","journal-title":"Commun. Res"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2024.1399739\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,21]],"date-time":"2024-05-21T04:42:17Z","timestamp":1716266537000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2024.1399739\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,21]]},"references-count":50,"alternative-id":["10.3389\/fdata.2024.1399739"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2024.1399739","relation":{},"ISSN":["2624-909X"],"issn-type":[{"value":"2624-909X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,21]]},"article-number":"1399739"}}