{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T14:24:08Z","timestamp":1772893448827,"version":"3.50.1"},"reference-count":59,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"name":"Credibility Coalition, a project of Meedan and Hacks\/Hackers"},{"name":"Lagrange Project of the Institute for Scientific Interchange Foundation"},{"name":"Fondazione Cassa di Risparmio di Torino"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Web"],"published-print":{"date-parts":[[2026,2,28]]},"abstract":"<jats:p>\n                    Over the last few years, verifying the credibility of information sources has become a fundamental need to combat disinformation. Here, we present a language-agnostic model designed to assess the reliability of web domains as sources in references across multiple language editions of Wikipedia. Utilizing editing activity data, the model evaluates domain reliability within different articles of varying controversiality, such as Climate Change, COVID-19, History, Media, and Biology topics. Crafting features that express domain usage across articles, the model effectively predicts domain reliability, achieving an F1 Macro score of approximately 0.80 for English and other high-resource languages. For mid-resource languages, we achieve 0.65, while the performance of low-resource languages varies. In all cases, the time the domain remains present in the articles (which we dub as\n                    <jats:italic toggle=\"yes\">permanence<\/jats:italic>\n                    ) is one of the most predictive features. We highlight the challenge of maintaining consistent model performance across languages of varying resource levels and demonstrate that adapting models from higher-resource languages can improve performance. We believe these findings can assist Wikipedia editors in their ongoing efforts to verify citations and may offer useful insights for other user-generated content communities.\n                  <\/jats:p>","DOI":"10.1145\/3777444","type":"journal-article","created":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T11:05:22Z","timestamp":1763463922000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Language-Agnostic Modeling of Source Reliability on Wikipedia"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2843-5279","authenticated-orcid":false,"given":"Jacopo","family":"D\u2019Ignazi","sequence":"first","affiliation":[{"name":"Universitat Pompeu Fabra","place":["Barcelona, Spain"]},{"name":"ISI Foundation","place":["Barcelona, Spain"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2271-3066","authenticated-orcid":false,"given":"Andreas","family":"Kaltenbrunner","sequence":"additional","affiliation":[{"name":"Universitat Oberta de Catalunya","place":["Barcelona, Spain"]},{"name":"ISI Foundation","place":["Barcelona, Spain"]},{"name":"Universitat Pompeu Fabra","place":["Barcelona, Spain"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5560-4109","authenticated-orcid":false,"given":"Yelena","family":"Mejova","sequence":"additional","affiliation":[{"name":"ISI Foundation","place":["Torino, Italy"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3068-5111","authenticated-orcid":false,"given":"Michele","family":"Tizzani","sequence":"additional","affiliation":[{"name":"ISI Foundation","place":["Torino, Italy"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8068-5916","authenticated-orcid":false,"given":"Kyriaki","family":"Kalimeri","sequence":"additional","affiliation":[{"name":"ISI Foundation","place":["Torino, Italy"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5474-0309","authenticated-orcid":false,"given":"Mariano G.","family":"Beir\u00f3","sequence":"additional","affiliation":[{"name":"Universidad de San Andr\u00e9s","place":["Buenos Aires, Argentina"]},{"name":"CONICET","place":["Buenos Aires, Argentina"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6017-4577","authenticated-orcid":false,"given":"Pablo","family":"Arag\u00f3n","sequence":"additional","affiliation":[{"name":"Universitat Pompeu Fabra","place":["Barcelona, Spain"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,2,19]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"Pablo Arag\u00f3n and Diego S\u00e1ez-Trumper. 2021. A preliminary approach to knowledge integrity risk assessment in Wikipedia projects. arXiv:2106.15940. Retrieved from https:\/\/arxiv.org\/abs\/2106.15940"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v18i1.31300"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3645445"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3543507.3583218"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3615254"},{"key":"e_1_3_3_7_2","unstructured":"Golding Barret. 2021. Iffy index of unreliable sources. (2021). Retrieved April 03 2024 from https:\/\/iffy.news\/index\/"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1093\/gigascience\/giab095"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702436"},{"key":"e_1_3_3_10_2","first-page":"1","volume-title":"Proceedings of the 8th Annual International Symposium on Wikis and Open Collaboration","author":"Chen Chih-Chun","year":"2012","unstructured":"Chih-Chun Chen and Camille Roth. 2012. \\(\\lbrace\\) \\(\\lbrace\\) Citation needed \\(\\rbrace\\) \\(\\rbrace\\) the dynamics of referencing in wikipedia. In Proceedings of the 8th Annual International Symposium on Wikis and Open Collaboration. Association for Computing Machinery, New York, NY, USA, 1\u20134."},{"key":"e_1_3_3_11_2","volume-title":"Proceedings of the Wiki Workshop","author":"Chou Ai-Jou","year":"2020","unstructured":"Ai-Jou Chou, Guilherme Gon\u00e7alves, Sam Walton, and Miriam Redi. 2020. Citation detective: a public dataset to improve and quantify wikipedia citation quality at scale. In Proceedings of the Wiki Workshop. Wiki Workshop."},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3412066"},{"key":"e_1_3_3_13_2","unstructured":"Noam Cohen. 2021. One Woman\u2019s Mission to Rewrite Nazi History on Wikipedia. WIRED. [Online; accessed 15-Apr-2024]."},{"issue":"01","key":"e_1_3_3_14_2","first-page":"1","article-title":"Language-Agnostic Modeling of Wikipedia Articles for Content Quality Assessment across Languages","volume":"18","author":"Das Paramita","year":"2024","unstructured":"Paramita Das, Isaac Johnson, Diego Saez-Trumper, and Pablo Arag\u00f3n. 2024. Language-Agnostic Modeling of Wikipedia Articles for Content Quality Assessment across Languages. Proceedings of the International AAAI Conference on Web and Social Media 18, 01 (2024), 1\u201311.","journal-title":"Proceedings of the International AAAI Conference on Web and Social Media"},{"key":"e_1_3_3_15_2","first-page":"453","volume-title":"Proceedings of the 31st International Conference on Computational Linguistics: Industry Track","author":"Das Paramita","year":"2025","unstructured":"Paramita Das, Amartya Roy, Ritabrata Chakraborty, and Animesh Mukherjee. 2025. On the effective transfer of knowledge from English to Hindi Wikipedia. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track. Association for Computational Linguistics, Abu Dhabi, UAE, 453\u2013465. Retrieved from https:\/\/aclanthology.org\/2025.coling-industry.39\/"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1517441113"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1423"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-main.1268"},{"issue":"0","key":"e_1_3_3_19_2","first-page":"146144482211497","article-title":"Introducing an \u201cinvisible enemy\u201d: a case study of knowledge construction regarding microplastics in Japanese Wikipedia","volume":"0","author":"Fu Mengyuan","year":"2023","unstructured":"Mengyuan Fu, Kunhao Yang, and Yuko Fujigaki. 2023. Introducing an \u201cinvisible enemy\u201d: a case study of knowledge construction regarding microplastics in Japanese Wikipedia. New Media and Society 0, 0 (2023), 14614448221149747.","journal-title":"New Media and Society"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3481939"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3614936"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/PacificVis56936.2023.00025"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2615569.2615684"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3415219"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.nlp.2024.100093"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2025.127503"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442442.3452347"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","unstructured":"Isaac Johnson Lucie-Aim\u00e9e Kaffee and Miriam Redi. 2024. Wikimedia data for AI: a review of Wikimedia datasets for NLP tasks and AI-assisted editing. In Proceedings of the First Workshop on Advancing Natural Language Processing for Wikipedia Association for Computational Linguistics Miami Florida USA 91\u2013101. DOI:10.18653\/v1\/2024.wikinlp-1.14","DOI":"10.18653\/v1\/2024.wikinlp-1.14"},{"key":"e_1_3_3_29_2","unstructured":"Isaac Johnson and Emily Lescak. 2022. Considerations for multilingual wikipedia research. arXiv:2204.02483. Retrieved from https:\/\/arxiv.org\/abs\/2204.02483"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3637338"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.envsci.2023.103553"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883085"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.3390\/computers8030060"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.3390\/info11050263"},{"key":"e_1_3_3_35_2","first-page":"4765","volume-title":"Proceedings of the Advances in Neural Information Processing Systems.","author":"Lundberg Scott M","year":"2017","unstructured":"Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Proceedings of the Advances in Neural Information Processing Systems.I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Curran Associates, Inc., Red Hook, NY, USA, 4765\u20134774. Retrieved from http:\/\/papers.nips.cc\/paper\/7062-a-unified-approach-to-interpreting-model-predictions.pdf"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3449805"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313618"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2021.1994633"},{"key":"e_1_3_3_39_2","unstructured":"Richard Rogers and Emina Sendijarevic. 2012. Neutral or national point of view? A comparison of srebrenica articles across Wikipedia\u2019s language versions. In Wikipedia Academy: Research and Free Knowledge: June 29 - July 1 2012 Berlin: Accepted Submissions Wikepedia Academy."},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1111\/jcom.12082"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-94105-9_3"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539597.3570478"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2023.110838"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599823"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3481961"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aap9559"},{"key":"e_1_3_3_47_2","unstructured":"Wikipedia contributors. 2024. English wikipedia. 2024. wikipedia:reliable sources\/Perennial sources. Retrieved April 12 2024 from https:\/\/en.wikipedia.org\/wiki\/Wikipedia:Reliable_sources\/Perennial_sources"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.18"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463253"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3615015"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3645468"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3614914"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3488560.3498457"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1108\/OIR-02-2023-0084"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330709"},{"key":"e_1_3_3_56_2","unstructured":"Dave Van Zandt. 2024. Media Bias\/Fact-Check. (2024). Retrieved April 23 2024 from https:\/\/mediabiasfactcheck.com\/"},{"key":"e_1_3_3_57_2","first-page":"5484","volume-title":"Proceedings of the Advances in Neural Information Processing Systems.","volume":"36","author":"Zhang Wenxuan","year":"2023","unstructured":"Wenxuan Zhang, Mahani Aljunied, Chang Gao, Yew Ken Chia, and Lidong Bing. 2023. M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models. In Proceedings of the Advances in Neural Information Processing Systems.A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, Curran Associates, Inc., New York, 5484\u20135505. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2023\/file\/117c5c8622b0d539f74f6d1fb082a2e9-Paper-Datasets_and_Benchmarks.pdf"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.491"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2020.102390"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.24723"}],"container-title":["ACM Transactions on the Web"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3777444","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T17:56:27Z","timestamp":1772819787000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3777444"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,19]]},"references-count":59,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,2,28]]}},"alternative-id":["10.1145\/3777444"],"URL":"https:\/\/doi.org\/10.1145\/3777444","relation":{},"ISSN":["1559-1131","1559-114X"],"issn-type":[{"value":"1559-1131","type":"print"},{"value":"1559-114X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,19]]},"assertion":[{"value":"2024-11-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-26","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}