{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T08:50:32Z","timestamp":1787907032698,"version":"build-2784847793"},"reference-count":49,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T00:00:00Z","timestamp":1709769600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T00:00:00Z","timestamp":1709769600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["EPJ Data Sci."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Artificial intelligence (AI)-powered recommender systems play a crucial role in determining the content that users are exposed to on social media platforms. However, the behavioural patterns of these systems are often opaque, complicating the evaluation of their impact on the dissemination and consumption of disinformation and misinformation. To begin addressing this evidence gap, this study presents a measurement approach that uses observed digital traces to infer the status of algorithmic amplification of low-credibility content on Twitter over a 14-day period in January 2023. Using an original dataset of \u2248 2.7 million posts on COVID-19 and climate change published on the platform, this study identifies tweets sharing information from low-credibility domains, and uses a bootstrapping model with two stratifications, a tweet\u2019s engagement level and a user\u2019s followers level, to compare any differences in impressions generated between low-credibility and high-credibility samples. Additional stratification variables of toxicity, political bias, and verified status are also examined. This analysis provides valuable observational evidence on whether the Twitter algorithm favours the visibility of low-credibility content, with results indicating that, on aggregate, tweets containing low-credibility URL domains perform better than tweets that do not across both datasets. However, this effect is largely attributable to a difference in high-engagement, high-followers tweets, which are very impactful in terms of impressions generation, and are more likely receive amplified visibility when containing low-credibility content. Furthermore, high toxicity tweets and those with right-leaning bias see heightened amplification, as do low-credibility tweets from verified accounts. Ultimately, this suggests that Twitter\u2019s recommender system may have facilitated the diffusion of false content by amplifying the visibility of low-credibility content with high-engagement generated by very influential users.<\/jats:p>","DOI":"10.1140\/epjds\/s13688-024-00456-3","type":"journal-article","created":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T12:19:43Z","timestamp":1709813983000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":41,"title":["Evaluating Twitter\u2019s algorithmic amplification of low-credibility content: an observational study"],"prefix":"10.1140","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5130-2258","authenticated-orcid":false,"given":"Giulio","family":"Corsi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,3,7]]},"reference":[{"key":"456_CR1","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/j.chb.2014.02.045","volume":"35","author":"I Pentina","year":"2014","unstructured":"Pentina I, Tarafdar M (2014) From \u201cinformation\u201d to \u201cknowing\u201d: exploring the role of social media in contemporary news consumption. Comput Hum Behav 35:211\u2013223","journal-title":"Comput Hum Behav"},{"key":"456_CR2","unstructured":"Shearer E, Gottfried J (2017) News consumption across social media in 2017. https:\/\/www.pewresearch.org\/journalism\/2017\/09\/07\/news-use-across-social-media-platforms-2017\/. Accessed: 20-Apr-2023"},{"key":"456_CR3","unstructured":"Walker M, Matsa KE (2021) News consumption across social media in 2021. https:\/\/www.pewresearch.org\/journalism\/2021\/09\/20\/news-consumption-across-social-media-in-2021\/. Accessed: 20-Apr-2023"},{"issue":"9","key":"456_CR4","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2023301118","volume":"118","author":"M Cinelli","year":"2021","unstructured":"Cinelli M, De Francisci Morales G, Galeazzi A, Quattrociocchi W, Starnini M (2021) The echo chamber effect on social media. Proc Natl Acad Sci 118(9):2023301118","journal-title":"Proc Natl Acad Sci"},{"key":"456_CR5","doi-asserted-by":"publisher","first-page":"15608","DOI":"10.1109\/ACCESS.2018.2810062","volume":"6","author":"A Anandhan","year":"2018","unstructured":"Anandhan A, Shuib L, Ismail MA, Mujtaba G (2018) Social media recommender systems: review and open research issues. IEEE Access 6:15608\u201315628","journal-title":"IEEE Access"},{"key":"456_CR6","unstructured":"Thorburn L (2022) How platform recommenders work. Understanding Recommenders. Accessed: 20-Apr-2023"},{"issue":"50","key":"456_CR7","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2102141118","volume":"118","author":"FP Santos","year":"2021","unstructured":"Santos FP, Lelkes Y, Levin SA (2021) Link recommendation algorithms and dynamics of polarization in online social networks. Proc Natl Acad Sci 118(50):2102141118","journal-title":"Proc Natl Acad Sci"},{"key":"456_CR8","volume-title":"The filter bubble: how the new personalized web is changing what we read and how we think","author":"E Pariser","year":"2011","unstructured":"Pariser E (2011) The filter bubble: how the new personalized web is changing what we read and how we think. Penguin, Baltimore"},{"key":"456_CR9","volume-title":"Understanding social media recommendation algorithms","author":"A Narayanan","year":"2023","unstructured":"Narayanan A (2023) Understanding social media recommendation algorithms"},{"key":"456_CR10","doi-asserted-by":"publisher","first-page":"714","DOI":"10.1145\/3442188.3445933","volume-title":"Proceedings of the 2021 ACM conference on fairness, accountability, and transparency","author":"S Milli","year":"2021","unstructured":"Milli S, Belli L, Hardt M (2021) From optimizing engagement to measuring value. In: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp 714\u2013722"},{"key":"456_CR11","unstructured":"Islam R, Keya KN, Pan S, Foulds J (2019) Mitigating demographic biases in social media-based recommender systems. KDD (Social Impact Track)"},{"key":"456_CR12","first-page":"855","volume-title":"Proceedings of the 15th ACM conference on recommender systems","author":"S Bhadani","year":"2021","unstructured":"Bhadani S (2021) Biases in recommendation system. In: Proceedings of the 15th ACM conference on recommender systems, pp\u00a0855\u2013859"},{"key":"456_CR13","unstructured":"Burns P (2023) What TikTok\u2019s Secret Heating Button Reveals About Virality Online. https:\/\/medium.com\/feedium\/algorithmic-heating-virality-is-a-choice-and-the-game-is-rigged-150307f1032a"},{"key":"456_CR14","unstructured":"Alatawi F, Cheng L, Tahir A, Karami M, Jiang B, Black T, Liu H (2021) A survey on echo chambers on social media: Description, detection and mitigation. arXiv preprint arXiv:2112.05084"},{"key":"456_CR15","volume":"15","author":"J Kaiser","year":"2021","unstructured":"Kaiser J, Rauchfleisch A, C\u00f3rdova Y (2021) Comparative approaches to mis\/disinformation| fighting Zika with honey: an analysis of youtube\u2019s video recommendations on Brazilian youtube. Int J Commun 15:19","journal-title":"Int J Commun"},{"key":"456_CR16","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4842-7155-1","volume-title":"How algorithms create and prevent fake news: exploring the impacts of social media, deepfakes, GPT-3, and more","author":"N Giansiracusa","year":"2021","unstructured":"Giansiracusa N (2021) How algorithms create and prevent fake news: exploring the impacts of social media, deepfakes, GPT-3, and more. Springer, Berlin"},{"key":"456_CR17","doi-asserted-by":"crossref","unstructured":"Leerssen P (2020) The soap box as a black box: regulating transparency in social media recommender systems. Eur J Law Technol 11(2)","DOI":"10.31228\/osf.io\/uhxcv"},{"key":"456_CR18","unstructured":"Liu Z, Zou L, Zou X, Wang C, Zhang B, Tang D, Zhu B, Zhu Y, Wu P, Wang K (2022) Monolith: real time recommendation system with collisionless embedding table. arXiv preprint arXiv:2209.07663"},{"key":"456_CR19","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1145\/3298689.3346997","volume-title":"Proceedings of the 13th ACM conference on recommender systems","author":"Z Zhao","year":"2019","unstructured":"Zhao Z, Hong L, Wei L, Chen J, Nath A, Andrews S, Kumthekar A, Sathiamoorthy M, Yi X, Chi E (2019) Recommending what video to watch next: a multitask ranking system. In: Proceedings of the 13th ACM conference on recommender systems, pp\u00a043\u201351"},{"key":"456_CR20","unstructured":"Lada A, Wang M, Yan T (2021) How does news feed predict what you want to see. Personalized ranking with machine learning. Retrieved 28:2021"},{"key":"456_CR21","volume-title":"The Twitter algorithm: TweepCred","author":"Twitter","year":"2023","unstructured":"Twitter (2023) The Twitter algorithm: TweepCred. https:\/\/github.com\/twitter\/the-algorithm\/blob\/main\/src\/scala\/com\/twitter\/graph\/batch\/job\/tweepcred\/README"},{"key":"456_CR22","unstructured":"Wang Z, She Q, Zhang J (2021) Masknet: introducing feature-wise multiplication to ctr ranking models by instance-guided mask. arXiv preprint arXiv:2102.07619"},{"key":"456_CR23","first-page":"1073","volume-title":"Proceedings of the international AAAI conference on web and social media","author":"J Pfeffer","year":"2023","unstructured":"Pfeffer J, Matter D, Jaidka K, Varol O, Mashhadi A, Lasser J, Assenmacher D, Wu S, Yang D, Brantner C (2023) Just another day on Twitter: a complete 24 hours of Twitter data. In: Proceedings of the international AAAI conference on web and social media, vol\u00a017, pp\u00a01073\u20131081"},{"issue":"1","key":"456_CR24","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-018-07761-2","volume":"10","author":"A Bovet","year":"2019","unstructured":"Bovet A, Makse HA (2019) Influence of fake news in Twitter during the 2016 us presidential election. Nat Commun 10(1):7","journal-title":"Nat Commun"},{"key":"456_CR25","doi-asserted-by":"crossref","unstructured":"Kouzy R, Abi Jaoude J, Kraitem A, El Alam MB, Karam B, Adib E, Zarka J, Traboulsi C, Akl EW, Baddour K (2020) Coronavirus goes viral: quantifying the covid-19 misinformation epidemic on twitter. Cureus 12(3)","DOI":"10.7759\/cureus.7255"},{"key":"456_CR26","doi-asserted-by":"crossref","unstructured":"Lin H, Lasser J, Lewandowsky S, Cole R, Gully A, Rand DG, Pennycook G (2023) High level of correspondence across different news domain quality rating sets. PNAS Nexus 286","DOI":"10.31234\/osf.io\/qy94s"},{"key":"456_CR27","unstructured":"Twitter (2017) Using Deep Learning at Scale in Twitter\u2019s Timelines. https:\/\/blog.twitter.com\/engineering\/en_us\/topics\/insights\/2017\/using-deep-learning-at-scale-in-twitters-timelines"},{"issue":"5","key":"456_CR28","volume":"11","author":"KMd Treen","year":"2020","unstructured":"Treen KMd, Williams HT, O\u2019Neill SJ (2020) Online misinformation about climate change. Wiley Interdiscip Rev: Clim Change 11(5):665","journal-title":"Wiley Interdiscip Rev: Clim Change"},{"key":"456_CR29","unstructured":"Brennen JS, Simon FM, Howard PN, Nielsen RK (2020) Types, sources, and claims of covid-19 misinformation. Thesis"},{"key":"456_CR30","unstructured":"Graham T, Bruns A, Zhu G, Campbell R (2020) Like a virus: the coordinated spread of coronavirus disinformation"},{"issue":"62","key":"456_CR31","doi-asserted-by":"publisher","DOI":"10.21105\/joss.03272","volume":"6","author":"C Barrie","year":"2021","unstructured":"Barrie C, Ho JC-T (2021) academictwitter: an r package to access the Twitter academic research product track v2 api endpoint. J Open Sour Softw 6(62):3272","journal-title":"J Open Sour Softw"},{"key":"456_CR32","doi-asserted-by":"crossref","unstructured":"Pierri F, DeVerna MR, Yang K-C, Axelrod D, Bryden J, Menczer F (2022) One year of covid-19 vaccine misinformation on twitter. arXiv preprint arXiv:2209.01675","DOI":"10.2196\/preprints.42227"},{"key":"456_CR33","doi-asserted-by":"crossref","unstructured":"Falkenberg M, Galeazzi A, Torricelli M, Di Marco N, Larosa F, Sas M, Mekacher A, Pearce W, Zollo F, Quattrociocchi W (2022) Growing polarization around climate change on social media. Nat Clim Change, 1\u20138","DOI":"10.1038\/s41558-022-01527-x"},{"key":"456_CR34","first-page":"1","volume":"17","author":"P Resnick","year":"2018","unstructured":"Resnick P, Ovadya A, Gilchrist G (2018) Iffy quotient: a platform health metric for misinformation. Cent Soc Media Responsib 17:1\u201320","journal-title":"Cent Soc Media Responsib"},{"key":"456_CR35","unstructured":"Newsguard (2023) Score and Rating Levels. https:\/\/www.newsguardtech.com\/ratings\/rating-process-criteria\/"},{"key":"456_CR36","unstructured":"Thorburn L, Stray J, Bengani P (2023) Making Amplification Measurable. https:\/\/medium.com\/understanding-recommenders\/making-amplification-measurable-2be548e5986c"},{"key":"456_CR37","doi-asserted-by":"publisher","DOI":"10.1201\/9780429246593","volume-title":"An introduction to the bootstrap","author":"B Efron","year":"1994","unstructured":"Efron B, Tibshirani RJ (1994) An introduction to the bootstrap. CRC Press, Boca Raton"},{"key":"456_CR38","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2019.02215","volume":"10","author":"K Jung","year":"2019","unstructured":"Jung K, Lee J, Gupta V, Cho G (2019) Comparison of bootstrap confidence interval methods for gsca using a Monte Carlo simulation. Front Psychol 10:2215","journal-title":"Front Psychol"},{"key":"456_CR39","first-page":"194","volume-title":"Supervised and unsupervised discretization of continuous features","author":"J Dougherty","year":"1995","unstructured":"Dougherty J, Kohavi R, Sahami M (1995) Supervised and unsupervised discretization of continuous features. Elsevier, Amsterdam, pp 194\u2013202"},{"key":"456_CR40","doi-asserted-by":"publisher","first-page":"3197","DOI":"10.1145\/3534678.3539147","volume-title":"Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining","author":"A Lees","year":"2022","unstructured":"Lees A, Tran VQ, Tay Y, Sorensen J, Gupta J, Metzler D, Vasserman L (2022) A new generation of perspective API: efficient multilingual character-level transformers. In: Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pp\u00a03197\u20133207"},{"key":"456_CR41","doi-asserted-by":"publisher","first-page":"1086","DOI":"10.1145\/3442381.3449861","volume-title":"Proceedings of the web conference 2021","author":"M Saveski","year":"2021","unstructured":"Saveski M, Roy B, Roy D (2021) The structure of toxic conversations on Twitter. In: Proceedings of the web conference 2021, pp\u00a01086\u20131097"},{"key":"456_CR42","unstructured":"Cuthbertson L, Kearney A, Dawson R, Zawaduk A, Cuthbertson E, Gordon-Tighe A, Mathewson KW (2019) Women, politics and Twitter: using machine learning to change the discourse. arXiv preprint arXiv:1911.11025"},{"key":"456_CR43","unstructured":"T\u00f6rnberg P (2023) Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. arXiv preprint arXiv:2304.06588"},{"key":"456_CR44","doi-asserted-by":"crossref","unstructured":"Gilardi F, Alizadeh M, Kubli M (2023) Chatgpt outperforms crowd-workers for text-annotation tasks. arXiv preprint arXiv:2303.15056","DOI":"10.1073\/pnas.2305016120"},{"issue":"3","key":"456_CR45","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1111\/j.2517-6161.1988.tb01732.x","volume":"50","author":"TJ Diciccio","year":"1988","unstructured":"Diciccio TJ, Romano JP (1988) A review of bootstrap confidence intervals. J R Stat Soc, Ser B, Stat Methodol 50(3):338\u2013354","journal-title":"J R Stat Soc, Ser B, Stat Methodol"},{"key":"456_CR46","unstructured":"Haroon M, Chhabra A, Liu X, Mohapatra P, Shafiq Z, Wojcieszak M (2022) Youtube, the great radicalizer? auditing and mitigating ideological biases in youtube recommendations. arXiv preprint arXiv:2203.10666"},{"key":"456_CR47","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1145\/3351095.3372879","volume-title":"Proceedings of the 2020 conference on fairness, accountability, and transparency","author":"MH Ribeiro","year":"2020","unstructured":"Ribeiro MH, Ottoni R, West R, Almeida VA, Meira W Jr (2020) Auditing radicalization pathways on youtube. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 131\u2013141"},{"issue":"28","key":"456_CR48","doi-asserted-by":"publisher","first-page":"7313","DOI":"10.1073\/pnas.1618923114","volume":"114","author":"WJ Brady","year":"2017","unstructured":"Brady WJ, Wills JA, Jost JT, Tucker JA, Van Bavel JJ (2017) Emotion shapes the diffusion of moralized content in social networks. Proc Natl Acad Sci 114(28):7313\u20137318","journal-title":"Proc Natl Acad Sci"},{"key":"456_CR49","unstructured":"Ovadya A, Thorburn L (2023) Bridging systems: open problems for countering destructive divisiveness across ranking, recommenders, and governance. arXiv preprint arXiv:2301.09976"}],"container-title":["EPJ Data Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1140\/epjds\/s13688-024-00456-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1140\/epjds\/s13688-024-00456-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1140\/epjds\/s13688-024-00456-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,13]],"date-time":"2024-11-13T18:47:46Z","timestamp":1731523666000},"score":1,"resource":{"primary":{"URL":"https:\/\/epjdatascience.springeropen.com\/articles\/10.1140\/epjds\/s13688-024-00456-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,7]]},"references-count":49,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["456"],"URL":"https:\/\/doi.org\/10.1140\/epjds\/s13688-024-00456-3","relation":{},"ISSN":["2193-1127"],"issn-type":[{"value":"2193-1127","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,7]]},"assertion":[{"value":"20 September 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The author declares no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"18"}}