{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T18:13:05Z","timestamp":1791483185229,"version":"4.3.4"},"reference-count":36,"publisher":"MIT Press","issue":"2","license":[{"start":{"date-parts":[[2021,3,16]],"date-time":"2021-03-16T00:00:00Z","timestamp":1615852800000},"content-version":"vor","delay-in-days":74,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,7,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Articles in high-impact journals are, on average, more frequently cited. But are they cited more often because those articles are somehow more \u201ccitable\u201d? Or are they cited more often simply because they are published in a high-impact journal? Although some evidence suggests the latter, the causal relationship is not clear. We here compare citations of preprints to citations of the published version to uncover the causal mechanism. We build on an earlier model of citation dynamics to infer the causal effect of journals on citations. We find that high-impact journals select articles that tend to attract more citations. At the same time, we find that high-impact journals augment the citation rate of published articles. Our results yield a deeper understanding of the role of journals in the research system. The use of journal metrics in research evaluation has been increasingly criticized in recent years and article-level citations are sometimes suggested as an alternative. Our results show that removing impact factors from evaluation does not negate the influence of journals. This insight has important implications for changing practices of research evaluation.<\/jats:p>","DOI":"10.1162\/qss_a_00128","type":"journal-article","created":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T13:07:56Z","timestamp":1616072876000},"page":"496-504","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":43,"title":["Inferring the causal effect of journals on citations"],"prefix":"10.1162","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3170-3879","authenticated-orcid":true,"given":"V. A.","family":"Traag","sequence":"first","affiliation":[{"name":"Centre for Science and Technology Studies (CWTS), Leiden University, the Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2021,7,15]]},"reference":[{"key":"2021072016151425800_bib1","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/j.joi.2018.11.003","article-title":"Predicting publication long-term impact through a combination of early citations and journal impact factor","volume":"13","author":"Abramo","year":"2019","journal-title":"Journal of Informetrics"},{"key":"2021072016151425800_bib2","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/s11192-007-1832-7","article-title":"Calibrating the zoom\u2014A test of Zitt\u2019s hypothesis","volume":"75","author":"Adams","year":"2008","journal-title":"Scientometrics"},{"key":"2021072016151425800_bib3","first-page":"100","article-title":"Controlling selection bias in causal inference","volume-title":"International Conference on Artificial Intelligence and Statistics,","author":"Bareinboim","year":"2012"},{"key":"2021072016151425800_bib4","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1108\/00220410810844150","article-title":"What do citation counts measure? A review of studies on citing behavior","volume":"64","author":"Bornmann","year":"2008","journal-title":"Journal of Documentation"},{"key":"2021072016151425800_bib5","doi-asserted-by":"crossref","first-page":"2847","DOI":"10.1001\/jama.287.21.2847","article-title":"Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals","volume":"287","author":"Callaham","year":"2002","journal-title":"JAMA"},{"key":"2021072016151425800_bib6","article-title":"Imperfect impact","author":"Cantrill","year":"2016","journal-title":"Chemical connections blog"},{"key":"2021072016151425800_bib7","article-title":"Impact factors \u2013 A self-fulfilling prophecy?","author":"Davis","year":"2010","journal-title":"The Scholarly Kitchen blog"},{"key":"2021072016151425800_bib8","doi-asserted-by":"crossref","first-page":"e1001675","DOI":"10.1371\/journal.pbio.1001675","article-title":"The assessment of science: The relative merits of post-publication review, the impact factor, and the number of citations","volume":"11","author":"Eyre-Walker","year":"2013","journal-title":"PLOS Biology"},{"key":"2021072016151425800_bib9","doi-asserted-by":"crossref","DOI":"10.1126\/science.aao0185","article-title":"Science of science","volume":"359","author":"Fortunato","year":"2018","journal-title":"Science"},{"key":"2021072016151425800_bib10","doi-asserted-by":"crossref","first-page":"1218","DOI":"10.1002\/asi.24326","article-title":"Scientific journals still matter in the era of academic search engines and preprint archives","volume":"71","author":"Kim","year":"2020","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"2021072016151425800_bib11","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1145\/2910896.2910909","article-title":"Comparing published scientific journal articles to their pre-print versions","volume-title":"Proceedings of the 16th ACM\/IEEE-CS on Joint Conference on Digital Libraries - JCDL \u201916","author":"Klein","year":"2016"},{"key":"2021072016151425800_bib12","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1093\/biomet\/ast066","article-title":"Measurement bias and effect restoration in causal inference","volume":"101","author":"Kuroki","year":"2014","journal-title":"Biometrika"},{"key":"2021072016151425800_bib13","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1002\/asi.21232","article-title":"The impact factor\u2019s Matthew Effect: A natural experiment in bibliometrics","volume":"61","author":"Larivi\u00e8re","year":"2010","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"2021072016151425800_bib14","first-page":"062109","article-title":"A simple proposal for the publication of journal citation distributions","author":"Larivi\u00e8re","year":"2016","journal-title":"bioRxiv"},{"key":"2021072016151425800_bib15","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-3-030-02511-3_1","article-title":"The journal impact factor: A brief history, critique, and discussion of adverse effects","volume-title":"Springer handbook of science and technology indicators","author":"Larivi\u00e8re","year":"2019"},{"key":"2021072016151425800_bib16","doi-asserted-by":"crossref","first-page":"1157","DOI":"10.1002\/asi.23044","article-title":"arXiv E-prints and the journal of record: An analysis of roles and relationships","volume":"65","author":"Larivi\u00e8re","year":"2014","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"2021072016151425800_bib17","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1016\/j.ipm.2010.09.005","article-title":"A combined bibliometric indicator to predict article impact","volume":"47","author":"Levitt","year":"2011","journal-title":"Information Processing and Management"},{"key":"2021072016151425800_bib18","doi-asserted-by":"crossref","first-page":"e47388","DOI":"10.7554\/eLife.47338","article-title":"Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations","volume":"8","author":"McKiernan","year":"2019","journal-title":"eLife"},{"key":"2021072016151425800_bib19","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1080\/13501780601049079","article-title":"Evidence of a Harvard and Chicago Matthew Effect","volume":"13","author":"Medoff","year":"2006","journal-title":"Journal of Economic Methodology"},{"key":"2021072016151425800_bib20","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1093\/reseval\/rvx023","article-title":"Thinking with indicators. Exploring the epistemic impacts of academic performance indicators in the life sciences","volume":"26","author":"M\u00fcller","year":"2017","journal-title":"Research Evaluation"},{"key":"2021072016151425800_bib21","doi-asserted-by":"crossref","first-page":"739","DOI":"10.1002\/asi.23209","article-title":"Factors affecting citation rates of research articles","volume":"66","author":"Onodera","year":"2015","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"2021072016151425800_bib22","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1177\/016555159502100108","article-title":"On the association between journal circulation and impact factor","volume":"21","author":"Peritz","year":"1995","journal-title":"Journal of Information Science"},{"key":"2021072016151425800_bib23","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1016\/j.jclinepi.2009.09.012","article-title":"Citation analysis of identical consensus statements revealed journal-related bias","volume":"63","author":"Perneger","year":"2010","journal-title":"Journal of Clinical Epidemiology"},{"key":"2021072016151425800_bib24","doi-asserted-by":"crossref","first-page":"15316","DOI":"10.1073\/pnas.1323111111","article-title":"Reputation and impact in academic careers","volume":"111","author":"Petersen","year":"2014","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2021072016151425800_bib25","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1002\/asi.4630270505","article-title":"A general theory of bibliometric and other cumulative advantage processes","volume":"27","author":"Price","year":"1976","journal-title":"Journal of the American Society for Information Science"},{"key":"2021072016151425800_bib26","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1007\/s11024-015-9274-5","article-title":"Accounting for impact? The Journal Impact Factor and the making of biomedical research in the Netherlands","volume":"53","author":"Rushforth","year":"2015","journal-title":"Minerva"},{"key":"2021072016151425800_bib27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/(SICI)1097-4571(199401)45:1<1::AID-ASI1>3.0.CO;2-Y","article-title":"Causal relationship between article citedness and journal impact","volume":"45","author":"Seglen","year":"1994","journal-title":"Journal of the American Society for Information Science"},{"key":"2021072016151425800_bib28","doi-asserted-by":"crossref","first-page":"160384","DOI":"10.1098\/rsos.160384","article-title":"The natural selection of bad science","volume":"3","author":"Smaldino","year":"2016","journal-title":"Royal Society Open Science"},{"key":"2021072016151425800_bib29","doi-asserted-by":"crossref","first-page":"642","DOI":"10.1016\/j.joi.2015.06.005","article-title":"Predicting the long-term citation impact of recent publications","volume":"9","author":"Stegehuis","year":"2015","journal-title":"Journal of Informetrics"},{"key":"2021072016151425800_bib30","article-title":"Replication data: Inferring the causal effect of journals on citations","author":"Traag","year":"2020","journal-title":"Zenodo"},{"key":"2021072016151425800_bib31","article-title":"Replication source code: Inferring the causal effect of journals on citations","author":"Traag","year":"2020","journal-title":"Zenodo"},{"key":"2021072016151425800_bib32","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1057\/s41599-019-0233-x","article-title":"Systematic analysis of agreement between metrics and peer review in the UK REF","volume":"5","author":"Traag","year":"2019","journal-title":"Palgrave Communications"},{"key":"2021072016151425800_bib33","doi-asserted-by":"crossref","DOI":"10.12688\/f1000research.23418.1","article-title":"Use of the journal impact factor for assessing individual articles need not be wrong","volume":"9","author":"Waltman","year":"2020","journal-title":"F1000Research"},{"key":"2021072016151425800_bib34","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1126\/science.1237825","article-title":"Quantifying long-term scientific impact","volume":"342","author":"Wang","year":"2013","journal-title":"Science"},{"key":"2021072016151425800_bib35","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1038\/d41586-019-01643-3","article-title":"Rethinking impact factors: Better ways to judge a journal","volume":"569","author":"Wouters","year":"2019","journal-title":"Nature"},{"key":"2021072016151425800_bib36","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1007\/s11192-005-0218-y","article-title":"Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalisation","volume":"63","author":"Zitt","year":"2005","journal-title":"Scientometrics"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/qss\/article-pdf\/2\/2\/496\/1930685\/qss_a_00128.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/qss\/article-pdf\/2\/2\/496\/1930685\/qss_a_00128.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,31]],"date-time":"2023-10-31T15:19:35Z","timestamp":1698765575000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/2\/2\/496\/98105\/Inferring-the-causal-effect-of-journals-on"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":36,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,7,15]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00128","relation":{"has-review":[{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v2\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v1\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v1\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v2\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v3\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v3\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v2\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v2\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v1\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00128\/v3\/decision1","asserted-by":"object"}]},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021]]},"published":{"date-parts":[[2021]]}}}