{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T20:33:53Z","timestamp":1784666033580,"version":"3.55.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T00:00:00Z","timestamp":1673222400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T00:00:00Z","timestamp":1673222400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2023,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In spite of previous research demonstrating the risks involved, and counsel against the practice as early as 1997, some research evaluations continue to use journal impact alone as a surrogate of the number of citations of hosted articles to assess the latter\u2019s impact. Such usage is also taken up by research administrators and policy-makers, with very serious implications. The aim of this work is to investigate the correlation between the citedness of a publication and the impact of the host journal. We extend the analyses of previous literature to all STEM fields. Then we also aim to assess whether this correlation varies across fields and is stronger for highly cited authors than for lowly cited ones. Our dataset consists of a total of almost one million authorships of 2010\u20132019 publications authored by about 28,000 professors in 230 research fields. Results show a low correlation between the two indicators, more so for lowly cited authors as compared to highly cited ones, although differences occur across fields.\n<\/jats:p>","DOI":"10.1007\/s11192-022-04622-0","type":"journal-article","created":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T05:05:56Z","timestamp":1673240756000},"page":"1877-1894","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Correlating article citedness and journal impact: an empirical investigation by field on a large-scale dataset"],"prefix":"10.1007","volume":"128","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0731-3635","authenticated-orcid":false,"given":"Giovanni","family":"Abramo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6977-6611","authenticated-orcid":false,"given":"Ciriaco Andrea","family":"D\u2019Angelo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Flavia","family":"Di Costa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,1,9]]},"reference":[{"issue":"3","key":"4622_CR1","doi-asserted-by":"publisher","first-page":"821","DOI":"10.1007\/s11192-010-0200-1","volume":"84","author":"G Abramo","year":"2010","unstructured":"Abramo, G., D\u2019Angelo, C. A., & Di Costa, F. (2010). Citations versus journal impact factor as proxy of quality: Could the latter ever be preferable? Scientometrics, 84(3), 821\u2013833. https:\/\/doi.org\/10.1007\/s11192-010-0200-1","journal-title":"Scientometrics"},{"issue":"1","key":"4622_CR2","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1016\/j.joi.2018.11.003","volume":"13","author":"G Abramo","year":"2019","unstructured":"Abramo, G., D\u2019Angelo, C. A., & Felici, G. (2019). Predicting long-term publication impact through a combination of early citations and journal impact factor. Journal of Informetrics, 13(1), 32\u201349. https:\/\/doi.org\/10.1016\/j.joi.2018.11.003","journal-title":"Journal of Informetrics"},{"issue":"2","key":"4622_CR3","first-page":"639","volume":"1","author":"M Antonoyiannakis","year":"2020","unstructured":"Antonoyiannakis, M. (2020). Impact factor volatility due to a single paper: A comprehensive analysis. Quantitative Science Studies, 1(2), 639\u2013663.","journal-title":"Quantitative Science Studies"},{"issue":"3","key":"4622_CR4","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1007\/s11192-007-2036-x","volume":"79","author":"\u00c9 Archambault","year":"2009","unstructured":"Archambault, \u00c9., & Larivi\u00e8re, V. (2009). History of the journal impact factor: Contingencies and consequences. Scientometrics, 79(3), 639\u2013653.","journal-title":"Scientometrics"},{"issue":"1","key":"4622_CR5","doi-asserted-by":"publisher","first-page":"17","DOI":"10.29024\/sar.28","volume":"2","author":"A Bonaccorsi","year":"2020","unstructured":"Bonaccorsi, A. (2020). Two decades of research assessment in Italy. Addressing the criticisms. Scholarly Assessment Reports, 2(1), 17. https:\/\/doi.org\/10.29024\/sar.28","journal-title":"Addressing the criticisms. Scholarly Assessment Reports"},{"issue":"1","key":"4622_CR6","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1016\/j.joi.2016.12.001","volume":"11","author":"L Bornmann","year":"2017","unstructured":"Bornmann, L., & Leydesdorff, L. (2017). Skewness of citation impact data and covariates of citation distributions: A large-scale empirical analysis based on Web of Science data. Journal of Informetrics, 11(1), 164\u2013175.","journal-title":"Journal of Informetrics"},{"issue":"1","key":"4622_CR7","doi-asserted-by":"publisher","first-page":"314","DOI":"10.1016\/j.joi.2019.01.009","volume":"13","author":"R Brito","year":"2019","unstructured":"Brito, R., & Rodr\u00edguez-Navarro, A. (2019). Evaluating research and researchers by the journal impact factor: Is it better than coin flipping? Journal of Informetrics, 13(1), 314\u2013324.","journal-title":"Journal of Informetrics"},{"issue":"7611","key":"4622_CR8","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1038\/nature.2016.20224","volume":"535","author":"E Callaway","year":"2016","unstructured":"Callaway, E. (2016). Publishing elite turns against impact factor. Nature, 535(7611), 210\u2013211.","journal-title":"Nature"},{"issue":"10","key":"4622_CR9","doi-asserted-by":"publisher","first-page":"4093","DOI":"10.1172\/JCI71859","volume":"123","author":"A Ciechanover","year":"2013","unstructured":"Ciechanover, A. (2013). A conversation with Aaron Ciechanover. Interview by Ushma S. Neill. The Journal of Clinical Investigation, 123(10), 4093\u20134094. https:\/\/doi.org\/10.1172\/JCI71859","journal-title":"The Journal of Clinical Investigation"},{"key":"4622_CR10","volume-title":"Relazione sulla ricerca e l\u2019innovazione in Italia","author":"Consiglio Nazionale delle Ricerche","year":"2019","unstructured":"Consiglio Nazionale delle Ricerche. (2019). Relazione sulla ricerca e l\u2019innovazione in Italia. Cnr edizioni."},{"issue":"2","key":"4622_CR11","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1002\/asi.21460","volume":"62","author":"CA D\u2019Angelo","year":"2011","unstructured":"D\u2019Angelo, C. A., Giuffrida, C., & Abramo, G. (2011). A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments. Journal of the American Society for Information Science and Technology, 62(2), 257\u2013269.","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"11","key":"4622_CR12","doi-asserted-by":"publisher","first-page":"536","DOI":"10.2519\/jospt.2021.10598","volume":"51","author":"D de Oliveira Silva","year":"2021","unstructured":"de Oliveira Silva, D., Taborda, B., Pazzinatto, M. F., Ardern, C. L., & Barton, C. J. (2021). The altmetric score has a stronger relationship with article citations than journal impact factor and open access status: A cross-sectional analysis of 4022 sport sciences articles. Journal of Orthopaedic and Sports Physical Therapy, 51(11), 536\u2013541. https:\/\/doi.org\/10.2519\/jospt.2021.10598","journal-title":"Journal of Orthopaedic and Sports Physical Therapy"},{"issue":"2","key":"4622_CR13","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1093\/reseval\/rvv038","volume":"25","author":"S De Rijcke","year":"2016","unstructured":"De Rijcke, S., Wouters, P., Rushforth, A., Franssen, T., & Hammarfelt, B. (2016). Evaluation practices and effects of indicator use\u2014A literature review. Research Evaluation, 25(2), 161\u2013169.","journal-title":"Research Evaluation"},{"key":"4622_CR14","unstructured":"DORA. (2012). San Francisco declaration of research assessment. Retrieved November 30, 2022, from https:\/\/sfdora.org\/read\/"},{"key":"4622_CR15","doi-asserted-by":"publisher","DOI":"10.1108\/S1876-0562(2005)05","volume-title":"Power laws in the information production process: Lotkaian informetrics","author":"L Egghe","year":"2005","unstructured":"Egghe, L. (2005). Power laws in the information production process: Lotkaian informetrics. Elsevier."},{"issue":"6837","key":"4622_CR16","doi-asserted-by":"publisher","first-page":"522","DOI":"10.1038\/35079156","volume":"411","author":"E Garfield","year":"2001","unstructured":"Garfield, E. (2001). Impact factors, and why they won\u2019t go away. Nature, 411(6837), 522. https:\/\/doi.org\/10.1038\/35079156","journal-title":"Nature"},{"key":"4622_CR150","doi-asserted-by":"crossref","unstructured":"Gl\u00e4nzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171\u2013193.","DOI":"10.1023\/A:1014848323806"},{"issue":"7548","key":"4622_CR17","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1038\/520429a","volume":"520","author":"D Hicks","year":"2015","unstructured":"Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429\u2013431.","journal-title":"Nature"},{"issue":"4","key":"4622_CR18","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1080\/13600800903191930","volume":"31","author":"SD Jarwal","year":"2009","unstructured":"Jarwal, S. D., Brion, A. M., & King, M. L. (2009). Measuring research quality using the journal impact factor, citations and \u2018Ranked Journals\u2019: Blunt instruments or inspired metrics? Journal of Higher Education Policy and Management, 31(4), 289\u2013300.","journal-title":"Journal of Higher Education Policy and Management"},{"issue":"2","key":"4622_CR19","doi-asserted-by":"publisher","first-page":"1249","DOI":"10.1007\/s11192-020-03812-y","volume":"126","author":"T Kiesslich","year":"2021","unstructured":"Kiesslich, T., Beyreis, M., Zimmermann, G., & Traweger, A. (2021). Citation inequality and the journal impact factor: Median, mean, (does it) matter? Scientometrics, 126(2), 1249\u20131269.","journal-title":"Scientometrics"},{"issue":"10","key":"4622_CR20","doi-asserted-by":"publisher","first-page":"1218","DOI":"10.1002\/asi.24326","volume":"71","author":"L Kim","year":"2020","unstructured":"Kim, L., Portenoy, J. H., West, J. D., & Stovel, K. W. (2020). Scientific journals still matter in the era of academic search engines and preprint archives. Journal of the Association for Information Science and Technology, 71(10), 1218\u20131226.","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"4622_CR21","doi-asserted-by":"publisher","DOI":"10.1101\/062109","author":"V Larivi\u00e8re","year":"2016","unstructured":"Larivi\u00e8re, V., Kiermer, V., MacCallum, C. J., McNutt, M., Patterson, M., Pulverer, B., Swaminathan, S., Taylor, S., & Curry, S. (2016). A simple proposal for the publication of journal citation distributions. BioRxiv. https:\/\/doi.org\/10.1101\/062109","journal-title":"BioRxiv"},{"key":"4622_CR22","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/978-3-030-02511-3_1","volume-title":"Springer handbook of science and technology indicators","author":"V Lariviere","year":"2019","unstructured":"Lariviere, V., & Sugimoto, C. R. (2019). The journal impact factor: A brief history, critique, and discussion of adverse effects. In W. Gl\u00e4nzel, H. F. Moed, U. Schmoch, & M. Thelwall (Eds.), Springer handbook of science and technology indicators (pp. 3\u201324). Springer."},{"issue":"2","key":"4622_CR23","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1002\/asi.20743","volume":"59","author":"L Leydesdorff","year":"2008","unstructured":"Leydesdorff, L. (2008). Caveats for the use of citation indicators in research and journal evaluation. Journal of the American Association for Information Science and Technology, 59(2), 278\u2013287.","journal-title":"Journal of the American Association for Information Science and Technology"},{"issue":"11","key":"4622_CR24","doi-asserted-by":"publisher","first-page":"2140","DOI":"10.1002\/asi.22731","volume":"63","author":"GA Lozano","year":"2012","unstructured":"Lozano, G. A., Larivi\u00e8re, V., & Gingras, Y. (2012). The weakening relationship between the impact factor and papers\u2019 citations in the digital age. Journal of the American Society for Information Science and Technology, 63(11), 2140\u20132145.","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"3","key":"4622_CR25","first-page":"62","volume":"39","author":"W Marx","year":"2013","unstructured":"Marx, W., & Bornmann, L. (2013). Journal impact factor: \u201cThe poor mans\u2019 citation analysis\u201d and alternative approaches. European Science Editing, 39(3), 62\u201363.","journal-title":"European Science Editing"},{"issue":"1","key":"4622_CR26","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1016\/j.joi.2016.12.006","volume":"11","author":"S Milojevi\u0107","year":"2017","unstructured":"Milojevi\u0107, S., Radicchi, F., & Bar-Ilan, J. (2017). Citation success index\u2014An intuitive pair-wise journal comparison metric. Journal of Informetrics, 11(1), 223\u2013231.","journal-title":"Journal of Informetrics"},{"issue":"1","key":"4622_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ejor.2015.04.002","volume":"246","author":"J Mingers","year":"2015","unstructured":"Mingers, J., & Leydesdorff, L. (2015). A review of theory and practice in scientometrics. European Journal of Operational Research, 246(1), 1\u201319.","journal-title":"European Journal of Operational Research"},{"issue":"1","key":"4622_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.29024\/sar.8","volume":"2","author":"HF Moed","year":"2020","unstructured":"Moed, H. F. (2020). Appropriate use of metrics in research assessment of autonomous academic institutions. Scholarly Assessment Reports, 2(1), 1.","journal-title":"Scholarly Assessment Reports"},{"issue":"6579","key":"4622_CR29","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1038\/381186a0","volume":"381","author":"HF Moed","year":"1996","unstructured":"Moed, H. F., & van Leeuwen, T. N. (1996). Impact factors can mislead. Nature, 381(6579), 186\u2013186.","journal-title":"Nature"},{"key":"4622_CR30","doi-asserted-by":"publisher","first-page":"1487","DOI":"10.3389\/fpsyg.2018.01487","volume":"9","author":"FM Paulus","year":"2018","unstructured":"Paulus, F. M., Cruz, N., & Krach, S. (2018). The impact factor fallacy. Frontiers in Psychology, 9, 1487.","journal-title":"Frontiers in Psychology"},{"issue":"2","key":"4622_CR31","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1007\/s100510050359","volume":"4","author":"S Radner","year":"1998","unstructured":"Radner, S. (1998). How popular is your paper? An empirical study of the citation distribution. The European Physical Journal B Condensed Matter and Complex Systems, 4(2), 131\u2013134.","journal-title":"The European Physical Journal B Condensed Matter and Complex Systems"},{"issue":"2","key":"4622_CR32","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1007\/s11024-015-9274-5","volume":"53","author":"A Rushforth","year":"2015","unstructured":"Rushforth, A., & De Rijcke, S. (2015). Accounting for impact? The journal impact factor and the making of biomedical research in the Netherlands. Minerva, 53(2), 117\u2013139.","journal-title":"Minerva"},{"issue":"8","key":"4622_CR33","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1016\/0968-0004(89)90163-1","volume":"14","author":"PO Seglen","year":"1989","unstructured":"Seglen, P. O. (1989). From bad to worse: Evaluation by journal impact. Trends in Biochemical Sciences, 14(8), 326\u2013327. https:\/\/doi.org\/10.1016\/0968-0004(89)90163-1","journal-title":"Trends in Biochemical Sciences"},{"issue":"9","key":"4622_CR34","doi-asserted-by":"publisher","first-page":"628","DOI":"10.1002\/(SICI)1097-4571(199210)43:9<628::AID-ASI5>3.0.CO;2-0","volume":"43","author":"PO Seglen","year":"1992","unstructured":"Seglen, P. O. (1992). The skewness of science. Journal of the American Society for Information Science, 43(9), 628\u2013638.","journal-title":"Journal of the American Society for Information Science"},{"issue":"1","key":"4622_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/(SICI)1097-4571(199401)45:1<1::AID-ASI1>3.0.CO;2-Y","volume":"45","author":"PO Seglen","year":"1994","unstructured":"Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45(1), 1\u201311.","journal-title":"Journal of the American Society for Information Science"},{"issue":"7079","key":"4622_CR36","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1136\/bmj.314.7079.497","volume":"314","author":"PO Seglen","year":"1997","unstructured":"Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. British Medical Journal, 314(7079), 498\u2013502.","journal-title":"British Medical Journal"},{"issue":"2","key":"4622_CR37","doi-asserted-by":"publisher","first-page":"496","DOI":"10.1162\/qss_a_00128","volume":"2","author":"VA Traag","year":"2021","unstructured":"Traag, V. A. (2021). Inferring the causal effect of journals on citations. Quantitative Science Studies, 2(2), 496\u2013504.","journal-title":"Quantitative Science Studies"},{"issue":"2","key":"4622_CR38","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1007\/s11192-005-0217-z","volume":"63","author":"TN Van Leeuwen","year":"2005","unstructured":"Van Leeuwen, T. N., & Moed, H. F. (2005). Characteristics of journal impact factors. The effects of uncitedness and citation distribution on the understanding of journal impact factors. Scientometrics, 63(2), 357\u2013371.","journal-title":"Scientometrics"},{"issue":"2","key":"4622_CR39","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1016\/j.joi.2016.02.007","volume":"10","author":"L Waltman","year":"2016","unstructured":"Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365\u2013391. https:\/\/doi.org\/10.1016\/j.joi.2016.02.007","journal-title":"Journal of Informetrics"},{"key":"4622_CR40","doi-asserted-by":"publisher","first-page":"366","DOI":"10.12688\/f1000research.23418.2","volume":"9","author":"L Waltman","year":"2021","unstructured":"Waltman, L., & Traag, V. A. (2021). Use of the journal impact factor for assessing individual articles: Statistically flawed or not? F1000Research, 9, 366.","journal-title":"F1000Research"},{"key":"4622_CR41","volume-title":"The metric tide: Literature review (Supplementary Report I to the independent review of the role of metrics in research assessment and management)","author":"P Wouters","year":"2015","unstructured":"Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., & Franssen, T. (2015). The metric tide: Literature review (Supplementary Report I to the independent review of the role of metrics in research assessment and management). HEFCE."},{"issue":"3","key":"4622_CR42","doi-asserted-by":"publisher","first-page":"e0174205","DOI":"10.1371\/journal.pone.0174205","volume":"12","author":"L Zhang","year":"2017","unstructured":"Zhang, L., Rousseau, R., & Sivertsen, G. (2017). Science deserves to be judged by its contents, not by its wrapping: Revisiting Seglen\u2019s work on journal impact and research evaluation. PLoS ONE, 12(3), e0174205.","journal-title":"PLoS ONE"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04622-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-022-04622-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04622-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,4]],"date-time":"2023-03-04T08:17:33Z","timestamp":1677917853000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-022-04622-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,9]]},"references-count":43,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,3]]}},"alternative-id":["4622"],"URL":"https:\/\/doi.org\/10.1007\/s11192-022-04622-0","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,9]]},"assertion":[{"value":"13 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 December 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 January 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in the manuscript. Giovanni Abramo and Ciriaco Andrea D\u2019Angelo are members of the Editorial Board of the journal.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}