{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T19:52:42Z","timestamp":1776109962034,"version":"3.50.1"},"reference-count":60,"publisher":"MIT Press","issue":"4","license":[{"start":{"date-parts":[[2024,10,4]],"date-time":"2024-10-04T00:00:00Z","timestamp":1728000000000},"content-version":"vor","delay-in-days":277,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Hellman Fellow"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>A recent analysis of scientific publication and patent citation networks by Park et al. (Nature, 2023) suggests that publications and patents are becoming less disruptive over time. Here we show that the reported decrease in disruptiveness is an artifact of systematic shifts in the structure of citation networks unrelated to innovation system capacity. Instead, the decline is attributable to \u201ccitation inflation,\u201d an unavoidable characteristic of real citation networks that manifests as a systematic time-dependent bias and renders cross-temporal analysis challenging. One driver of citation inflation is the ever-increasing lengths of reference lists over time, which in turn increases the density of links in citation networks, and causes the disruption index to converge to 0. The impact of this systematic bias further stymies efforts to correlate disruption to other measures that are also time dependent, such as team size and citation counts. In order to demonstrate this fundamental measurement problem, we present three complementary lines of critique (deductive, empirical and computational modeling), and also make available an ensemble of synthetic citation networks that can be used to test alternative citation-based indices for systematic bias.<\/jats:p>","DOI":"10.1162\/qss_a_00333","type":"journal-article","created":{"date-parts":[[2024,10,4]],"date-time":"2024-10-04T16:18:09Z","timestamp":1728058689000},"page":"936-953","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":17,"title":["The disruption index is biased by citation inflation"],"prefix":"10.1162","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0955-3483","authenticated-orcid":true,"given":"Alexander Michael","family":"Petersen","sequence":"first","affiliation":[{"name":"Department of Management of Complex Systems, Ernest and Julio Gallo Management Program, School of Engineering, University of California, Merced, Merced, CA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1181-8222","authenticated-orcid":true,"given":"Felber","family":"Arroyave","sequence":"additional","affiliation":[{"name":"Department of Management of Complex Systems, Ernest and Julio Gallo Management Program, School of Engineering, University of California, Merced, Merced, CA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7056-1303","authenticated-orcid":true,"given":"Fabio","family":"Pammolli","sequence":"additional","affiliation":[{"name":"Department of Management, Economics and Industrial Engineering, Politecnico di Milano, Milan, Italy"}]}],"member":"281","published-online":{"date-parts":[[2024,11,1]]},"reference":[{"issue":"4","key":"2024122514552196400_bib1","doi-asserted-by":"publisher","first-page":"101204","DOI":"10.1016\/j.joi.2021.101204","article-title":"The effects of citation-based research evaluation schemes on self-citation behavior","volume":"15","author":"Abramo","year":"2021","journal-title":"Journal of Informetrics"},{"issue":"13","key":"2024122514552196400_bib2","doi-asserted-by":"publisher","first-page":"1106","DOI":"10.1002\/asi.10151","article-title":"Is the relationship between numbers of references and paper lengths the same for all sciences?","volume":"53","author":"Abt","year":"2002","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"1","key":"2024122514552196400_bib3","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1002\/asi.20936","article-title":"Differences in impact factor across fields and over time","volume":"60","author":"Althouse","year":"2009","journal-title":"JASIST"},{"key":"2024122514552196400_bib4","volume-title":"Network science","author":"Barab\u00e0si","year":"2016"},{"issue":"3\u20134","key":"2024122514552196400_bib5","doi-asserted-by":"publisher","first-page":"590","DOI":"10.1016\/S0378-4371(02)00736-7","article-title":"Evolution of the social network of scientific collaborations","volume":"311","author":"Barab\u00e0si","year":"2002","journal-title":"Physica A"},{"issue":"2","key":"2024122514552196400_bib6","doi-asserted-by":"publisher","first-page":"23500066","DOI":"10.1142\/S0219525923500066","article-title":"Is disruption decreasing, or is it accelerating?","volume":"26","author":"Bentley","year":"2023","journal-title":"Advances in Complex Systems"},{"issue":"1","key":"2024122514552196400_bib7","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1108\/00220410810844150","article-title":"What do citation counts measure? A review of studies on citing behavior","volume":"64","author":"Bornmann","year":"2008","journal-title":"Journal of Documentation"},{"issue":"3","key":"2024122514552196400_bib8","doi-asserted-by":"publisher","first-page":"1242","DOI":"10.1162\/qss_a_00068","article-title":"Are disruption index indicators convergently valid? The comparison of several indicator variants with assessments by peers","volume":"1","author":"Bornmann","year":"2020","journal-title":"Quantitative Science Studies"},{"issue":"2","key":"2024122514552196400_bib9","doi-asserted-by":"publisher","first-page":"e280207","DOI":"10.3145\/epi.2019.mar.07","article-title":"Disruption index depends on length of citation window","volume":"28","author":"Bornmann","year":"2019","journal-title":"El Profesional de la Informaci\u00f3n (EPI)"},{"issue":"8","key":"2024122514552196400_bib10","doi-asserted-by":"publisher","first-page":"e0255849","DOI":"10.1371\/journal.pone.0255849","article-title":"Literary runaway: Increasingly more references cited per academic research article from 1980 to 2019","volume":"16","author":"Dai","year":"2021","journal-title":"PLOS ONE"},{"key":"2024122514552196400_bib11","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1007\/s11192-007-1777-2","article-title":"Does self-citation pay?","volume":"72","author":"Fowler","year":"2007","journal-title":"Scientometrics"},{"issue":"3","key":"2024122514552196400_bib12","doi-asserted-by":"publisher","first-page":"791","DOI":"10.1287\/mnsc.2015.2366","article-title":"A dynamic network measure of technological change","volume":"63","author":"Funk","year":"2017","journal-title":"Management Science"},{"key":"2024122514552196400_bib13","doi-asserted-by":"publisher","first-page":"511","DOI":"10.1023\/B:SCIE.0000034391.06240.2a","article-title":"Towards a model for diachronous and synchronous citation analyses","volume":"60","author":"Gl\u00e4nzel","year":"2004","journal-title":"Scientometrics"},{"key":"2024122514552196400_bib14","article-title":"Manipulated attention: Do digital platforms promote bias in science?","author":"Helbing","year":"2023","journal-title":"ResearchGate"},{"key":"2024122514552196400_bib15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2402.14583","article-title":"Dataset artefacts are the hidden drivers of the declining disruptiveness in science","author":"Holst","year":"2024","journal-title":"arXiv"},{"issue":"2","key":"2024122514552196400_bib16","doi-asserted-by":"publisher","first-page":"101015","DOI":"10.1016\/j.joi.2020.101015","article-title":"Patent citation inflation: The phenomenon, its measurement, and relative indicators to temper its effects","volume":"14","author":"Huang","year":"2020","journal-title":"Journal of Informetrics"},{"issue":"1","key":"2024122514552196400_bib17","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1016\/j.jpsychores.2014.11.008","article-title":"A generalized view of self-citation: Direct, co-author, collaborative, and coercive induced self-citation","volume":"78","author":"Ioannidis","year":"2015","journal-title":"Journal of Psychosomatic Research"},{"issue":"8","key":"2024122514552196400_bib18","doi-asserted-by":"publisher","first-page":"e3000384","DOI":"10.1371\/journal.pbio.3000384","article-title":"A standardized citation metrics author database annotated for scientific field","volume":"17","author":"Ioannidis","year":"2019","journal-title":"PLOS Biology"},{"issue":"9","key":"2024122514552196400_bib19","doi-asserted-by":"publisher","first-page":"e13151","DOI":"10.1111\/eci.13151","article-title":"A user\u2019s guide to inflated and manipulated impact factors","volume":"49","author":"Ioannidis","year":"2019","journal-title":"European Journal of Clinical Investigation"},{"issue":"4","key":"2024122514552196400_bib20","doi-asserted-by":"publisher","first-page":"567","DOI":"10.1209\/epl\/i2003-00166-9","article-title":"Measuring preferential attachment in evolving networks","volume":"61","author":"Jeong","year":"2003","journal-title":"Europhysics Letters"},{"issue":"7943","key":"2024122514552196400_bib21","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1038\/d41586-022-04577-5","article-title":"\u2018Disruptive\u2019 science has declined\u2014and no one knows why","volume":"613","author":"Kozlov","year":"2023","journal-title":"Nature"},{"issue":"2","key":"2024122514552196400_bib22","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1002\/asi.24852","article-title":"The decrease in uncited articles and its effect on the concentration of citations","volume":"75","author":"Kozlowski","year":"2024","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"3","key":"2024122514552196400_bib23","doi-asserted-by":"publisher","first-page":"036118","DOI":"10.1103\/PhysRevE.71.036118","article-title":"Network growth by copying","volume":"71","author":"Krapivsky","year":"2005","journal-title":"Physical Review E"},{"key":"2024122514552196400_bib24","doi-asserted-by":"publisher","first-page":"562","DOI":"10.1177\/00031224231168074","article-title":"What types of novelty are most disruptive?","volume":"88","author":"Leahey","year":"2023","journal-title":"American Sociological Review"},{"key":"2024122514552196400_bib25","doi-asserted-by":"publisher","first-page":"601","DOI":"10.1007\/s11192-023-04873-5","article-title":"What do we know about the disruption indicator in scientometrics? An overview of the literature","volume":"129","author":"Leibel","year":"2024","journal-title":"Scientometrics"},{"issue":"1","key":"2024122514552196400_bib26","doi-asserted-by":"publisher","first-page":"e300121","DOI":"10.3145\/epi.2021.ene.21","article-title":"A proposal to revise the disruption index","volume":"30","author":"Leydesdorff","year":"2021","journal-title":"El Profesional de la Informaci\u00f3n (EPI)"},{"issue":"5","key":"2024122514552196400_bib27","doi-asserted-by":"publisher","first-page":"104992","DOI":"10.1016\/j.respol.2024.104992","article-title":"Is there a secular decline in disruptive patents? Correcting for measurement bias","volume":"53","author":"Macher","year":"2024","journal-title":"Research Policy"},{"issue":"1","key":"2024122514552196400_bib28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.respol.2015.09.001","article-title":"Editors\u2019 JIF-boosting stratagems\u2014Which are appropriate and which not?","volume":"45","author":"Martin","year":"2016","journal-title":"Research Policy"},{"key":"2024122514552196400_bib29","first-page":"157","article-title":"Synchronous and diachronous citation distributions","volume-title":"Informetrics 87\/88: Select Proceedings of the 1st International Conference on Bibliometrics and Theoretical Aspects of Information Retrieval","author":"Nakamoto","year":"1988"},{"key":"2024122514552196400_bib30","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1007\/s11192-020-03764-3","article-title":"Number of references: A large-scale study of interval ratios","volume":"126","author":"Nicolaisen","year":"2021","journal-title":"Scientometrics"},{"issue":"3","key":"2024122514552196400_bib31","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1016\/S0304-3932(03)00028-X","article-title":"The quest for prosperity without inflation","volume":"50","author":"Orphanides","year":"2003","journal-title":"Journal of Monetary Economics"},{"key":"2024122514552196400_bib32","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1016\/S1573-4498(05)80009-8","article-title":"Money, inflation and growth","volume-title":"Handbook of monetary economics","author":"Orphanides","year":"1990"},{"issue":"3","key":"2024122514552196400_bib33","doi-asserted-by":"publisher","first-page":"656","DOI":"10.1016\/j.joi.2018.06.005","article-title":"The memory of science: Inflation, myopia, and the knowledge network","volume":"12","author":"Pan","year":"2018","journal-title":"Journal of Informetrics"},{"key":"2024122514552196400_bib34","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1038\/s41586-022-05543-x","article-title":"Papers and patents are becoming less disruptive over time","volume":"613","author":"Park","year":"2023","journal-title":"Nature"},{"key":"2024122514552196400_bib35","doi-asserted-by":"publisher","first-page":"700","DOI":"10.1038\/nphys3110","article-title":"Together we stand","volume":"10","author":"Pavlidis","year":"2014","journal-title":"Nature Physics"},{"issue":"4","key":"2024122514552196400_bib36","doi-asserted-by":"publisher","first-page":"100974","DOI":"10.1016\/j.joi.2019.100974","article-title":"Megajournal mismanagement: Manuscript decision bias and anomalous editor activity at PLOS ONE","volume":"13","author":"Petersen","year":"2019","journal-title":"Journal of Informetrics"},{"key":"2024122514552196400_bib37","doi-asserted-by":"publisher","first-page":"2150016","DOI":"10.1142\/S0219525921500168","article-title":"Evolution of biomedical innovation quantified via billions of distinct article-level MeSH keyword combinations","volume":"25","author":"Petersen","year":"2022","journal-title":"Advances in Complex Systems"},{"key":"2024122514552196400_bib38","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1057\/s41599-021-00869-9","article-title":"Grand challenges and emergent modes of convergence science","volume":"8","author":"Petersen","year":"2021","journal-title":"Humanities and Social Sciences Communications"},{"issue":"1","key":"2024122514552196400_bib39","doi-asserted-by":"publisher","first-page":"101605","DOI":"10.1016\/j.joi.2024.101605","article-title":"The disruption index suffers from citation inflation: Re-analysis of temporal CD trend and relationship with team size reveal discrepancies","volume":"19","author":"Petersen","year":"2025","journal-title":"Journal of Informetrics"},{"issue":"43","key":"2024122514552196400_bib40","doi-asserted-by":"publisher","first-page":"15316","DOI":"10.1073\/pnas.1323111111","article-title":"Reputation and impact in academic careers","volume":"111","author":"Petersen","year":"2014","journal-title":"Proceedings of the National Academy of Sciences of the USA"},{"issue":"8","key":"2024122514552196400_bib41","doi-asserted-by":"publisher","first-page":"eaat4211","DOI":"10.1126\/sciadv.aat4211","article-title":"Cross-disciplinary evolution of the genomics revolution","volume":"4","author":"Petersen","year":"2018","journal-title":"Science Advances"},{"issue":"7","key":"2024122514552196400_bib42","doi-asserted-by":"publisher","first-page":"1855","DOI":"10.1016\/j.respol.2019.04.009","article-title":"Methods to account for citation inflation in research evaluation","volume":"48","author":"Petersen","year":"2018","journal-title":"Research Policy"},{"key":"2024122514552196400_bib43","doi-asserted-by":"publisher","first-page":"109821","DOI":"10.1016\/j.chaos.2020.109821","article-title":"Renormalizing individual performance metrics for cultural heritage management of sports records","volume":"136","author":"Petersen","year":"2020","journal-title":"Chaos, Solitons & Fractals"},{"key":"2024122514552196400_bib44","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1140\/epjb\/e2010-10647-1","article-title":"Methods for detrending success metrics to account for inflationary and deflationary factors","volume":"79","author":"Petersen","year":"2011","journal-title":"European Physical Journal B"},{"issue":"3","key":"2024122514552196400_bib45","doi-asserted-by":"publisher","first-page":"036114","DOI":"10.1103\/PhysRevE.81.036114","article-title":"Methods for measuring the citations and productivity of scientists across time and discipline","volume":"81","author":"Petersen","year":"2010","journal-title":"Physical Review E"},{"issue":"37","key":"2024122514552196400_bib46","doi-asserted-by":"publisher","first-page":"16023","DOI":"10.1073\/pnas.1010757107","article-title":"Nonuniversal power law scaling in the probability distribution of scientific citations","volume":"107","author":"Peterson","year":"2010","journal-title":"Proceedings of the National Academy of Sciences of the USA"},{"issue":"2","key":"2024122514552196400_bib47","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1162\/qss_a_00191","article-title":"Do women undertake interdisciplinary research more than men, and do self-citations bias observed differences?","volume":"3","author":"Pinheiro","year":"2022","journal-title":"Quantitative Science Studies"},{"key":"2024122514552196400_bib439","doi-asserted-by":"publisher","DOI":"10.3386\/w32468","article-title":"Working Paper, Working Paper Series 32468. National Bureau of Economic Research","author":"Qiu","year":"2024","journal-title":"Paper tiger? Chinese science and home bias in citations"},{"issue":"45","key":"2024122514552196400_bib48","doi-asserted-by":"publisher","first-page":"17268","DOI":"10.1073\/pnas.0806977105","article-title":"Universality of citation distributions: Toward an objective measure of scientific impact","volume":"105","author":"Radicchi","year":"2008","journal-title":"Proceedings of the National Academy of Sciences of the USA"},{"issue":"6","key":"2024122514552196400_bib49","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1063\/1.1996475","article-title":"Citation statistics from 110 years of Physical Review","volume":"58","author":"Redner","year":"2005","journal-title":"Physics Today"},{"key":"2024122514552196400_bib50","doi-asserted-by":"publisher","first-page":"121071","DOI":"10.1016\/j.techfore.2021.121071","article-title":"Rethinking the disruption index as a measure of scientific and technological advances","volume":"172","author":"Ruan","year":"2021","journal-title":"Technological Forecasting and Social Change"},{"issue":"1","key":"2024122514552196400_bib51","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.joi.2017.11.003","article-title":"Reference density trends in the major disciplines","volume":"12","author":"S\u00e1nchez-Gil","year":"2018","journal-title":"Journal of Informetrics"},{"issue":"34","key":"2024122514552196400_bib52","doi-asserted-by":"publisher","first-page":"12325","DOI":"10.1073\/pnas.1401992111","article-title":"Collective credit allocation in science","volume":"111","author":"Shen","year":"2014","journal-title":"Proceedings of the National Academy of Sciences of the USA"},{"issue":"4","key":"2024122514552196400_bib53","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1093\/biomet\/42.3-4.425","article-title":"On a class of skew distribution functions","volume":"42","author":"Simon","year":"1955","journal-title":"Biometrika"},{"key":"2024122514552196400_bib54","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1145\/2740908.2742839","article-title":"An overview of Microsoft Academic Service (MAS) and applications","volume-title":"Proceedings of the 24th International Conference on World Wide Web","author":"Sinha","year":"2015"},{"issue":"9","key":"2024122514552196400_bib55","doi-asserted-by":"publisher","first-page":"1923","DOI":"10.1002\/asi.23302","article-title":"Is there a clubbing effect underlying Chinese research citation Increases?","volume":"66","author":"Tang","year":"2015","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"7744","key":"2024122514552196400_bib56","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1038\/s41586-019-0941-9","article-title":"Large teams develop and small teams disrupt science and technology","volume":"566","author":"Wu","year":"2019","journal-title":"Nature"},{"key":"2024122514552196400_bib57","doi-asserted-by":"publisher","DOI":"10.31235\/osf.io\/d3wpk","article-title":"A confusing definition of disruption","author":"Wu","year":"2019","journal-title":"SocArXiv"},{"issue":"5827","key":"2024122514552196400_bib58","doi-asserted-by":"publisher","first-page":"1036","DOI":"10.1126\/science.1136099","article-title":"The increasing dominance of teams in production of knowledge","volume":"316","author":"Wuchty","year":"2007","journal-title":"Science"},{"key":"2024122514552196400_bib59","doi-asserted-by":"publisher","first-page":"2350003","DOI":"10.1142\/S0219525923500030","article-title":"Biomedical convergence facilitated by the emergence of technological and informatic capabilities","volume":"26","author":"Yang","year":"2023","journal-title":"Advances in Complex Systems"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/5\/4\/936\/2482656\/qss_a_00333.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/5\/4\/936\/2482656\/qss_a_00333.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,25]],"date-time":"2024-12-25T09:55:31Z","timestamp":1735120531000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/5\/4\/936\/124788\/The-disruption-index-is-biased-by-citation"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":60,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,11,1]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00333","relation":{"has-review":[{"id-type":"doi","id":"10.1162\/QSS_A_00333\/v1\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00333\/v1\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00333\/v1\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00333\/v2\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00333\/v2\/response1","asserted-by":"object"}]},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}