{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T01:08:37Z","timestamp":1785460117290,"version":"3.56.0"},"reference-count":99,"publisher":"MIT Press","issue":"3","license":[{"start":{"date-parts":[[2024,6,28]],"date-time":"2024-06-28T00:00:00Z","timestamp":1719532800000},"content-version":"vor","delay-in-days":179,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Centre for Science and Technology Studies of Leiden University"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Initially created as a marketing and benchmarking tool, global university rankings have evolved into a part of research evaluation and policy initiatives. Governments around the world, concerned about the low rankings of national universities, have launched numerous excellence initiatives in higher education. The aim of this study is to provide an overview of the literature on the use of university rankings in research evaluation and excellence initiatives. A systematic review of the literature on rankings in the context of research evaluation and excellence initiatives was performed. The majority of the review is based on English-language sources, but the review also covers literature from Russia, where the role of rankings is emphasized by their mention in the title and goal of the policy project 5top100. In addition, the review also includes gray literature. The general academic consensus is that rankings cannot be used in research assessment, though a few authors have a positive or neutral attitude. The literature reveals a lot of issues, such as technical and methodological flaws; biases of different nature, including bias towards research in evaluation of both universities and individual researchers; conflicts of interest; and risks to national identity. However, we must acknowledge the complexity of rejecting rankings.<\/jats:p>","DOI":"10.1162\/qss_a_00317","type":"journal-article","created":{"date-parts":[[2024,6,28]],"date-time":"2024-06-28T12:57:50Z","timestamp":1719579470000},"page":"533-555","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":20,"title":["University rankings in the context of research evaluation: A state-of-the-art review"],"prefix":"10.1162","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7890-7532","authenticated-orcid":true,"given":"Dmitry","family":"Kochetkov","sequence":"first","affiliation":[{"name":"Centre for Science and Technology Studies, Leiden University, Leiden, The Netherlands"},{"name":"Department of Probability Theory and Cybersecurity, RUDN University, Moscow, Russia"},{"name":"Institute for the Study of Science, Russian Academy of Sciences, Moscow, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2024,8,1]]},"reference":[{"issue":"1","key":"2024093020300709000_bib1","doi-asserted-by":"publisher","first-page":"112","DOI":"10.17323\/1814-9545-2017-1-112-127","article-title":"Bibliometric evaluation of research performance: Where do we stand?","volume":"2017","author":"Abramo","year":"2017","journal-title":"Voprosy Obrazovaniya\/Educational Studies Moscow"},{"key":"2024093020300709000_bib2","doi-asserted-by":"publisher","DOI":"10.1596\/978-0-8213-8805-1","volume-title":"The road to academic excellence: The making of world-class research universities","author":"Altbach","year":"2011"},{"issue":"3","key":"2024093020300709000_bib3","first-page":"5","article-title":"Global university rankings as a new phenomenon in Russian higher education","volume":"7","author":"Arefiev","year":"2014","journal-title":"Sotsiologicheskaya Nauk i Sotsial\u2019naya Praktika [Sociological Sciences and Social Practice]"},{"key":"2024093020300709000_bib4","first-page":"213","article-title":"On the participation of Russian universities in international rankings","volume":"13","author":"Arefiev","year":"2015","journal-title":"Rossiya Reformiruyushchayasya [Russia in Reform]"},{"key":"2024093020300709000_bib5","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1016\/j.econedurev.2014.03.003","article-title":"Determinants of the international mobility of students","volume":"41","author":"Beine","year":"2014","journal-title":"Economics of Education Review"},{"issue":"1","key":"2024093020300709000_bib6","doi-asserted-by":"publisher","first-page":"4995","DOI":"10.1038\/s41598-022-08859-w","article-title":"Territorial bias in university rankings: A complex network approach","volume":"12","author":"Bellantuono","year":"2022","journal-title":"Scientific Reports"},{"issue":"1","key":"2024093020300709000_bib7","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1007\/s11192-011-0419-5","article-title":"Improving quality assessment of composite indicators in university rankings: A case study of French and German universities of excellence","volume":"89","author":"Benito","year":"2011","journal-title":"Scientometrics"},{"key":"2024093020300709000_bib8","volume-title":"Berlin principles on ranking of higher education institutions","year":"2006"},{"issue":"1","key":"2024093020300709000_bib9","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1007\/s11192-009-0115-x","article-title":"Should you believe in the Shanghai ranking?","volume":"84","author":"Billaut","year":"2010","journal-title":"Scientometrics"},{"issue":"4","key":"2024093020300709000_bib10","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1093\/scipol\/scx078","article-title":"Universities as celebrities? How the media select information from a large research assessment exercise","volume":"45","author":"Blasi","year":"2018","journal-title":"Science and Public Policy"},{"issue":"1","key":"2024093020300709000_bib11","first-page":"15","article-title":"The role of the state in creating world-class universities","volume":"28","author":"Bogolib","year":"2016","journal-title":"University Economic Bulletin"},{"key":"2024093020300709000_bib12","first-page":"17","article-title":"World university rankings: Catch up and overtake. Is it necessary? Part 1. World University Rankings","volume":"4","author":"Bolsherotov","year":"2013","journal-title":"Zhilishchnoe Stroitel\u2019stvo [Housing Construction]"},{"key":"2024093020300709000_bib13","first-page":"57","article-title":"Analysis of the methodology and practice of forming university rankings","volume":"2","author":"Bolsherotov","year":"2020","journal-title":"Vesnik Hrodzenskaha Dziarzhaunaha Universiteta Imia Ianki Kupaly. Seryia 3. Filalohiia. Pedahohika. Psikhalohiia [Bulletin of Yanka Kupala State University of Grodno. Series 3. Philology. Pedagogy. Psychology]"},{"key":"2024093020300709000_bib14","volume-title":"University rankings: Diversity, excellence and the European initiative","author":"Boulton","year":"2010"},{"key":"2024093020300709000_bib15","article-title":"So verr\u00fcckt k\u00f6nnen Rankings sein","author":"Brankovic","year":"2020","journal-title":"Frankfurter Allgemeine"},{"key":"2024093020300709000_bib16","volume-title":"Bulletin of the Accounts Chamber Vol. 2 (279)","year":"2021"},{"key":"2024093020300709000_bib17","doi-asserted-by":"publisher","first-page":"791","DOI":"10.1007\/s10734-022-00942-5","article-title":"Does conflict of interest distort global university rankings?","volume":"86","author":"Chirikov","year":"2023","journal-title":"Higher Education"},{"key":"2024093020300709000_bib18","first-page":"23","volume-title":"Agreement on reforming research assessment","author":"CoARA","year":"2022"},{"key":"2024093020300709000_bib19","doi-asserted-by":"publisher","DOI":"10.6084\/m9.figshare.21701624","volume-title":"Harnessing the Metric Tide: Indicators, infrastructures & priorities for UK responsible research assessment","author":"Curry","year":"2022"},{"issue":"5","key":"2024093020300709000_bib20","doi-asserted-by":"publisher","first-page":"102387","DOI":"10.1016\/j.acalib.2021.102387","article-title":"University rankings and institutional affiliations: Role of academic librarians","volume":"47","author":"Dimzov","year":"2021","journal-title":"Journal of Academic Librarianship"},{"issue":"1","key":"2024093020300709000_bib21","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1007\/s11192-010-0280-y","article-title":"On using the Shanghai ranking to assess the research performance of university systems","volume":"86","author":"Docampo","year":"2011","journal-title":"Scientometrics"},{"issue":"2","key":"2024093020300709000_bib22","doi-asserted-by":"publisher","first-page":"1347","DOI":"10.1007\/s11192-013-1143-0","article-title":"On the internal dynamics of the Shanghai ranking","volume":"98","author":"Docampo","year":"2014","journal-title":"Scientometrics"},{"issue":"6","key":"2024093020300709000_bib23","doi-asserted-by":"publisher","first-page":"6","DOI":"10.15826\/umpa.2017.06.070","article-title":"Frontier university: Towards a new generation of development institutions","volume":"21","author":"Efimov","year":"2017","journal-title":"Universitetskoye Upravleniye: Praktika i Analiz [University Management: Practice and Analysis]"},{"key":"2024093020300709000_bib24","volume-title":"Engines of anxiety: Academic rankings, reputation, and accountability","author":"Espeland","year":"2016"},{"key":"2024093020300709000_bib25","volume-title":"Key considerations for the use of rankings by higher education institutions","author":"European University Association","year":"2023"},{"issue":"1","key":"2024093020300709000_bib26","first-page":"79","article-title":"University rankings: A review of methodological flaws","volume":"30","author":"Fauzi","year":"2020","journal-title":"Issues in Educational Research"},{"key":"2024093020300709000_bib27","volume-title":"Project 5-100: Program results (in Russian)","author":"Forbes Education","year":"2020"},{"issue":"4","key":"2024093020300709000_bib28","first-page":"9","article-title":"University rankings as an element of strategic management in higher education","volume":"73","author":"Frank","year":"2017","journal-title":"Global\u2019nyy Nauchnyy Potentsial [Global Scientific Potential]"},{"key":"2024093020300709000_bib29","doi-asserted-by":"publisher","first-page":"680023","DOI":"10.3389\/frma.2021.680023","article-title":"Mis-measuring our universities: Why global university rankings don\u2019t add up","volume":"6","author":"Gadd","year":"2021","journal-title":"Frontiers in Research Metrics and Analytics"},{"issue":"4","key":"2024093020300709000_bib30","doi-asserted-by":"publisher","first-page":"554","DOI":"10.1080\/03057925.2018.1538770","article-title":"\u201cHeavy mountains\u201d for Chinese humanities and social science academics in the quest for world-class universities","volume":"50","author":"Gao","year":"2020","journal-title":"Compare"},{"issue":"1","key":"2024093020300709000_bib31","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.aula.2015.08.001","article-title":"El impacto de las pol\u00edticas de evaluaci\u00f3n del profesorado en la posici\u00f3n en los r\u00e1nquines universitarios: El caso de la Universidad Polit\u00e9cnica de Catalu\u00f1a","volume":"44","author":"Garc\u00eda-Berro","year":"2016","journal-title":"Aula Abierta"},{"issue":"2","key":"2024093020300709000_bib32","first-page":"27","article-title":"Excellence in higher education: Experience and perspectives","volume":"18","author":"Gazizova","year":"2015","journal-title":"Professional\u2019noye Obrazovaniye v Rossii i Zarubezhom [Professional Education in Russia and Abroad]"},{"key":"2024093020300709000_bib33","article-title":"Global University Rankings 2007: Interview with Simon Marginson","year":"2007","journal-title":"GlobalHigherEd"},{"key":"2024093020300709000_bib34","volume-title":"Enhancing quality: From policy to practice (Enhancing Quality through Innovative Policy & Practice\/EQUIP)","author":"Gover","year":"2018"},{"key":"2024093020300709000_bib35","article-title":"Vying for a spot on the world\u2019s A list","author":"Guttenplan","year":"2013","journal-title":"New York Times"},{"issue":"3","key":"2024093020300709000_bib36","first-page":"15","article-title":"Positions and prospects of the participants of the 5-100-2020 project in international university rankings","volume":"17","author":"Guzikova","year":"2014","journal-title":"Voprosy Metodiki Prepodavaniya v Vuze [Teaching Methodology in Higher Education]"},{"key":"2024093020300709000_bib37","article-title":"University rankings and their critics \u2013 a symbiotic relationship?","author":"Hamann","year":"2023","journal-title":"LSE Impact Blog"},{"key":"2024093020300709000_bib38","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1057\/s41599-017-0011-6","article-title":"Global science, national research, and the question of university rankings","volume":"3","author":"Hazelkorn","year":"2017","journal-title":"Palgrave Communications"},{"key":"2024093020300709000_bib39","volume-title":"Rankings in institutional strategies and processes: Impact or illusion?","author":"Hazelkorn","year":"2014"},{"issue":"7548","key":"2024093020300709000_bib40","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1038\/520429a","article-title":"Bibliometrics: The Leiden Manifesto for research metrics","volume":"520","author":"Hicks","year":"2015","journal-title":"Nature"},{"key":"2024093020300709000_bib41","article-title":"Are global university rankings losing their credibility?","author":"Holmes","year":"2015","journal-title":"WONKHE"},{"issue":"2","key":"2024093020300709000_bib42","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1162\/qss_a_00031","article-title":"Comparison of bibliographic data sources: Implications for the robustness of university rankings","volume":"1","author":"Huang","year":"2020","journal-title":"Quantitative Science Studies"},{"issue":"1","key":"2024093020300709000_bib43","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1093\/reseval\/rvr003","article-title":"Opening the black box of QS world university rankings","volume":"21","author":"Huang","year":"2012","journal-title":"Research Evaluation"},{"issue":"2","key":"2024093020300709000_bib44","doi-asserted-by":"publisher","DOI":"10.15366\/reice2016.14.2.001","article-title":"Rankings universit\u00e1rios mundiais. Que dizem os estudos internacionais?","volume":"14","author":"Iesbik Valmorbida","year":"2016","journal-title":"REICE. Revista Iberoamericana Sobre Calidad, Eficacia y Cambio en Educaci\u00f3n"},{"key":"2024093020300709000_bib45","volume-title":"More than our rank","author":"INORMS","year":"2022"},{"key":"2024093020300709000_bib46","volume-title":"Fair and responsible university assessment: Application to the global university rankings and beyond","author":"INORMS Research Evaluation Working Group","year":"2022"},{"key":"2024093020300709000_bib47","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1186\/1741-7015-5-30","article-title":"International ranking systems for universities and institutions: A critical appraisal","volume":"5","author":"Ioannidis","year":"2007","journal-title":"BMC Medicine"},{"issue":"1","key":"2024093020300709000_bib48","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1007\/s11192-018-2666-1","article-title":"University rankings: What do they really show?","volume":"115","author":"Johnes","year":"2018","journal-title":"Scientometrics"},{"issue":"1","key":"2024093020300709000_bib49","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1353\/rhe.2019.0094","article-title":"Sisyphus revisited: Efficiency developments in European universities 2011\u20132016 according to ranking and budget data","volume":"43","author":"Klumpp","year":"2019","journal-title":"Review of Higher Education"},{"key":"2024093020300709000_bib50","doi-asserted-by":"publisher","DOI":"10.59350\/6na1c-z9n39","article-title":"Priority-2030: The new excellence initiative from Russia","author":"Kochetkov","year":"2022","journal-title":"LeidenMadtrics"},{"issue":"1","key":"2024093020300709000_bib52","doi-asserted-by":"publisher","first-page":"576","DOI":"10.1057\/s41599-023-02082-2","article-title":"Review of the Russian-language academic literature on university rankings and a global perspective","volume":"10","author":"Kochetkov","year":"2023","journal-title":"Humanities and Social Sciences Communications"},{"key":"2024093020300709000_bib53","article-title":"U.S. News & World Report to revamp parts of its law-school ranking","author":"Korn","year":"2023","journal-title":"Wall Street Journal"},{"issue":"4","key":"2024093020300709000_bib251","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3145\/EPI.2021.JUL.08","article-title":"The Shanghai Global Ranking of Academic Subjects: Room for improvement","volume":"30","author":"Krauskopf","year":"2021","journal-title":"Profesional de La Informacion"},{"issue":"2","key":"2024093020300709000_bib54","first-page":"17","article-title":"World university rankings and the program \u201c5-100-2020\u201d of the Ministry of Education and Science of the Russian Federation as a way to increase the competitiveness of Russian universities","volume":"31","author":"Kushneva","year":"2014","journal-title":"Obshchestvo. Sreda. Razvitiye [Society. Environment. Development]"},{"key":"2024093020300709000_bib55","doi-asserted-by":"publisher","first-page":"134","DOI":"10.33933\/2074-2762-2019-54-134-144","article-title":"Consequences of the fascination with quantitative performance indicators in science and higher education","volume":"54","author":"Lazar","year":"2019","journal-title":"Uchenyye Zapiski Rossiyskogo Gosudarstvennogo Gidrometeorologicheskogo Universiteta [Proceedings of the Russian State Hydrometeorological University]"},{"key":"2024093020300709000_bib56","first-page":"125","article-title":"International rankings as a tool for assessing the competitiveness of the university","volume":"10","author":"Leonova","year":"2017","journal-title":"Vestnik Universiteta [Bulletin of the University]"},{"issue":"4","key":"2024093020300709000_bib57","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1057\/s41307-016-0015-7","article-title":"The global ranking regime and the reconfiguration of higher education: Comparative case studies on research assessment exercises in China, Hong Kong, and Japan","volume":"29","author":"Li","year":"2016","journal-title":"Higher Education Policy"},{"issue":"3","key":"2024093020300709000_bib58","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1007\/s10734-017-0147-8","article-title":"The building of weak expertise: The work of global university rankers","volume":"75","author":"Lim","year":"2018","journal-title":"Higher Education"},{"key":"2024093020300709000_bib59","doi-asserted-by":"publisher","first-page":"444","DOI":"10.4337\/9781788974981.00044","article-title":"The business of university rankings: The case of the Times Higher Education","volume-title":"Research handbook on university rankings: History, methodology, influence and impact","author":"Lim","year":"2021"},{"key":"2024093020300709000_bib60","volume-title":"Exploring higher education indicators","author":"Loukkola","year":"2020"},{"issue":"3","key":"2024093020300709000_bib61","doi-asserted-by":"publisher","first-page":"43","DOI":"10.5937\/StraMan1903043L","article-title":"Indicators of global university rankings: The theoretical issues","volume":"24","author":"Luki\u0107","year":"2019","journal-title":"Strategic Management"},{"issue":"3","key":"2024093020300709000_bib62","doi-asserted-by":"publisher","first-page":"e143","DOI":"10.3989\/redc.2016.3.1339","article-title":"Influencia del programa Campus de Excelencia Internacional en la posici\u00f3n de las universidades espa\u00f1olas en el ranking de Shangh\u00e1i","volume":"39","author":"Luque-Mart\u00ednez","year":"2016","journal-title":"Revista Espa\u00f1ola de Documentaci\u00f3n Cient\u00edfica"},{"key":"2024093020300709000_bib63","article-title":"World university rankings: How much influence do they really have?","author":"Luxbacher","year":"2013","journal-title":"The Guardian"},{"issue":"5","key":"2024093020300709000_bib64","doi-asserted-by":"publisher","first-page":"847","DOI":"10.1007\/s10734-019-00471-8","article-title":"Ratings, rankings, research evaluation: How do Schools of Education behave strategically within stratified UK higher education?","volume":"79","author":"Marques","year":"2020","journal-title":"Higher Education"},{"key":"2024093020300709000_bib65","doi-asserted-by":"publisher","first-page":"101264","DOI":"10.1016\/j.tele.2019.101264","article-title":"Linking YouTube and university rankings: Research performance as predictor of online video impact","volume":"43","author":"Meseguer-Martinez","year":"2019","journal-title":"Telematics and Informatics"},{"key":"2024093020300709000_bib66","first-page":"150","article-title":"World-class universities: What is it?","volume":"1","author":"Milkevich","year":"2008","journal-title":"Izvestiya Yuzhnogo Federal\u2019nogo Universiteta. Filologicheskiye Nauki [Proceedings of the Southern Federal University. Philological Sciences]"},{"issue":"2","key":"2024093020300709000_bib67","doi-asserted-by":"publisher","first-page":"967","DOI":"10.1007\/s11192-016-2212-y","article-title":"A critical comparative analysis of five world university rankings","volume":"110","author":"Moed","year":"2017","journal-title":"Scientometrics"},{"key":"2024093020300709000_bib68","article-title":"World university rankings","author":"Moran","year":"2010","journal-title":"Sandwalk"},{"key":"2024093020300709000_bib69","doi-asserted-by":"publisher","DOI":"10.37941\/PB\/2023\/1","volume-title":"World-class universities? Interrogating the biases and coloniality of global university rankings","author":"Nassiri-Ansari","year":"2023"},{"issue":"1","key":"2024093020300709000_bib70","first-page":"5","article-title":"On the problems of forming world-class universities and positioning rankings","volume":"15","author":"Nikolenko","year":"2014","journal-title":"Sechenovskiy Vestnik [Sechenov Medical Journal]"},{"key":"2024093020300709000_bib71","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1016\/j.techfore.2016.03.029","article-title":"Is measuring the knowledge creation of universities possible?: A review of university rankings","volume":"123","author":"Olcay","year":"2017","journal-title":"Technological Forecasting and Social Change"},{"issue":"1","key":"2024093020300709000_bib72","doi-asserted-by":"publisher","first-page":"61","DOI":"10.14429\/djlit.41.1.16328","article-title":"Quest for ranking excellence: Impact study of research metrics","volume":"41","author":"Pakkan","year":"2021","journal-title":"DESIDOC Journal of Library and Information Technology"},{"issue":"2","key":"2024093020300709000_bib73","doi-asserted-by":"publisher","first-page":"e204","DOI":"10.3989\/redc.2018.2.1462","article-title":"Model for estimating Academic Ranking of World Universities (Shanghai Ranking) scores","volume":"41","author":"Pandiella-Dominique","year":"2018","journal-title":"Revista Espanola de Documentacion Cientifica"},{"issue":"1","key":"2024093020300709000_bib74","first-page":"7","article-title":"World-class universities: Formation strategies","volume":"16","author":"Pankova","year":"2015","journal-title":"Vestnik Nauki Sibiri [Bulletin of Science of Siberia]"},{"issue":"4","key":"2024093020300709000_bib75","doi-asserted-by":"publisher","first-page":"105","DOI":"10.24888\/2073-8439-2019-48-4-105-113","article-title":"Participation in world university rankings as a determining factor influencing the educational policy of countries and the development of universities","volume":"48","author":"Puzatykh","year":"2019","journal-title":"Psikhologiya Obrazovaniya v Polikul\u2019turnom Prostranstve [Educational Psychology in Polycultural Space]"},{"issue":"4","key":"2024093020300709000_bib255","first-page":"18","article-title":"Independent living rankings and educational outcomes","volume":"9","author":"Pyatenko","year":"2019","journal-title":"Original\u2019nyye Issledovaniya [Original Research]"},{"key":"2024093020300709000_bib76","volume-title":"Ranking the university: On the effects of rankings on the academic community and how to overcome them","year":"2023"},{"key":"2024093020300709000_bib77","volume-title":"Global university rankings and their impact","author":"Rauhvargers","year":"2011"},{"key":"2024093020300709000_bib78","volume-title":"Global university rankings and their impact. Report II","author":"Rauhvargers","year":"2013"},{"issue":"3","key":"2024093020300709000_bib79","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1093\/reseval\/rvz014","article-title":"Mining university rankings: Publication output and citation impact as their basis","volume":"28","author":"Robinson-Garcia","year":"2019","journal-title":"Research Evaluation"},{"key":"2024093020300709000_bib80","doi-asserted-by":"publisher","DOI":"10.31235\/osf.io\/cdmqz","article-title":"The rise of \u2018responsible metrics\u2019 as a professional reform movement: A collective action frames perspective","author":"Rushforth","year":"2022","journal-title":"SocArXiv"},{"issue":"2","key":"2024093020300709000_bib81","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1007\/s11192-013-0986-8","article-title":"What do global university rankings really measure? The search for the X factor and the X entity","volume":"97","author":"Saf\u00f3n","year":"2013","journal-title":"Scientometrics"},{"issue":"3","key":"2024093020300709000_bib82","doi-asserted-by":"publisher","first-page":"2199","DOI":"10.1007\/s11192-020-03722-z","article-title":"Analyzing the impact of reputational bias on global university rankings based on objective research performance data: The case of the Shanghai Ranking (ARWU)","volume":"125","author":"Saf\u00f3n","year":"2020","journal-title":"Scientometrics"},{"key":"2024093020300709000_bib83","author":"San Francisco Declaration on Research Assessment (DORA)","year":"2013"},{"issue":"3","key":"2024093020300709000_bib84","doi-asserted-by":"publisher","first-page":"1109","DOI":"10.1162\/qss_a_00052","article-title":"A longitudinal analysis of university rankings","volume":"1","author":"Selten","year":"2020","journal-title":"Quantitative Science Studies"},{"issue":"6","key":"2024093020300709000_bib85","doi-asserted-by":"publisher","first-page":"922","DOI":"10.1080\/03057925.2020.1736403","article-title":"Globalisation or westernisation? The influence of global university rankings in the context of the Republic of China (Taiwan)","volume":"50","author":"Shreeve","year":"2020","journal-title":"Compare"},{"issue":"1","key":"2024093020300709000_bib86","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1080\/1360080X.2016.1254431","article-title":"The seven deadly sins of world university ranking: A summary from several papers","volume":"39","author":"Soh","year":"2017","journal-title":"Journal of Higher Education Policy and Management"},{"issue":"3","key":"2024093020300709000_bib253","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1145\/3356773.3356776","article-title":"Grey literature: A safe bridge between academy and industry?","volume":"44","author":"Soldani","year":"2020","journal-title":"ACM SIGSOFT Software Engineering Notes"},{"issue":"2","key":"2024093020300709000_bib87","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1108\/IJCED-08-2018-0029","article-title":"Academic excellence as \u201ccompetitiveness enhancement\u201d in Russian higher education","volume":"21","author":"Tsvetkova","year":"2019","journal-title":"International Journal of Comparative Education and Development"},{"issue":"1","key":"2024093020300709000_bib88","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1007\/s11192-020-03637-9","article-title":"Complex networks for benchmarking in global universities rankings","volume":"125","author":"Tuesta","year":"2020","journal-title":"Scientometrics"},{"key":"2024093020300709000_bib89","author":"U-Map","year":".."},{"key":"2024093020300709000_bib90","article-title":"Ten years of global university rankings","author":"Usher","year":"2013","journal-title":"Higher Education Strategy Associates"},{"issue":"1","key":"2024093020300709000_bib91","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1007\/s11192-005-0008-6","article-title":"Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods","volume":"62","author":"van Raan","year":"2005","journal-title":"Scientometrics"},{"issue":"2","key":"2024093020300709000_bib92","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1007\/s11192-011-0382-1","article-title":"Severe language effect in university rankings: Particularly Germany and France are wronged in citation-based rankings","volume":"88","author":"van Raan","year":"2011","journal-title":"Scientometrics"},{"issue":"6","key":"2024093020300709000_bib93","doi-asserted-by":"publisher","first-page":"956","DOI":"10.1177\/1745691617699971","article-title":"Cultural change: The how and the why","volume":"12","author":"Varnum","year":"2017","journal-title":"Perspectives on Psychological Science"},{"issue":"3","key":"2024093020300709000_bib94","doi-asserted-by":"publisher","first-page":"e0193762","DOI":"10.1371\/journal.pone.0193762","article-title":"Are university rankings useful to improve research? A systematic review","volume":"13","author":"Vernon","year":"2018","journal-title":"PLOS ONE"},{"key":"2024093020300709000_bib95","article-title":"Ten principles for the responsible use of university rankings","author":"Waltman","year":"2020","journal-title":"CWTS Blog"},{"key":"2024093020300709000_bib96","doi-asserted-by":"publisher","DOI":"10.13140\/RG.2.1.4929.1363","volume-title":"The Metric Tide: Report of the independent review of the role of metrics in research assessment and management","author":"Wilsdon","year":"2015"},{"issue":"4","key":"2024093020300709000_bib97","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1007\/s11125-015-9365-y","article-title":"Global university rankings: The \u201cOlympic Games\u201d of higher education?","volume":"45","author":"Yudkevich","year":"2015","journal-title":"Prospects"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/5\/3\/533\/2472669\/qss_a_00317.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/5\/3\/533\/2472669\/qss_a_00317.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T16:30:30Z","timestamp":1727713830000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/5\/3\/533\/123261\/University-rankings-in-the-context-of-research"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":99,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,8,1]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00317","relation":{"has-review":[{"id-type":"doi","id":"10.1162\/QSS_A_00317\/v1\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00317\/v1\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00317\/v2\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00317\/v2\/decision1","asserted-by":"object"}],"has-preprint":[{"id-type":"doi","id":"10.31235\/osf.io\/hykze","asserted-by":"object"}]},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}