{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T22:57:58Z","timestamp":1776207478189,"version":"3.50.1"},"reference-count":50,"publisher":"MIT Press","issue":"3","content-domain":{"domain":["www.mitpressjournals.org"],"crossmark-restriction":true},"short-container-title":["Quantitative Science Studies"],"published-print":{"date-parts":[[2020,8]]},"abstract":"<jats:p>Pressured by globalization and demand for public organizations to be accountable, efficient, and transparent, university rankings have become an important tool for assessing the quality of higher education institutions. It is therefore important to assess exactly what these rankings measure. Here, the three major global university rankings\u2014the Academic Ranking of World Universities, the Times Higher Education ranking and the Quacquarelli Symonds World University Rankings\u2014are studied. After a description of the ranking methodologies, it is shown that university rankings are stable over time but that there is variation between the three rankings. Furthermore, using principal component analysis and exploratory factor analysis, we demonstrate that the variables used to construct the rankings primarily measure two underlying factors: a university\u2019s reputation and its research performance. By correlating these factors and plotting regional aggregates of universities on the two factors, differences between the rankings are made visible. Last, we elaborate how the results from these analysis can be viewed in light of often-voiced critiques of the ranking process. This indicates that the variables used by the rankings might not capture the concepts they claim to measure. The study provides evidence of the ambiguous nature of university rankings quantification of university performance.<\/jats:p>","DOI":"10.1162\/qss_a_00052","type":"journal-article","created":{"date-parts":[[2020,5,28]],"date-time":"2020-05-28T19:05:22Z","timestamp":1590692722000},"page":"1109-1135","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":55,"title":["A longitudinal analysis of university rankings"],"prefix":"10.1162","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6155-6569","authenticated-orcid":true,"given":"Friso","family":"Selten","sequence":"first","affiliation":[{"name":"Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0068-716X","authenticated-orcid":true,"given":"Cameron","family":"Neylon","sequence":"additional","affiliation":[{"name":"Centre for Culture and Technology, Curtin University, Perth, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9656-5932","authenticated-orcid":true,"given":"Chun-Kai","family":"Huang","sequence":"additional","affiliation":[{"name":"Centre for Culture and Technology, Curtin University, Perth, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0183-6910","authenticated-orcid":true,"given":"Paul","family":"Groth","sequence":"additional","affiliation":[{"name":"Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands"}]}],"member":"281","reference":[{"key":"bib1","volume-title":"Arwu2018 methodology","author":"Academic Ranking of World Universities","year":"2018"},{"issue":"1","key":"bib2","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s11192-010-0190-z","volume":"85","author":"Aguillo I.","year":"2010","journal-title":"Scientometrics"},{"issue":"1","key":"bib3","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1080\/00091383.2012.636001","volume":"44","author":"Altbach P. G.","year":"2012","journal-title":"Change: The Magazine of Higher Learning"},{"key":"bib4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/978-94-6209-338-6","volume-title":"The international imperative in higher education","author":"Altbach P. G.","year":"2013"},{"issue":"3","key":"bib5","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1177\/1028315307303542","volume":"11","author":"Altbach P. G.","year":"2007","journal-title":"Journal of Studies in International Education"},{"issue":"1","key":"bib6","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.joi.2006.08.001","volume":"1","author":"Bar-Ilan J.","year":"2007","journal-title":"Journal of Informetrics"},{"issue":"1","key":"bib7","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1007\/s11192-009-0115-x","volume":"84","author":"Billaut J.-C.","year":"2009","journal-title":"Scientometrics"},{"issue":"2","key":"bib8","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1037\/0033-2909.103.2.276","volume":"103","author":"Cliff N.","year":"1988","journal-title":"Psychological Bulletin"},{"issue":"4","key":"bib9","doi-asserted-by":"crossref","first-page":"663","DOI":"10.2307\/3588284","volume":"38","author":"Curry M. J.","year":"2004","journal-title":"TESOL Quarterly"},{"issue":"2","key":"bib10","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1007\/s11192-009-0076-0","volume":"83","author":"Dehon C.","year":"2009","journal-title":"Scientometrics"},{"key":"bib11","volume-title":"Grid release 2019-05-06","author":"Digital-science","year":"2019"},{"issue":"1","key":"bib12","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/s11192-010-0280-y","volume":"86","author":"Docampo D.","year":"2011","journal-title":"Scientometrics"},{"issue":"4","key":"bib13","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1007\/s00038-012-0416-3","volume":"58","author":"Eisinga R.","year":"2013","journal-title":"International Journal of Public Health"},{"issue":"1","key":"bib14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1086\/517897","volume":"113","author":"Espeland W. N.","year":"2007","journal-title":"American Journal of Sociology"},{"key":"bib15","volume-title":"Discovering statistics using IBM SPSS statistics","author":"Field A.","year":"2013"},{"key":"bib16","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/10719.001.0001","volume-title":"Bibliometrics and research evaluation: Uses and abuses","author":"Gingras Y.","year":"2016"},{"key":"bib17","volume-title":"Statistics for the behavioral sciences","author":"Gravetter F. J.","year":"2016"},{"key":"bib18","volume-title":"Exploratory factor analysis","author":"Habing B.","year":"2003"},{"key":"bib19","volume-title":"Multivariate data analysis","author":"Hair J. F.","year":"2014"},{"issue":"2","key":"bib20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1787\/hemp-v19-art12-en","volume":"19","author":"Hazelkorn E.","year":"2007","journal-title":"Higher Education Management and Policy"},{"issue":"1","key":"bib21","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1093\/reseval\/rvr003","volume":"21","author":"Huang M.-H.","year":"2012","journal-title":"Research Evaluation"},{"issue":"1","key":"bib22","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1198\/004017004000000563","volume":"47","author":"Hubert M.","year":"2005","journal-title":"Technometrics"},{"key":"bib23","unstructured":"INORMS Research Evaluation Group. (2019). What makes a fair and responsible university ranking? Draft criteria for comment (Technical Report)."},{"issue":"1","key":"bib24","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1177\/001316446002000116","volume":"20","author":"Kaiser H. F.","year":"1960","journal-title":"Educational and Psychological Measurement"},{"issue":"1","key":"bib25","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1007\/BF02291575","volume":"39","author":"Kaiser H. F.","year":"1974","journal-title":"Psychometrika"},{"issue":"2","key":"bib26","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1080\/13600800701351660","volume":"29","author":"Marginson S.","year":"2007","journal-title":"Journal of Higher Education Policy and Management"},{"issue":"1","key":"bib27","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1111\/ejed.12061","volume":"49","author":"Marginson S.","year":"2014","journal-title":"European Journal of Education"},{"issue":"2","key":"bib28","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1007\/s11192-016-2212-y","volume":"110","author":"Moed H. F.","year":"2017","journal-title":"Scientometrics"},{"key":"bib29","doi-asserted-by":"crossref","first-page":"86","DOI":"10.4135\/9781412995627.d8","author":"Osborne J. W.","year":"2008","journal-title":"Best Practices in Quantitative Methods"},{"key":"bib30","doi-asserted-by":"crossref","DOI":"10.4135\/9781412984898","volume-title":"Making sense of factor analysis","author":"Pett M.","year":"2003"},{"key":"bib31","volume-title":"Altmetrics: A manifesto","author":"Priem J.","year":"2010"},{"issue":"4","key":"bib32","doi-asserted-by":"crossref","first-page":"544","DOI":"10.1353\/jhe.2013.0022","volume":"84","author":"Pusser B.","year":"2013","journal-title":"Journal of Higher Education"},{"key":"bib33","volume-title":"World university ranking methodology","author":"QS World University Ranking","year":"2018"},{"issue":"3","key":"bib34","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1093\/reseval\/rvz014","volume":"28","author":"Robinson-Garcia N.","year":"2019","journal-title":"Research Evaluation"},{"issue":"1","key":"bib35","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1177\/0020852300661004","volume":"66","author":"Romzek B. S.","year":"2000","journal-title":"International Review of Administrative Sciences"},{"issue":"2","key":"bib36","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1007\/s11192-013-0986-8","volume":"97","author":"Saf\u00f3n V.","year":"2013","journal-title":"Scientometrics"},{"issue":"2","key":"bib37","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1007\/s11192-019-03214-9","volume":"121","author":"Saf\u00f3n V.","year":"2019","journal-title":"Scientometrics"},{"issue":"1","key":"bib38","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.respol.2010.09.003","volume":"40","author":"Saisana M.","year":"2011","journal-title":"Research Policy"},{"key":"bib39","first-page":"113","author":"Scott P.","year":"2013","journal-title":"Rankings and Accountability in Higher Education: Uses and Misuses"},{"key":"bib40","doi-asserted-by":"crossref","DOI":"10.1162\/qss_a_00052","volume-title":"A Longitudinal Analysis of University Rankings","author":"Selten F.","year":"2020"},{"issue":"3","key":"bib41","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1080\/1360080X.2015.1035523","volume":"37","author":"Soh K.","year":"2015","journal-title":"Journal of Higher Education Policy and Management"},{"issue":"2","key":"bib42","doi-asserted-by":"crossref","first-page":"101","DOI":"10.3354\/esep00141","volume":"13","author":"Stergiou K. I.","year":"2014","journal-title":"Ethics in Science and Environmental Politics"},{"issue":"3","key":"bib43","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1080\/13600800701457855","volume":"29","author":"Taylor P.","year":"2007","journal-title":"Journal of Higher Education Policy and Management"},{"key":"bib44","volume-title":"World university rankings 2019: Methodology","author":"THE World University Ranking","year":"2018"},{"key":"bib46","volume-title":"Canadian Education Report Series","author":"Usher A.","year":"2006"},{"issue":"2","key":"bib47","doi-asserted-by":"crossref","first-page":"189","DOI":"10.2143\/EP.16.2.2041651","volume":"16","author":"Van Parijs P.","year":"2009","journal-title":"Ethical Perspectives"},{"issue":"1","key":"bib48","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1007\/s11192-005-0008-6","volume":"62","author":"Van Raan A. F.","year":"2005","journal-title":"Scientometrics"},{"issue":"1","key":"bib49","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1207\/s15327906mbr2501_12","volume":"25","author":"Velicer W. F.","year":"1990","journal-title":"Multivariate Behavioral Research"},{"issue":"3","key":"bib50","doi-asserted-by":"crossref","first-page":"e0193762","DOI":"10.1371\/journal.pone.0193762","volume":"13","author":"Vernon M. M.","year":"2018","journal-title":"PLOS ONE"},{"key":"bib51","volume-title":"Wikidata","author":"Wikidata Contributors","year":"2019"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/qss_a_00052","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,17]],"date-time":"2021-03-17T16:10:31Z","timestamp":1615997431000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/1\/3\/1109-1135\/96094"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8]]},"references-count":50,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,8]]}},"alternative-id":["10.1162\/qss_a_00052"],"URL":"https:\/\/doi.org\/10.1162\/qss_a_00052","relation":{},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8]]},"assertion":[{"value":"2019-08-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-04-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-09-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}