{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T23:11:03Z","timestamp":1788390663604,"version":"build-2803163510"},"reference-count":49,"publisher":"MIT Press","issue":"1","license":[{"start":{"date-parts":[[2021,3,8]],"date-time":"2021-03-08T00:00:00Z","timestamp":1615161600000},"content-version":"vor","delay-in-days":66,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61603309"],"award-info":[{"award-number":["61603309"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,4,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>University ranking has become an important indicator for prospective students, job recruiters, and government administrators. The fact that a university rarely has the same position in different rankings motivates us to ask: To what extent could a university\u2019s best rank deviate from its \u201ctrue\u201d position? Here we focus on 14 rankings of Chinese universities. We find that a university\u2019s rank in different rankings is not consistent. However, the relative positions for a particular set of universities are more similar. The increased similarity is not distributed uniformly among all rankings. Instead, the 14 rankings demonstrate four clusters where rankings are more similar inside the cluster than outside. We find that a university\u2019s best rank strongly correlates with its consensus rank, which is, on average, 38% higher (towards the top). Therefore, the best rank usually advertised by a university adequately reflects the collective opinion of experts. We can trust it, but with a discount. With the best rank and proportionality relationship, a university\u2019s consensus rank can be estimated with reasonable accuracy. Our work not only reveals previously unknown patterns in university rankings but also introduces a set of tools that can be readily applied to future studies.<\/jats:p>","DOI":"10.1162\/qss_a_00101","type":"journal-article","created":{"date-parts":[[2021,1,27]],"date-time":"2021-01-27T16:25:10Z","timestamp":1611764710000},"page":"335-349","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":8,"title":["The rank boost by inconsistency in university rankings: Evidence from\n                    14 rankings of Chinese universities"],"prefix":"10.1162","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4299-8764","authenticated-orcid":true,"given":"Wenyu","family":"Chen","sequence":"first","affiliation":[{"name":"College of Computer and Information Science, Southwest University, Chongqing, 400715, P. R. China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9052-3208","authenticated-orcid":true,"given":"Zhangqian","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of National Defense Economy, Army Logistics University of Chinese People\u2019s Liberation Army, Chongqing, 500106, P. R. China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2337-2857","authenticated-orcid":true,"given":"Tao","family":"Jia","sequence":"additional","affiliation":[{"name":"College of Computer and Information Science, Southwest University, Chongqing, 400715, P. R. China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2021,4,8]]},"reference":[{"issue":"4","key":"2021041320051612800_bib1","doi-asserted-by":"crossref","first-page":"889","DOI":"10.1016\/j.joi.2016.07.004","article-title":"A comparison of university performance scores and ranks by\n                        MNCS and FSS","volume":"10","author":"Abramo","year":"2016","journal-title":"Journal of Informetrics"},{"issue":"1","key":"2021041320051612800_bib2","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s11192-010-0190-z","article-title":"Comparing university\n                        rankings","volume":"85","author":"Aguillo","year":"2010","journal-title":"Scientometrics"},{"issue":"2","key":"2021041320051612800_bib3","doi-asserted-by":"crossref","first-page":"667","DOI":"10.1016\/j.ejor.2015.08.048","article-title":"Accurate algorithms for identifying the\n                        median ranking when dealing with weak and partial rankings under the Kemeny\n                        axiomatic approach","volume":"249","author":"Amodio","year":"2016","journal-title":"European Journal of Operational\n                        Research"},{"issue":"1","key":"2021041320051612800_bib4","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1080\/09737766.2018.1550043","article-title":"On the necessity of multiple university\n                        rankings","volume":"13","author":"Angelis","year":"2019","journal-title":"COLLNET Journal of Scientometrics and\n                        Information Management"},{"key":"2021041320051612800_bib5","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1007\/978-3-319-06764-3_72","article-title":"A critical review on world university ranking in terms of top\n                        four ranking systems","volume-title":"New trends in networking,\n                        computing, e-learning, systems sciences, and engineering","author":"Anowar","year":"2015"},{"issue":"1","key":"2021041320051612800_bib6","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.joi.2006.08.001","article-title":"Some measures for comparing citation\n                        databases","volume":"1","author":"Bar-Ilan","year":"2007","journal-title":"Journal of Informetrics"},{"issue":"1","key":"2021041320051612800_bib7","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s11162-010-9185-0","article-title":"College rankings as an interorganizational dependency:\n                        Establishing the foundation for strategic and institutional\n                        accounts","volume":"52","author":"Bastedo","year":"2011","journal-title":"Research in Higher Education"},{"issue":"11","key":"2021041320051612800_bib8","doi-asserted-by":"crossref","DOI":"10.14778\/2809974.2809982","article-title":"Rank aggregation with ties: Experiments\n                        and analysis","volume":"8","author":"Brancotte","year":"2015","journal-title":"Proceedings of the VLDB\n                        Endowment"},{"issue":"3","key":"2021041320051612800_bib9","doi-asserted-by":"crossref","first-page":"813","DOI":"10.1007\/s11192-015-1586-6","article-title":"A comparative analysis of global and\n                        national university ranking systems","volume":"103","author":"\u00c7ak\u0131r","year":"2015","journal-title":"Scientometrics"},{"issue":"1","key":"2021041320051612800_bib10","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/s11192-012-0724-7","article-title":"A comparative study on world university\n                        rankings: A bibliometric survey","volume":"92","author":"Chen","year":"2012","journal-title":"Scientometrics"},{"issue":"8","key":"2021041320051612800_bib11","doi-asserted-by":"crossref","first-page":"080201","DOI":"10.7498\/aps.69.20191584","article-title":"Comparison of performance of rank\n                        aggregation algorithms in aggregating a small number of long rank lists (in\n                        Chinese)","volume":"69","author":"Chen","year":"2020","journal-title":"Acta Physica Sinica"},{"key":"2021041320051612800_bib12","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1007\/978-3-642-22351-8_5","article-title":"Using medians to generate consensus\n                        rankings for biological data","volume-title":"Scientific and\n                        statistical database management","author":"Cohen-Boulakia","year":"2011"},{"issue":"4","key":"2021041320051612800_bib13","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1016\/j.cor.2005.05.030","article-title":"Creating a consensus ranking of\n                        proposals from reviewers\u2019 partial ordinal rankings","volume":"34","author":"Cook","year":"2007","journal-title":"Computers & Operations Research"},{"issue":"3","key":"2021041320051612800_bib14","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1111\/j.1911-3838.2010.00011.x","article-title":"Aggregating incomplete lists of journal rankings: An\n                        application to academic accounting journals","volume":"9","author":"Cook","year":"2010","journal-title":"Accounting Perspectives"},{"key":"2021041320051612800_bib15","first-page":"697","article-title":"A computational study of the Kemeny rule\n                        for preference aggregation","volume-title":"Proceedings of the\n                        19th National Conference on Artificial Intelligence","author":"Davenport","year":"2004"},{"key":"2021041320051612800_bib16","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1145\/371920.372165","article-title":"Rank aggregation methods for the\n                        web","volume-title":"Proceedings of the 10th International\n                        Conference on World Wide Web","author":"Dwork","year":"2001"},{"issue":"2","key":"2021041320051612800_bib17","article-title":"ConsRank: Compute the median ranking(s)\n                        according to the Kemeny\u2019s axiomatic approach","volume":"1","author":"D\u2019Ambrosio","year":"2015","journal-title":"R package version"},{"issue":"6379","key":"2021041320051612800_bib18","doi-asserted-by":"crossref","first-page":"eaao0185","DOI":"10.1126\/science.aao0185","article-title":"Science of science","volume":"359","author":"Fortunato","year":"2018","journal-title":"Science"},{"issue":"3","key":"2021041320051612800_bib19","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1016\/j.joi.2018.04.005","article-title":"Ranking dynamics and\n                        volatility","volume":"12","author":"Garcia-Zorita","year":"2018","journal-title":"Journal of Informetrics"},{"key":"2021041320051612800_bib20","article-title":"Rankings and quality assurance: Do\n                        rankings measure quality","volume-title":"Policy Brief","author":"Hazelkorn","year":"2015"},{"issue":"12","key":"2021041320051612800_bib21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v071.i12","article-title":"Permallows: An R package for mallows and\n                        generalized mallows models","volume":"71","author":"Irurozki","year":"2016","journal-title":"Journal of Statistical\n                        Software"},{"issue":"4","key":"2021041320051612800_bib22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41562-017-0078","article-title":"Quantifying patterns of research-interest\n                        evolution","volume":"1","author":"Jia","year":"2017","journal-title":"Nature Human Behaviour"},{"issue":"1","key":"2021041320051612800_bib23","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1093\/bib\/bbx101","article-title":"A comparative study of rank aggregation\n                        methods for partial and top ranked lists in genomic\n                        applications","volume":"20","author":"Li","year":"2019","journal-title":"Briefings in Bioinformatics"},{"key":"2021041320051612800_bib24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.physrep.2017.05.001","article-title":"Ranking in evolving complex\n                        networks","volume":"689","author":"Liao","year":"2017","journal-title":"Physics Reports"},{"issue":"3","key":"2021041320051612800_bib25","first-page":"179","article-title":"On the rank changes of world-class\n                        universities in the past 10 years and its enlightenment (in\n                        Chinese)","volume":"2017","author":"Liu","year":"2017","journal-title":"Research in Higher Education of\n                        Engineering"},{"key":"2021041320051612800_bib26","doi-asserted-by":"crossref","DOI":"10.1162\/qss_a_00099","article-title":"The dominance of big teams in\n                        China\u2019s scientific output","author":"Liu","year":"2021","journal-title":"Quantitative\n                        Science Studies"},{"issue":"1","key":"2021041320051612800_bib27","first-page":"40","article-title":"A correlation analysis on the sequences\n                        of the three world university rankings\u2013WRWU, QS & ARWU: An\n                        empirical study based on twenty-two China\u2019s domestic universities\n                        ranking results (in Chinese)","volume":"27","author":"Liu","year":"2011","journal-title":"Education\n                        Science"},{"key":"2021041320051612800_bib28","article-title":"Consensus ranking under the exponential\n                    model","author":"Meila","year":"2012","journal-title":"arXiv preprint"},{"issue":"2","key":"2021041320051612800_bib29","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1007\/s11192-016-2212-y","article-title":"A critical comparative analysis of five world university\n                        rankings","volume":"110","author":"Moed","year":"2017","journal-title":"Scientometrics"},{"issue":"3","key":"2021041320051612800_bib30","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1016\/j.joi.2018.06.005","article-title":"The memory of science: Inflation,\n                        myopia, and the knowledge network","volume":"12","author":"Pan","year":"2018","journal-title":"Journal of\n                        Informetrics"},{"issue":"2","key":"2021041320051612800_bib31","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1007\/s11081-011-9135-5","article-title":"A minimum violations ranking\n                        method","volume":"13","author":"Pedings","year":"2012","journal-title":"Optimization and Engineering"},{"issue":"2","key":"2021041320051612800_bib32","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1007\/s11192-014-1263-1","article-title":"An insight into the importance of\n                        national university rankings in an international context: the case of the\n                        I-UGR rankings of Spanish universities","volume":"101","author":"Robinson-Garc\u00eda","year":"2014","journal-title":"Scientometrics"},{"issue":"12","key":"2021041320051612800_bib33","doi-asserted-by":"crossref","first-page":"39","DOI":"10.4236\/jss.2019.712004","article-title":"Critical treatise on university ranking\n                        systems","volume":"7","author":"Sayed","year":"2019","journal-title":"Open Journal of Social Sciences"},{"issue":"3","key":"2021041320051612800_bib34","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1162\/qss_a_00052","article-title":"A longitudinal analysis of university\n                        rankings","volume":"1","author":"Selten","year":"2019","journal-title":"Quantitative Science Studies"},{"issue":"2","key":"2021041320051612800_bib35","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1007\/s11192-016-2065-4","article-title":"Correlation among top 100 universities\n                        in the major six global rankings: Policy implications","volume":"109","author":"Shehatta","year":"2016","journal-title":"Scientometrics"},{"issue":"15","key":"2021041320051612800_bib36","first-page":"95","article-title":"A comparative and empirical study of\n                        world university ranking system based on ARWU (in Chinese)","volume":"61","author":"Shi","year":"2017","journal-title":"Library and Information Service"},{"key":"2021041320051612800_bib37","volume-title":"Mathematical models in the social\n                        sciences","author":"Snell","year":"1962"},{"issue":"4","key":"2021041320051612800_bib38","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1080\/21568235.2012.662837","article-title":"World university rankings: Take with a large pinch of\n                        salt","volume":"1","author":"Soh","year":"2011","journal-title":"European Journal of Higher Education"},{"issue":"3","key":"2021041320051612800_bib39","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0193762","article-title":"Are university rankings useful to\n                        improve research? A systematic review","volume":"13","author":"Vernon","year":"2018","journal-title":"PLOS\n                        ONE"},{"issue":"1","key":"2021041320051612800_bib40","doi-asserted-by":"crossref","first-page":"013101","DOI":"10.1063\/1.5129036","article-title":"Measuring similarity in co-occurrence\n                        data using ego-networks","volume":"30","author":"Wang","year":"2020","journal-title":"Chaos: An Interdisciplinary\n                        Journal of Nonlinear Science"},{"issue":"4","key":"2021041320051612800_bib41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1852102.1852106","article-title":"A similarity measure for indefinite\n                        rankings","volume":"28","author":"Webber","year":"2010","journal-title":"ACM Transactions on Information\n                        Systems"},{"issue":"4","key":"2021041320051612800_bib42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2478\/jdis-2019-0017","article-title":"Infrastructure of scientometrics: The\n                        big and network picture","volume":"4","author":"Wu","year":"2019","journal-title":"Journal of Data and\n                        Information Science"},{"issue":"1","key":"2021041320051612800_bib43","article-title":"On comprehensive world university\n                        ranking based on ranking aggregation (in Chinese)","volume":"2011","author":"Wu","year":"2019","journal-title":"Journal of Higher Education Research"},{"issue":"7744","key":"2021041320051612800_bib44","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1038\/s41586-019-0941-9","article-title":"Large teams develop and small teams disrupt science and\n                        technology","volume":"566","author":"Wu","year":"2019","journal-title":"Nature"},{"key":"2021041320051612800_bib45","first-page":"1","article-title":"Graph-based rank aggregation method for\n                        high-dimensional and partial rankings","author":"Xiao","year":"2019","journal-title":"Journal of\n                        the Operational Research Society"},{"issue":"7","key":"2021041320051612800_bib46","doi-asserted-by":"crossref","first-page":"556","DOI":"10.1002\/nav.21771","article-title":"Comparison of rank aggregation methods\n                        based on inherent ability","volume":"64","author":"Xiao","year":"2017","journal-title":"Naval Research\n                        Logistics"},{"issue":"1","key":"2021041320051612800_bib47","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.joi.2017.11.002","article-title":"Measuring the inefficiency of Chinese\n                        research universities based on a two-stage network DEA\n                    model","volume":"12","author":"Yang","year":"2018","journal-title":"Journal of Informetrics"},{"issue":"2","key":"2021041320051612800_bib48","first-page":"8","article-title":"Correlation analysis on domestic and\n                        foreign university rankings: An empirical study of China\u2019s forty\n                        universities\u2019 sequence in the four major rankings (in\n                        Chinese)","volume":"2011","author":"Zhang","year":"2011","journal-title":"China Agricultural Education"},{"key":"2021041320051612800_bib49","first-page":"44","article-title":"The influence of university ranking on\n                        students\u2019 choice of schools and university enrollment\u2014taking\n                        the university rankings of US News and World Report as an example (in\n                        Chinese)","volume":"5","author":"Zhang","year":"2011","journal-title":"Higher Education Exploration"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/qss\/article-pdf\/2\/1\/335\/1906566\/qss_a_00101.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/qss\/article-pdf\/2\/1\/335\/1906566\/qss_a_00101.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,4,15]],"date-time":"2021-04-15T00:27:55Z","timestamp":1618446475000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/2\/1\/335\/97568\/The-rank-boost-by-inconsistency-in-university"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":49,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,4,8]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00101","relation":{},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021]]},"published":{"date-parts":[[2021]]}}}