{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T04:27:27Z","timestamp":1776486447755,"version":"3.51.2"},"reference-count":49,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2018,2,7]],"date-time":"2018-02-07T00:00:00Z","timestamp":1517961600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2018,4]]},"DOI":"10.1007\/s11192-018-2666-1","type":"journal-article","created":{"date-parts":[[2018,2,7]],"date-time":"2018-02-07T13:51:08Z","timestamp":1518011468000},"page":"585-606","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":114,"title":["University rankings: What do they really show?"],"prefix":"10.1007","volume":"115","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1607-1810","authenticated-orcid":false,"given":"Jill","family":"Johnes","sequence":"first","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2018,2,7]]},"reference":[{"key":"2666_CR1","unstructured":"Allcock, D., Johnes, J., & Virmani, S. (2017). Efficiency and VC pay: Exploring the value conundrum. In European workshop on efficiency and productivity analysis, London, 13th\u201315th June."},{"key":"2666_CR2","doi-asserted-by":"publisher","DOI":"10.1080\/03075079.2015.1019450","author":"R Bachan","year":"2015","unstructured":"Bachan, R. (2015). Grade inflation in UK higher education. Studies in Higher Education. https:\/\/doi.org\/10.1080\/03075079.2015.1019450 .","journal-title":"Studies in Higher Education"},{"key":"2666_CR3","volume-title":"Peeling the DEA onion: Layering and rank-ordering DMUs using tiered DEA","author":"RS Barr","year":"2000","unstructured":"Barr, R. S., Durchholz, M. L., & Seiford, L. (2000). Peeling the DEA onion: Layering and rank-ordering DMUs using tiered DEA. Dallas, TX: Southern Methodist University Technical Report."},{"key":"2666_CR4","unstructured":"Bekhradnia, B. (2016). International university rankings: For good or ill? HEPI report. http:\/\/www.hepi.ac.uk\/wp-content\/uploads\/2016\/12\/Hepi_International-university-rankings-For-good-or-for-ill-REPORT-89-10_12_16_Screen.pdfHigherEducationPolicyInstitute ."},{"key":"2666_CR5","doi-asserted-by":"publisher","DOI":"10.14293\/S2199-1006.1.SOR-EDU.AETBZC.v1","author":"A Boring","year":"2016","unstructured":"Boring, A., Ottoboni, K., & Stark, P. B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. ScienceOpen Research. https:\/\/doi.org\/10.14293\/S2199-1006.1.SOR-EDU.AETBZC.v1 .","journal-title":"ScienceOpen Research"},{"issue":"1","key":"2666_CR6","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1007\/s10479-006-0039-2","volume":"145","author":"M-L Bougnol","year":"2006","unstructured":"Bougnol, M.-L., & Dula, J. H. (2006). Validating DEA as a ranking tool: An application of DEA to assess performance in higher education. Annals of Operations Research, 145(1), 339\u2013365.","journal-title":"Annals of Operations Research"},{"key":"2666_CR7","doi-asserted-by":"crossref","DOI":"10.5962\/bhl.title.979","volume-title":"American men of science: A biographical dictionary","author":"JM Cattell","year":"1906","unstructured":"Cattell, J. M. (1906). American men of science: A biographical dictionary. New York, NY: Science Press."},{"issue":"4","key":"2666_CR8","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1016\/0377-2217(78)90138-8","volume":"2","author":"A Charnes","year":"1978","unstructured":"Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2(4), 429\u2013444.","journal-title":"European Journal of Operational Research"},{"issue":"1","key":"2666_CR9","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1007\/s11205-006-9029-7","volume":"82","author":"L Cherchye","year":"2007","unstructured":"Cherchye, L., Moesen, W., Rogge, N., & Van Puyenbroeck, T. (2007). An introduction to \u2018benefit of the doubt\u2019 composite indicators. Social Indicators Research, 82(1), 111\u2013145.","journal-title":"Social Indicators Research"},{"issue":"3\u20134","key":"2666_CR10","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1016\/j.jpubeco.2011.10.005","volume":"96","author":"G Fraja De","year":"2012","unstructured":"De Fraja, G., & Valbonesi, P. (2012). The design of the university system. Journal of Public Economics, 96(3\u20134), 317\u2013330.","journal-title":"Journal of Public Economics"},{"key":"2666_CR11","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1163\/9789087908164_008","volume-title":"University rankings, diversity, and the new landscape of higher education","author":"DD Dill","year":"2009","unstructured":"Dill, D. D. (2009). Convergence and diversity: The role and influence of university ranking. In B. M. Kehm & B. Stensaker (Eds.), University rankings, diversity, and the new landscape of higher education (pp. 97\u2013116). Rotterdam: Sense Publishers."},{"issue":"4","key":"2666_CR12","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1007\/s10734-004-1746-8","volume":"49","author":"DD Dill","year":"2005","unstructured":"Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national analysis of university ranking systems. Higher Education, 49(4), 495\u2013533.","journal-title":"Higher Education"},{"key":"2666_CR13","volume-title":"Careers: More means better","author":"M Dixon","year":"1976","unstructured":"Dixon, M. (1976). Careers: More means better. London: The Financial Times."},{"key":"2666_CR14","volume-title":"Jobs column: What happened to universities\u2019 graduates?","author":"M Dixon","year":"1985","unstructured":"Dixon, M. (1985). Jobs column: What happened to universities\u2019 graduates?. London: The Financial Times."},{"key":"2666_CR15","volume-title":"Jobs column: Benefits, and risks, of trying for a degree","author":"M Dixon","year":"1989","unstructured":"Dixon, M. (1989). Jobs column: Benefits, and risks, of trying for a degree. London: The Financial Times."},{"issue":"1","key":"2666_CR16","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1257\/jep.26.1.193","volume":"26","author":"RG Ehrenberg","year":"2012","unstructured":"Ehrenberg, R. G. (2012). American higher education in transition. Journal of Economic Perspectives, 26(1), 193\u2013216.","journal-title":"Journal of Economic Perspectives"},{"key":"2666_CR17","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1007\/978-94-007-1116-7_3","volume-title":"University rankings: Theoretical basis, methodology and impacts on global higher education","author":"G Harman","year":"2011","unstructured":"Harman, G. (2011). Competitors of rankings: New directions in quality assurance and accountability. In J. C. Shin, R. K. Toutkoushian, & U. Teichler (Eds.), University rankings: Theoretical basis, methodology and impacts on global higher education (pp. 35\u201354). Dordrecht: Springer."},{"issue":"4","key":"2666_CR18","first-page":"6","volume":"2","author":"E Hazelkorn","year":"2015","unstructured":"Hazelkorn, E. (2015). How the geo-politics of rankings is shaping behaviour. Higher Education in Russia and Beyond, 2(4), 6\u20137.","journal-title":"Higher Education in Russia and Beyond"},{"key":"2666_CR19","volume-title":"Counting what is measured or measuring what counts? League tables and their impact on higher education institutions in England","author":"HEFCE","year":"2008","unstructured":"HEFCE. (2008). Counting what is measured or measuring what counts? League tables and their impact on higher education institutions in England. Bristol: Higher Education Funding Council for England."},{"key":"2666_CR20","volume-title":"Collaborations, alliances and mergers in higher education: Consultation on lessons learned and guidance for institutions","author":"HEFCE","year":"2012","unstructured":"HEFCE. (2012). Collaborations, alliances and mergers in higher education: Consultation on lessons learned and guidance for institutions. London: HEFCE 2012\/06 Higher Education Funding Council for England."},{"issue":"2","key":"2666_CR21","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1016\/j.respol.2011.09.007","volume":"41","author":"D Hicks","year":"2012","unstructured":"Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251.","journal-title":"Research Policy"},{"issue":"6","key":"2666_CR22","doi-asserted-by":"publisher","first-page":"419","DOI":"10.3152\/030234209X461006","volume":"36","author":"L Himanen","year":"2009","unstructured":"Himanen, L., Auranen, O., Puuska, H.-M., & Nieminen, M. (2009). Influence of research funding and science policy on university research performance: A comparison of five countries. Science and Public Policy, 36(6), 419\u2013430.","journal-title":"Science and Public Policy"},{"key":"2666_CR23","volume-title":"A study of the graduate schools of America","author":"RM Hughes","year":"1925","unstructured":"Hughes, R. M. (1925). A study of the graduate schools of America. Oxford, OH: Miami University Press."},{"issue":"2","key":"2666_CR24","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1093\/oxrep\/8.2.19","volume":"8","author":"G Johnes","year":"1992","unstructured":"Johnes, G. (1992). Performance indicators in higher education: A survey of recent work. Oxford Review of Economic Policy, 8(2), 19\u201334.","journal-title":"Oxford Review of Economic Policy"},{"key":"2666_CR25","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/S0377-2217(96)90048-X","volume":"89","author":"J Johnes","year":"1996","unstructured":"Johnes, J. (1996). Performance assessment in higher education in Britain. European Journal of Operational Research, 89, 18\u201333.","journal-title":"European Journal of Operational Research"},{"key":"2666_CR26","doi-asserted-by":"crossref","first-page":"462","DOI":"10.4337\/9781845421694","volume-title":"International handbook on the economics of education","author":"G Johnes","year":"2004","unstructured":"Johnes, G. (2004). Standards and grade inflation. In G. Johnes & J. Johnes (Eds.), International handbook on the economics of education (pp. 462\u2013483). Cheltenham: Edward Elgar."},{"issue":"3","key":"2666_CR27","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1016\/j.ejor.2014.10.043","volume":"243","author":"J Johnes","year":"2015","unstructured":"Johnes, J. (2015). Operational research in education. European Journal of Operational Research, 243(3), 683\u2013696.","journal-title":"European Journal of Operational Research"},{"key":"2666_CR28","volume-title":"Valuing higher education: An appreciation of the work of Gareth Williams","author":"J Johnes","year":"2016","unstructured":"Johnes, J. (2016). Performance indicators and rankings in higher education. In R. Barnett, P. Temple, & P. Scott (Eds.), Valuing higher education: An appreciation of the work of Gareth Williams. London: UCL Institute of Education Press."},{"key":"2666_CR29","doi-asserted-by":"publisher","DOI":"10.1111\/manc.12138","author":"G Johnes","year":"2015","unstructured":"Johnes, G., & Soo, K. T. (2015). Grades across universities over time. The Manchester School. https:\/\/doi.org\/10.1111\/manc.12138 .","journal-title":"The Manchester School"},{"key":"2666_CR30","unstructured":"Jump, P. (2014). Cut 50% of universities and bar undergraduates from Oxbridge. Times Higher Education. London, Times Supplements Ltd 25th June 2014."},{"issue":"1","key":"2666_CR31","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1111\/ejed.12064","volume":"49","author":"BM Kehm","year":"2014","unstructured":"Kehm, B. M. (2014). Global university rankings\u2014impacts and unintended side effects. European Journal of Education, 49(1), 102\u2013112.","journal-title":"European Journal of Education"},{"issue":"5\u20136","key":"2666_CR32","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1016\/j.jpubeco.2010.02.002","volume":"94","author":"S Kelchtermans","year":"2010","unstructured":"Kelchtermans, S., & Verboven, F. (2010). Program duplication in higher education is not necessarily bad. Journal of Public Economics, 94(5\u20136), 397\u2013409.","journal-title":"Journal of Public Economics"},{"key":"2666_CR33","volume-title":"Counting what is measured or measuring what counts? League tables and their impact on higher education institutions in England","author":"W Locke","year":"2008","unstructured":"Locke, W., Verbik, L., Richardson, J. T. E., & Roger, K. (2008). Counting what is measured or measuring what counts? League tables and their impact on higher education institutions in England. Bristol: Higher Education Funding Council for England."},{"key":"2666_CR34","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1007\/978-94-007-1116-7_5","volume-title":"University rankings: Theoretical basis, methodology and impacts on global higher education","author":"B Longden","year":"2011","unstructured":"Longden, B. (2011). Ranking indicators and weights. In J. C. Shin, R. K. Toutkoushian, & U. Teichler (Eds.), University rankings: Theoretical basis, methodology and impacts on global higher education (pp. 73\u2013104). Dordrecht: Springer."},{"issue":"1","key":"2666_CR35","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1111\/ejed.12061","volume":"49","author":"S Marginson","year":"2014","unstructured":"Marginson, S. (2014). University rankings and social science. European Journal of Education, 49(1), 45\u201359.","journal-title":"European Journal of Education"},{"issue":"3\u20134","key":"2666_CR36","doi-asserted-by":"publisher","first-page":"306","DOI":"10.1177\/1028315307303544","volume":"11","author":"S Marginson","year":"2007","unstructured":"Marginson, S., & Wende, V. D. M. (2007). To rank or to be ranked: The impact of global rankings in higher education. Journal of Studies in International Education, 11(3\u20134), 306\u2013329.","journal-title":"Journal of Studies in International Education"},{"key":"2666_CR37","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1007\/978-94-007-1116-7_10","volume-title":"University rankings: Theoretical basis, methodology and impacts on global higher education","author":"CC Morphew","year":"2011","unstructured":"Morphew, C. C., & Swanson, C. (2011). On the efficacy of raising your university\u2019s ranking. In J. C. Shin, R. K. Toutkoushian, & U. Teichler (Eds.), University rankings: Theoretical basis, methodology and impacts on global higher education (pp. 185\u2013200). Dordrecht: Springer."},{"key":"2666_CR38","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1016\/j.ijedudev.2016.11.004","volume":"52","author":"SM Muller","year":"2017","unstructured":"Muller, S. M. (2017). Academics as rent seekers: Distorted incentives in higher education, with reference to the South African case. International Journal of Educational Development, 52, 58\u201367.","journal-title":"International Journal of Educational Development"},{"key":"2666_CR39","unstructured":"Newman, M. (2008). Students urged to inflate national survey marks to improve job options. Times Higher Education London, Times Supplements Ltd. 15th May 2008: 7 15th May 2008."},{"key":"2666_CR40","volume-title":"How should we measure higher education? A fundamental review of the Performance Indicators, Part 2: The evidence report","author":"E Pollard","year":"2013","unstructured":"Pollard, E., Williams, M., Williams, J., Bertram, C., & Buzzeo, J. (2013). How should we measure higher education? A fundamental review of the Performance Indicators, Part 2: The evidence report. Brighton: Institute for Employment Studies."},{"issue":"3","key":"2666_CR41","doi-asserted-by":"publisher","first-page":"1764","DOI":"10.1111\/j.1465-7295.2012.00491.x","volume":"51","author":"SV Popov","year":"2013","unstructured":"Popov, S. V., & Bernhardt, D. (2013). University competition, grading standards, and grade inflation. Economic Inquiry, 51(3), 1764\u20131778.","journal-title":"Economic Inquiry"},{"issue":"1","key":"2666_CR42","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1111\/1468-2273.00233","volume":"57","author":"H Rolfe","year":"2003","unstructured":"Rolfe, H. (2003). University strategy in an age of uncertainty: The effect of higher education funding on old and new universities. Higher Education Quarterly, 57(1), 24\u201347.","journal-title":"Higher Education Quarterly"},{"issue":"1","key":"2666_CR43","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1016\/j.respol.2010.09.003","volume":"40","author":"M Saisana","year":"2011","unstructured":"Saisana, M., d\u2019Hombres, B., & Saltelli, A. (2011). Rickety numbers: Volatility of university rankings and policy implications. Research Policy, 40(1), 165\u2013177.","journal-title":"Research Policy"},{"key":"2666_CR44","volume-title":"Handbook on constructing composite indicators: Methodology and user guide","author":"A Saltelli","year":"2005","unstructured":"Saltelli, A., Nardo, M., Tarantola, S., Giovannini, E., Hoffman, A., & Saisana, M. (2005). Handbook on constructing composite indicators: Methodology and user guide. Paris: OECD."},{"key":"2666_CR45","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-94-007-1116-7","volume-title":"University rankings: Theoretical basis, methodology and impacts on global higher education","author":"JC Shin","year":"2011","unstructured":"Shin, J. C., & Toutkoushian, R. K. (2011). The past, present, and future of university rankings. In J. C. Shin, R. K. Toutkoushian, & U. Teichler (Eds.), University rankings: Theoretical basis, methodology and impacts on global higher education (pp. 1\u201318). Dordrecht: Springer."},{"key":"2666_CR46","doi-asserted-by":"publisher","DOI":"10.14293\/S2199-1006.1.SOR-EDU.AOFRQA.v1","author":"PB Stark","year":"2014","unstructured":"Stark, P. B., & Freishtat, R. (2014). An evaluation of course evaluations. ScienceOpen Research. https:\/\/doi.org\/10.14293\/S2199-1006.1.SOR-EDU.AOFRQA.v1 .","journal-title":"ScienceOpen Research"},{"key":"2666_CR47","first-page":"3","volume-title":"University rankings, diversity, and the new landscape of higher education","author":"A Usher","year":"2009","unstructured":"Usher, A., & Medow, J. (2009). A global survey of university rankings and league tables. In B. M. Kehm & B. Stensaker (Eds.), University rankings, diversity, and the new landscape of higher education (pp. 3\u201318). Rotterdam: Sense Publishers."},{"key":"2666_CR48","first-page":"92","volume-title":"The state of public management","author":"DL Weimer","year":"1996","unstructured":"Weimer, D. L., & Vining, A. R. (1996). Economics. In D. F. Kettle & H. B. Milward (Eds.), The state of public management (pp. 92\u2013117). Baltimore, MA: Johns Hopkins University."},{"issue":"2","key":"2666_CR49","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1108\/09684889710165107","volume":"5","author":"M Yorke","year":"1997","unstructured":"Yorke, M. (1997). A good league table guide? Quality Assurance in Education, 5(2), 61\u201372.","journal-title":"Quality Assurance in Education"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11192-018-2666-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-018-2666-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-018-2666-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,31]],"date-time":"2023-08-31T22:05:53Z","timestamp":1693519553000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11192-018-2666-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,2,7]]},"references-count":49,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2018,4]]}},"alternative-id":["2666"],"URL":"https:\/\/doi.org\/10.1007\/s11192-018-2666-1","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,2,7]]},"assertion":[{"value":"23 October 2017","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 February 2018","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}