{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T04:38:48Z","timestamp":1768970328655,"version":"3.49.0"},"reference-count":48,"publisher":"Emerald","issue":"4","license":[{"start":{"date-parts":[[2011,7,26]],"date-time":"2011-07-26T00:00:00Z","timestamp":1311638400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2011,7,26]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-heading\">Purpose<\/jats:title><jats:p>Several bibliometric indicators that are extensively used to estimate research performance have not been validated against an external criterion of excellence. This paper aims to investigate whether this validation is possible using the number of Nobel Prize awards.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title><jats:p>This study uses several analytical treatments of the data to investigate: whether Nobel Prize awards are sporadic events or they depend on the scientific activity of countries or institutions and can be used in research evaluation; and the association between the number of Nobel Prize achievements and conventional bibliometric indicators across countries and institutions.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Findings<\/jats:title><jats:p>This study finds that conventional bibliometric indicators, numbers of publications, citations, and top 1 per cent most cited publications, correlate with the number of Nobel Prize achievements in several advanced countries with similar research abilities. Contrarily, in countries and institutions with more variable research characteristics, there is no association between conventional bibliometric indicators and the number of Nobel Prize achievements, and their use as indicators of research excellence is not valid. In contrast, the number of national articles in <jats:italic>Nature<\/jats:italic> and <jats:italic>Science<\/jats:italic> correlates with the number of Nobel Prize achievements across countries and institutions.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Practical implications<\/jats:title><jats:p>Science administrators implementing research evaluations and research incentives based on conventional bibliometric indicators should consider that increasing the scores of these indicators does not imply an improvement in research excellence.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title><jats:p>The study demonstrates that Nobel Prize achievements are not singular events that occur by chance. Therefore, the number of Nobel Prize achievements can be used to validate bibliometric indicators.<\/jats:p><\/jats:sec>","DOI":"10.1108\/00220411111145007","type":"journal-article","created":{"date-parts":[[2011,7,25]],"date-time":"2011-07-25T10:59:56Z","timestamp":1311591596000},"page":"582-600","source":"Crossref","is-referenced-by-count":32,"title":["Measuring research excellence"],"prefix":"10.1108","volume":"67","author":[{"given":"Alonso","family":"Rodr\u00edguez\u2010Navarro","sequence":"first","affiliation":[]}],"member":"140","reference":[{"key":"key2022012320353745200_b1","doi-asserted-by":"crossref","unstructured":"Abt, H.A. (2000), \u201cDo important papers produce high citation counts?\u201d, Scientometrics, Vol. 48 No. 1, pp. 65\u201070.","DOI":"10.1023\/A:1005680318379"},{"key":"key2022012320353745200_b2","doi-asserted-by":"crossref","unstructured":"Adams, J. (2009), \u201cThe use of bibliometrics to measure research quality in UK higher education institutions\u201d, Archivum Immunologiae et Therapiae Experimentalis, Vol. 57 No. 1, pp. 19\u201032.","DOI":"10.1007\/s00005-009-0003-3"},{"key":"key2022012320353745200_b3","doi-asserted-by":"crossref","unstructured":"Adams, J., Gurney, K. and Marshall, S. (2007), \u201cProfiling citation impact: a new methodology\u201d, Scientometrics, Vol. 72 No. 2, pp. 325\u201044.","DOI":"10.1007\/s11192-007-1696-x"},{"key":"key2022012320353745200_b4","doi-asserted-by":"crossref","unstructured":"Aksnes, D.W. (2003), \u201cCharacteristics of highly cited papers\u201d, Research Evaluation, Vol. 12 No. 3, pp. 159\u201070.","DOI":"10.3152\/147154403781776645"},{"key":"key2022012320353745200_b6","doi-asserted-by":"crossref","unstructured":"Aksnes, D.W. and Rip, A. (2009), \u201cResearchers' perceptions of citations\u201d, Research Policy, Vol. 38, pp. 895\u2010905.","DOI":"10.1016\/j.respol.2009.02.001"},{"key":"key2022012320353745200_b5","doi-asserted-by":"crossref","unstructured":"Aksnes, D.W. and Sivertsen, G. (2004), \u201cThe effect of highly cited papers on national citations indicators\u201d, Scientometrics, Vol. 59 No. 2, pp. 213\u201024.","DOI":"10.1023\/B:SCIE.0000018529.58334.eb"},{"key":"key2022012320353745200_b7","doi-asserted-by":"crossref","unstructured":"Almeida, J.A.S., Pais, A.A.C.C. and Formosinho, S.J. (2009), \u201cScience indicators and science patterns in Europe\u201d, Journal of Informetrics, Vol. 3, pp. 134\u201042.","DOI":"10.1016\/j.joi.2009.01.001"},{"key":"key2022012320353745200_b8","doi-asserted-by":"crossref","unstructured":"Archambault, E. and Larivi\u00e8re, V. (2009), \u201cHistory of the journal impact factor: contingencies and consequences\u201d, Scientometrics, Vol. 79, pp. 635\u201049.","DOI":"10.1007\/s11192-007-2036-x"},{"key":"key2022012320353745200_b9","doi-asserted-by":"crossref","unstructured":"Basu, A. (2006), \u201cUsing ISI's \u2018Highly Cited Researchers\u2019 to obtain a country level indicator of citation excellence\u201d, Scientometrics, Vol. 68 No. 3, pp. 361\u201075.","DOI":"10.1007\/s11192-006-0117-x"},{"key":"key2022012320353745200_b10","doi-asserted-by":"crossref","unstructured":"Batty, M. (2003), \u201cThe geography of science citation\u201d, Environment and Planning A, Vol. 35, pp. 761\u20105.","DOI":"10.1068\/a3505com"},{"key":"key2022012320353745200_b11","doi-asserted-by":"crossref","unstructured":"Boero, F. (2008), \u201cItaly: restoring axed funds is not enough to cure all ills\u201d, Nature, Vol. 456, p. 166.","DOI":"10.1038\/456166b"},{"key":"key2022012320353745200_b12","doi-asserted-by":"crossref","unstructured":"Bornmann, L.R. and Daniel, H.D. (2008), \u201cWhat do citation counts measure? A review of studies on citing behavior\u201d, Journal of Documentation, Vol. 64 No. 1, pp. 45\u201080.","DOI":"10.1108\/00220410810844150"},{"key":"key2022012320353745200_b13","doi-asserted-by":"crossref","unstructured":"Bornmann, L., Mutz, R., Neuhaus, C. and Daniel, H.D. (2008), \u201cCitation counts for research evaluations: standards of good practice for analyzing bibliometric data and presenting and interpreting results\u201d, Ethics in Science and Environmental Politics, Vol. 8, pp. 93\u2010102.","DOI":"10.3354\/esep00084"},{"key":"key2022012320353745200_b14","doi-asserted-by":"crossref","unstructured":"Butler, L. (2003), \u201cExplaining Australia's increased share of ISI publications \u2013 the effects of a funding formula based on publication counts\u201d, Research Policy, Vol. 32, pp. 143\u201055.","DOI":"10.1016\/S0048-7333(02)00007-0"},{"key":"key2022012320353745200_b15","doi-asserted-by":"crossref","unstructured":"Campanario, J.M. (1996), \u201cUsing citation classics to study the incidence of serendipity in scientific discovery\u201d, Scientometrics, Vol. 37 No. 1, pp. 3\u201024.","DOI":"10.1007\/BF02093482"},{"key":"key2022012320353745200_b16","doi-asserted-by":"crossref","unstructured":"Charlton, B.G. (2007a), \u201cScientometric identification of elite \u2018revolutionary science\u2019 research institutions by analysis of trends in Nobel Prizes 1947\u20102006\u201d, Medical Hypotheses, Vol. 68, pp. 931\u20104.","DOI":"10.1016\/j.mehy.2006.12.006"},{"key":"key2022012320353745200_b17","doi-asserted-by":"crossref","unstructured":"Charlton, B.G. (2007b), \u201cWhich are the best nations and institutions for revolutionary science 1987\u20102006? Analysis using a combined metric of Nobel Prizes, Fields medals, Lasker awards and Turing awards (NFLT metric)\u201d, Medical Hypotheses, Vol. 68, pp. 1191\u20104.","DOI":"10.1016\/j.mehy.2006.12.007"},{"key":"key2022012320353745200_b18","doi-asserted-by":"crossref","unstructured":"Charlton, B.G. (2007c), \u201cMeasuring revolutionary science 1992\u20102006 using Nobel Prizes, Lasker (clinical medicine) awards and Gairdner awards (NLG metric)\u201d, Medical Hypotheses, Vol. 69, pp. 1\u20105.","DOI":"10.1016\/j.mehy.2007.01.001"},{"key":"key2022012320353745200_b19","unstructured":"Colquhoun, D. (2007), \u201cHow to get good science\u201d, Physiology News, Vol. 69, pp. 12\u201014."},{"key":"key2022012320353745200_b20","doi-asserted-by":"crossref","unstructured":"Dosi, G., Llerena, P. and Labini, M.S. (2006), \u201cThe relationships between science, technologies and their industrial exploitation: an illustration through the myths and realities of the so\u2010called \u2018European Paradox\u2019\u201d, Research Policy, Vol. 35, pp. 1450\u201064.","DOI":"10.1016\/j.respol.2006.09.012"},{"key":"key2022012320353745200_b21","unstructured":"European Commission (2008), \u201cA more research\u2010intensive and integrated European Research Area\u201d, Science, Technology and Competitiveness, Key Figures Report 2008\/2009, European Commission, Luxembourg."},{"key":"key2022012320353745200_b22","unstructured":"Garfield, E. (1977), \u201cThe 250 most\u2010cited primary authors, 1961\u20101975. Part II. Correlation between citedness, Nobel Prizes, and academic memberships\u201d, Essays of an Information Scientist, Vol. 3, pp. 337\u201047."},{"key":"key2022012320353745200_b23","unstructured":"Garfield, E. (1986), \u201cDo Nobel\u2010Prize winners write citation classics?\u201d, Essays of an Information Scientist, Vol. 9, pp. 182\u20107."},{"key":"key2022012320353745200_b24","doi-asserted-by":"crossref","unstructured":"Garfield, E. and Welljams\u2010Dorof, A. (1992), \u201cOf Nobel class: a citation perspective of high impact research authors\u201d, Theoretical Medicine, Vol. 13, pp. 117\u201035.","DOI":"10.1007\/BF02163625"},{"key":"key2022012320353745200_b25","doi-asserted-by":"crossref","unstructured":"Gingras, Y. and Wallace, M.L. (2010), \u201cWhy it has become more difficult to predict Nobel Prize winners: a bibliometric analysis of nominees and winners of the chemistry and physics prizes (1901\u20102007)\u201d, Scientometrics, Vol. 82 No. 2, pp. 401\u201012.","DOI":"10.1007\/s11192-009-0035-9"},{"key":"key2022012320353745200_b26","doi-asserted-by":"crossref","unstructured":"Harnad, S. (2009), \u201cOpen access scientometrics and the UK research assessment exercise\u201d, Scientometrics, Vol. 79 No. 1, pp. 147\u201056.","DOI":"10.1007\/s11192-009-0409-z"},{"key":"key2022012320353745200_b27","doi-asserted-by":"crossref","unstructured":"King, D.A. (2004), \u201cThe scientific impact of nations. What different countries get for their research spending\u201d, Nature, Vol. 430, pp. 311\u20106.","DOI":"10.1038\/430311a"},{"key":"key2022012320353745200_b28","doi-asserted-by":"crossref","unstructured":"Kostoff, R.N. (1998), \u201cThe use and misuse of citation analysis in research evaluation\u201d, Scientometrics, Vol. 43 No. 1, pp. 27\u201043.","DOI":"10.1007\/BF02458392"},{"key":"key2022012320353745200_b29","unstructured":"Kuhn, T. (1970), The Structure of Scientific Revolutions, University of Chicago Press, Chicago, IL."},{"key":"key2022012320353745200_b30","doi-asserted-by":"crossref","unstructured":"Lawrence, P.A. (2003), \u201cThe politics of publication. Authors, reviewers and editors must act to protect the quality of research\u201d, Nature, Vol. 422, pp. 259\u201061.","DOI":"10.1038\/422259a"},{"key":"key2022012320353745200_b31","doi-asserted-by":"crossref","unstructured":"Lawrence, P.A. (2009), \u201cReal lives and white lies in the funding of scientific research\u201d, PLoS Biology, Vol. 7 No. 9, e1000197.","DOI":"10.1371\/journal.pbio.1000197"},{"key":"key2022012320353745200_b32","unstructured":"Leydesdorff, L. (2005), \u201cEvaluation of research and evolution of science indicators\u201d, Current Science, Vol. 89 No. 9, pp. 1510\u20107."},{"key":"key2022012320353745200_b33","doi-asserted-by":"crossref","unstructured":"May, R.M. (1997), \u201cThe scientific wealth of nations\u201d, Science, Vol. 275, pp. 793\u20106.","DOI":"10.1126\/science.275.5301.793"},{"key":"key2022012320353745200_b34","unstructured":"Moed, H.F. (2005), Citation Analysis in Research Evaluation, Information Science and Knowledge Management, Vol. 9, Springer, Dordrecht."},{"key":"key2022012320353745200_b35","doi-asserted-by":"crossref","unstructured":"Moed, H.F. (2008), \u201cUK Research assessment exercises: informed judgments on research quality or quantity?\u201d, Scientometrics, Vol. 74 No. 1, pp. 153\u201061.","DOI":"10.1007\/s11192-008-0108-1"},{"key":"key2022012320353745200_b36","doi-asserted-by":"crossref","unstructured":"Nicolaisen, J. (2007), \u201cCitation analysis\u201d, Annual Review of Information Science and Technology, Vol. 41, pp. 609\u201041.","DOI":"10.1002\/aris.2007.1440410120"},{"key":"key2022012320353745200_b37","doi-asserted-by":"crossref","unstructured":"Notkins, A.L. (2008), \u201cNeutralizing the impact factor culture\u201d, Science, Vol. 322, p. 191.","DOI":"10.1126\/science.322.5899.191a"},{"key":"key2022012320353745200_b38","unstructured":"OECD (2007), OECD Science, Technology and Industry: Scoreboard, OECD, Paris."},{"key":"key2022012320353745200_b39","doi-asserted-by":"crossref","unstructured":"Parnas, D.T. (2007), \u201cStop the numbers game: counting papers slow down the rate of scientific progress\u201d, Communications of the ACM, Vol. 50, pp. 19\u201021.","DOI":"10.1145\/1297797.1297815"},{"key":"key2022012320353745200_b40","doi-asserted-by":"crossref","unstructured":"Phelan, T.J. (1999), \u201cA compendium of issues for citation analysis\u201d, Scientometrics, Vol. 45 No. 1, pp. 117\u201036.","DOI":"10.1007\/BF02458472"},{"key":"key2022012320353745200_b41","doi-asserted-by":"crossref","unstructured":"Rodr\u00edguez\u2010Navarro, A. (2009), \u201cSound research, unimportant discoveries: research, universities, and formal evaluation of research in Spain\u201d, Journal of the American Society for Information Science and Technology, Vol. 60 No. 9, pp. 1845\u201058.","DOI":"10.1002\/asi.21104"},{"key":"key2022012320353745200_b42","doi-asserted-by":"crossref","unstructured":"Root\u2010Bernstein, R.S. (1989), \u201cStrategies of research\u201d, Research Technology Management, Vol. 32 No. 3, pp. 36\u201041.","DOI":"10.1080\/08956308.1989.11670599"},{"key":"key2022012320353745200_b43","doi-asserted-by":"crossref","unstructured":"Seglen, P.O. (1992), \u201cThe skewness of science\u201d, Journal of the American Society of Information Sciences, Vol. 43, pp. 628\u201038.","DOI":"10.1002\/(SICI)1097-4571(199210)43:9<628::AID-ASI5>3.0.CO;2-0"},{"key":"key2022012320353745200_b44","doi-asserted-by":"crossref","unstructured":"Shelton, R.D. and Holdridge, G.M. (2004), \u201cThe US\u2010EU race for leadership of science and technology: qualitative and quantitative indicators\u201d, Scientometrics, Vol. 60 No. 3, pp. 353\u201063.","DOI":"10.1023\/B:SCIE.0000034379.75236.d9"},{"key":"key2022012320353745200_b46","doi-asserted-by":"crossref","unstructured":"Simons, K. (2008), \u201cThe misused impact factor\u201d, Science, Vol. 322, p. 165.","DOI":"10.1126\/science.1165316"},{"key":"key2022012320353745200_b45","doi-asserted-by":"crossref","unstructured":"Steele, C., Butler, L. and Kingsley, D. (2006), \u201cThe publishing imperative: the pervasive influence of publication metrics\u201d, Learned Publishing, Vol. 19, pp. 277\u201090.","DOI":"10.1087\/095315106778690751"},{"key":"key2022012320353745200_b47","unstructured":"Van Raan, A.F.J. (2000), \u201cThe Pandora's box of citation analysis: measuring scientific excellence \u2013 the last evil?\u201d, in Cronin, B. and Atkins, H.B. (Eds), The Web of Knowledge \u2013 A Festschrift in honor of Eugene Garfield, ASIS Monograph Series, Information Today Inc., Medford, NJ, pp. 251\u201077."},{"key":"key2022012320353745200_frg48","unstructured":"Van Leeuwen, T.N., Visser, M.S., Moed, H.F., Nederhof, T.J. and Van Raan, A.F.J. (2003), \u201cThe Holy Grail of science policy: exploring and combining bibliometric tools in search of scientific excellence\u201d, Scientometrics, Vol. 57 No. 2, pp. 257\u201080."}],"container-title":["Journal of Documentation"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/00220411111145007","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220411111145007\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00220411111145007\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T23:38:21Z","timestamp":1753400301000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jd\/article\/67\/4\/582-600\/210784"}},"subtitle":["Number of Nobel Prize achievements versus conventional bibliometric indicators"],"short-title":[],"issued":{"date-parts":[[2011,7,26]]},"references-count":48,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2011,7,26]]}},"alternative-id":["10.1108\/00220411111145007"],"URL":"https:\/\/doi.org\/10.1108\/00220411111145007","relation":{},"ISSN":["0022-0418"],"issn-type":[{"value":"0022-0418","type":"print"}],"subject":[],"published":{"date-parts":[[2011,7,26]]}}}