{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T22:10:31Z","timestamp":1759961431530,"version":"3.41.2"},"reference-count":47,"publisher":"Emerald","issue":"1\/2","license":[{"start":{"date-parts":[[2003,3,1]],"date-time":"2003-03-01T00:00:00Z","timestamp":1046476800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2003,3,1]]},"abstract":"<jats:p>Biomedical research evaluation has traditionally been based on analysis of outputs and their citations by other papers. However we should try to map the routes by which research actually improves patient care and reduces illness, and develop indicators for them. We must allow for the lengthy time\u2010scales involved and the importance of researchers being physically close to healthcare professionals, whose practice can be improved through international and governmental regulations and through approved guidelines. Each of these will depend on a body of research evidence. We must also evaluate the effects of research on policy makers and the public, who often learn about it through the World Wide Web and through the mass media, particularly newspapers. The latter provide a major bibliometric resource but one that needs to be tapped in individual countries using common standards in order to provide internationally\u2010comparable indicators.<\/jats:p>","DOI":"10.1108\/00012530310462698","type":"journal-article","created":{"date-parts":[[2003,4,9]],"date-time":"2003-04-09T11:25:16Z","timestamp":1049887516000},"page":"32-42","source":"Crossref","is-referenced-by-count":16,"title":["Beyond outputs: new measures of biomedical research impact"],"prefix":"10.1108","volume":"55","author":[{"given":"Grant","family":"Lewison","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"key":"key2022013120341262000_B1","doi-asserted-by":"crossref","unstructured":"Anderson, P. (1999), \u201cAnother media scare about MMR vaccine hits Britain\u201d, BMJ, Vol. 318, 12 June, p. 1578.","DOI":"10.1136\/bmj.318.7198.1578"},{"key":"key2022013120341262000_B2","unstructured":"Atiogbe, P. (2001), \u201cBibliometric analysis of the international Codex Alimentarius food pesticide standards\u201d, MSc dissertation, Department of Information Science, City University, London."},{"key":"key2022013120341262000_B3","unstructured":"Chemistry & Industry (2001), \u201cPolicy \u2013 EU backs the use of stem cells in research\u201d, Chemistry & Industry, Vol. 19, 1 October, p. 601."},{"key":"key2022013120341262000_B4","doi-asserted-by":"crossref","unstructured":"Balaban, A.T. (1996), \u201cHow should citations to articles in high\u2010impact and low\u2010impact journals be evaluated, or what is a citation worth\u201d, Scientometrics, Vol. 37 No. 3, pp. 495\u20108.","DOI":"10.1007\/BF02019260"},{"key":"key2022013120341262000_B5","doi-asserted-by":"crossref","unstructured":"Birchard, K. (1999), \u201cNew European Commissioner for food safety stamps his authority\u201d, The Lancet, Vol. 354, 18 September, p. 1012.","DOI":"10.1016\/S0140-6736(05)76629-0"},{"key":"key2022013120341262000_B6","unstructured":"Boseley, S. (1999), \u201cDoctors throw doubt on safety of GM food\u201d, The Guardian, 18 May, p. 7."},{"key":"key2022013120341262000_B7","doi-asserted-by":"crossref","unstructured":"Bourke, P. and Butler, L. (1996), \u201cPublication types, citation rates and evaluation\u201d, Scientometrics, Vol. 37 No. 3, pp. 473\u201094.","DOI":"10.1007\/BF02019259"},{"key":"key2022013120341262000_B8","unstructured":"Braid, M. (2000), \u201cThe siege of Huntingdon\u201d, The Independent, Review Section, 22 June, p. 1."},{"key":"key2022013120341262000_B9","doi-asserted-by":"crossref","unstructured":"Burton, J.L. and Wells, M. (2001), \u201cThe Alder Hey affair \u2013 implications for pathology practice\u201d, Journal of Clinical Pathology, Vol. 54 No. 11, pp. 820\u20103.","DOI":"10.1136\/jcp.54.11.820"},{"key":"key2022013120341262000_B10","unstructured":"Colborn, T., Dumanoski, D. and Myers, J.P. (1996), Our Stolen Future \u2013 Are We Threatening our Fertility, Intelligence and Survival?, Dutton, New York, NY."},{"key":"key2022013120341262000_B11","doi-asserted-by":"crossref","unstructured":"Doll, R. and Peto, R. (1981), \u201cThe causes of cancer: quantitative estimates of avoidable risks of cancer in the United States today\u201d, Journal of the National Cancer Institute, Vol. 66 No. 6, pp. 1191\u2010308.","DOI":"10.1093\/jnci\/66.6.1192"},{"key":"key2022013120341262000_B12","unstructured":"Dyer, G. (2002), \u201cA shot in the arm for home\u2010grown stem cell research\u201d, Financial Times, 31 August, p. 3."},{"key":"key2022013120341262000_B13","unstructured":"(The) Economist (1999b), \u201cFrankenstein foods \u2013 seeds of discontent\u201d, The Economist, 20 February, No. 8107, pp. 19, 105."},{"key":"key2022013120341262000_B14","doi-asserted-by":"crossref","unstructured":"Finkel, E., Redfearn, J., Kaiser, J. and Vogel, G. (2001), \u201cScienceScope: unclear forecast\u201d, Science, Vol. 291, 2 February, p. 807.","DOI":"10.1126\/science.291.5505.807C"},{"key":"key2022013120341262000_B15","unstructured":"Firn, D. (2000), \u201cGlaxo issues warning over the future of R&D\u201d, Financial Times, 22 February, p. 2."},{"key":"key2022013120341262000_B16","unstructured":"Garfield, E. and Welljams\u2010Dorof, A. (1992), \u201cThe uses and limitations of citation data as science indicators \u2013 an overview for students and nonspecialists, Introduction to citation data \u2013 their use as quantitative indicators for science and technology evaluation and policy making\u201d, Science and Public Policy, Vol. 19, pp. 321\u20107."},{"key":"key2022013120341262000_B17","doi-asserted-by":"crossref","unstructured":"Gough, M. (1989), \u201cEstimating cancer mortality: epidemiological and toxicological methods produce similar assessments\u201d, Environmental Science and Technology, Vol. 23 No. 8, pp. 935\u20100.","DOI":"10.1021\/es00066a602"},{"key":"key2022013120341262000_B18","doi-asserted-by":"crossref","unstructured":"Grant, J., Cottrell, R., Cluzeau, F. and Fawcett, G. (2000), \u201cEvaluating \u2018payback\u2019 on biomedical research from papers cited in clinical guidelines: applied bibliometric study\u201d, BMJ, Vol. 320 No. 7242, 22 April, pp. 1107\u201011.","DOI":"10.1136\/bmj.320.7242.1107"},{"key":"key2022013120341262000_B19","unstructured":"Gregoriadis, L. (1999), \u201cGM foods overtake BSE as top safety concern, says survey\u201d, The Guardian, 4 September, p. 6."},{"key":"key2022013120341262000_B20","doi-asserted-by":"crossref","unstructured":"Hahn, R.W. (1996), \u201cRegulatory reform: what do the government\u2019s numbers tell us?\u201d, in Hahn, R.W. (Ed.), Risks, Costs, and Lives Saved: Getting Better Results from Regulation, Oxford University Press, New York, NY.","DOI":"10.1093\/oso\/9780195211740.001.0001"},{"key":"key2022013120341262000_B21","doi-asserted-by":"crossref","unstructured":"Kmietowicz, Z. (2000), \u201cNICE\u2019s appraisal procedures attacked\u201d, BMJ, Vol. 321, 21 October, p. 980.","DOI":"10.1136\/bmj.321.7274.1432\/c"},{"key":"key2022013120341262000_B22","doi-asserted-by":"crossref","unstructured":"Kondro, W. (2000), \u201cCanadian group calls for more environmental\u2010health studies\u201d, The Lancet, Vol. 356 No. 9233, 9 September, p. 921.","DOI":"10.1016\/S0140-6736(05)73911-8"},{"key":"key2022013120341262000_B23","unstructured":"Konisky, D.M. (1999), Comparative Risk Projects: a Methodology for Cross\u2010project Analysis of Human Health Risk Rankings, Discussion Paper 99\u201046, Resources for the Future, available at: www.rff.org\/disc_papers\/abstracts\/9946.htm"},{"key":"key2022013120341262000_B24","doi-asserted-by":"crossref","unstructured":"Kostoff, R.N. (1998), \u201cThe use and misuse of citation analysis in research evaluation \u2013 comments on theories of citation\u201d, Scientometrics, Vol. 43 No. 1, pp. 325\u201033.","DOI":"10.1007\/BF02458392"},{"key":"key2022013120341262000_B25","doi-asserted-by":"crossref","unstructured":"(The) Lancet (1999), \u201cA nasty start for NICE\u201d, The Lancet, Vol. 354, 16 October, p. 1313.","DOI":"10.1016\/S0140-6736(99)00181-6"},{"key":"key2022013120341262000_B26","unstructured":"Laurance, J. (2002), \u201cMMR scare has led to huge slump in child immunisation\u201d, The Independent, 20 September, p. 2."},{"key":"key2022013120341262000_B27","unstructured":"Lee, J. (1998), \u201cDirecting biomedical research\u201d, The Biologist, Vol. 45 No. 4, p. 147."},{"key":"key2022013120341262000_B28","doi-asserted-by":"crossref","unstructured":"Lehrman, S. (1995), \u201cBallot urged on ageing centre plans\u201d, Nature, Vol. 373, 12 January, p. 92.","DOI":"10.1038\/373092b0"},{"key":"key2022013120341262000_B29","doi-asserted-by":"crossref","unstructured":"Leta, J., Lannes, D. and de Meis, L. (1998), \u201cHuman resources and scientific productivity in Brazil\u201d, Scientometrics, Vol. 41 No. 3, pp. 313\u201024.","DOI":"10.1007\/BF02459048"},{"key":"key2022013120341262000_B30","doi-asserted-by":"crossref","unstructured":"Lewison, G. (1998), \u201cGastroenterology research in the United Kingdom: funding sources and impact\u201d, Gut, Vol. 43 No. 2, pp. 288\u201093.","DOI":"10.1136\/gut.43.2.288"},{"key":"key2022013120341262000_B31","doi-asserted-by":"crossref","unstructured":"Lewison, G. (2002a), \u201cResearchers\u2019 and users\u2019 perceptions of the relative standing of biomedical papers in different journals\u201d, Scientometrics, Vol. 53 No. 2, pp. 229\u201040.","DOI":"10.1023\/A:1014804608785"},{"key":"key2022013120341262000_B32","doi-asserted-by":"crossref","unstructured":"Lewison, G. (2002b), \u201cFrom biomedical research to health improvement\u201d, Scientometrics, Vol. 54 No. 2, pp. 179\u201092.","DOI":"10.1023\/A:1016005710371"},{"key":"key2022013120341262000_B33","doi-asserted-by":"crossref","unstructured":"Loff, B. (2000), \u201cInformed consent for tissue retention discussed in Australia\u201d, The Lancet, Vol. 356, 11 November, p. 1663.","DOI":"10.1016\/S0140-6736(05)70375-5"},{"key":"key2022013120341262000_B34","doi-asserted-by":"crossref","unstructured":"McGinnis, J.M. and Foege, W.H. (1993), \u201cActual causes of death in the United States\u201d, Journal of the American Medical Association, Vol. 270 No. 18, pp. 2207\u201012.","DOI":"10.1001\/jama.270.18.2207"},{"key":"key2022013120341262000_B35","doi-asserted-by":"crossref","unstructured":"Meho, L.I. and Sonnenwald, D.H. (2000), \u201cCitation ranking versus peer evaluation of senior faculty research performance \u2013 a case\u2010study of Kurdish scholarship\u201d, Journal of the American Society of Information Science, Vol. 51 No. 2, pp. 123\u201038.","DOI":"10.1002\/(SICI)1097-4571(2000)51:2<123::AID-ASI4>3.0.CO;2-N"},{"key":"key2022013120341262000_B36","doi-asserted-by":"crossref","unstructured":"Narin, F. (1994), \u201cPatent bibliometrics\u201d, Scientometrics, Vol. 30 No. 1, pp. 147\u201055.","DOI":"10.1007\/BF02017219"},{"key":"key2022013120341262000_B37","doi-asserted-by":"crossref","unstructured":"Narin, F. and Olivastro, D. (1998), \u201cLinkage between patents and papers \u2013 an interim EPO\/US comparison\u201d, Scientometrics, Vol. 41 No. 1\u20102, pp. 51\u20109.","DOI":"10.1007\/BF02457966"},{"key":"key2022013120341262000_B38","doi-asserted-by":"crossref","unstructured":"Normile, D. (2000), \u201cAsia gets a taste of genetic food fights\u201d, Science, Vol. 289, 25 August, pp. 1279\u201081.","DOI":"10.1126\/science.289.5483.1279"},{"key":"key2022013120341262000_B39","doi-asserted-by":"crossref","unstructured":"Noyons, E.C.M., Luwel, M. and Moed, H.F. (1998), \u201cAssessment of Flemish research and development in the field of information technology \u2013 a bibliometric evaluation based on publication and patent data, combined with OECD research input statistics\u201d, Research Policy, Vol. 27 No. 3, pp. 285\u2010300.","DOI":"10.1016\/S0048-7333(98)00046-8"},{"key":"key2022013120341262000_B40","doi-asserted-by":"crossref","unstructured":"O\u2019Brien, C. (1996), \u201c...as Minister apologizes for misleading\u201d, Nature, Vol. 384 No. 6610, 19\u201026 December, p. 604.","DOI":"10.1038\/384604b0"},{"key":"key2022013120341262000_B41","unstructured":"Patel, K. (1994), \u201cViewers switch on to science\u201d, Times Higher, 24 September, p. 2."},{"key":"key2022013120341262000_B42","unstructured":"Research Fortnight (2000), \u201cUK falls behind Japan in biomedicine\u201d, Research Fortnight, Vol. 6, 23 February, p. 4."},{"key":"key2022013120341262000_B43","unstructured":"Research Fortnight (2002), \u201cWhat is left to medical science?\u201d, Research Fortnight, Vol. 8, 10 July, pp. 18\u201019."},{"key":"key2022013120341262000_B44","doi-asserted-by":"crossref","unstructured":"So, C.Y.K. (1998), \u201cCitation ranking versus expert judgement in evaluating communication scholars \u2013 effects of research speciality size and individual prominence\u201d, Scientometrics, Vol. 41 No. 3, pp. 325\u201033.","DOI":"10.1007\/BF02459049"},{"key":"key2022013120341262000_B45","unstructured":"Spooner, M.H. (2002), \u201cMeasles outbreaks in UK linked to fears about MMR vaccine\u201d, Canadian Medical Association Journal, Vol. 166 No. 8, p. 1075."},{"key":"key2022013120341262000_B46","unstructured":"The Times (2001), \u201c2001esearch centre plan rejected by council\u201d, Times Higher, 5 January, p. 2."},{"key":"key2022013120341262000_B47","unstructured":"Vidal, J. (2001), \u201cGM trials face delay as crops destroyed: company examines damage to six test sites\u201d, The Guardian, 9 June, p. 20."}],"container-title":["Aslib Proceedings"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/00012530310462698","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00012530310462698\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/00012530310462698\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T11:36:24Z","timestamp":1753356984000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/ajim\/article\/55\/1-2\/32-42\/21539"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2003,3,1]]},"references-count":47,"journal-issue":{"issue":"1\/2","published-print":{"date-parts":[[2003,3,1]]}},"alternative-id":["10.1108\/00012530310462698"],"URL":"https:\/\/doi.org\/10.1108\/00012530310462698","relation":{},"ISSN":["0001-253X"],"issn-type":[{"type":"print","value":"0001-253X"}],"subject":[],"published":{"date-parts":[[2003,3,1]]}}}