{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T02:16:32Z","timestamp":1775096192541,"version":"3.50.1"},"reference-count":24,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2011,3,20]],"date-time":"2011-03-20T00:00:00Z","timestamp":1300579200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2011,7]]},"DOI":"10.1007\/s11192-011-0373-2","type":"journal-article","created":{"date-parts":[[2011,3,19]],"date-time":"2011-03-19T06:29:00Z","timestamp":1300516140000},"page":"229-238","source":"Crossref","is-referenced-by-count":18,"title":["National research assessment exercises: the effects of changing the rules of the game during the game"],"prefix":"10.1007","volume":"88","author":[{"given":"Giovanni","family":"Abramo","sequence":"first","affiliation":[]},{"given":"Ciriaco Andrea","family":"D\u2019Angelo","sequence":"additional","affiliation":[]},{"given":"Flavia","family":"Di Costa","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2011,3,20]]},"reference":[{"issue":"2","key":"373_CR1","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1007\/s11192-010-0297-2","volume":"86","author":"G Abramo","year":"2011","unstructured":"Abramo, G., & D\u2019Angelo, C. A. (2011). National-scale research performance assessment at the individual level. Scientometrics, 86(2), 347\u2013364.","journal-title":"Scientometrics"},{"issue":"1","key":"373_CR2","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1016\/j.respol.2008.11.001","volume":"38","author":"G Abramo","year":"2009","unstructured":"Abramo, G., D\u2019Angelo, C. A., & Caprasecca, A. (2009). Allocative efficiency in public research funding: can bibliometrics help? Research Policy, 38(1), 206\u2013215.","journal-title":"Research Policy"},{"key":"373_CR3","unstructured":"Abramo G., D\u2019Angelo C. A., & Cicero T. (2011). The dispersion of research performance within and between universities as an indicator of the competitive intensity in higher education systems, working paper LabRTT, a short abstract available on http:\/\/www.disp.uniroma2.it\/laboratorioRTT\/TESTI\/Working%20paper\/RESPOL_Cicero.pdf ."},{"issue":"2","key":"373_CR4","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1007\/s11192-007-1942-2","volume":"76","author":"G Abramo","year":"2008","unstructured":"Abramo, G., D\u2019Angelo, C. A., & Pugini, F. (2008). The measurement of Italian universities\u2019 research productivity by a non parametric\u2013bibliometric methodology. Scientometrics, 76(2), 225\u2013244.","journal-title":"Scientometrics"},{"issue":"1","key":"373_CR5","doi-asserted-by":"crossref","first-page":"33","DOI":"10.3152\/147154404781776563","volume":"13","author":"DW Aksnes","year":"2004","unstructured":"Aksnes, D. W., & Taxt, R. E. (2004). Peers reviews and bibliometric indicators: a comparative study at Norwegian University. Research Evaluation, 13(1), 33\u201341.","journal-title":"Research Evaluation"},{"issue":"1","key":"373_CR6","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/S0048-7333(02)00007-0","volume":"32","author":"L Butler","year":"2003","unstructured":"Butler, L. (2003). Explaining Australia\u2019s increased share of ISI publications. The effects of a funding formula based on publication counts. Research Policy, 32(1), 143\u2013155.","journal-title":"Research Policy"},{"issue":"2","key":"373_CR7","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1002\/asi.21460","volume":"62","author":"CA D\u2019Angelo","year":"2011","unstructured":"D\u2019Angelo, C. A., Giuffrida, C., & Abramo, G. (2011). A heuristic approach to author name disambiguation in large-scale bibliometric databases. Journal of the American Society for Information Science and Technology, 62(2), 257\u2013269.","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"2","key":"373_CR8","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1023\/B:SCIE.0000018532.70146.02","volume":"59","author":"K Debackere","year":"2004","unstructured":"Debackere, K., & Gl\u00e4nzel, W. (2004). Using a bibliometric approach to support research policy making: The case of the Flemish BOF-key. Scientometrics, 59(2), 253\u2013276.","journal-title":"Scientometrics"},{"issue":"4","key":"373_CR9","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1023\/B:MINE.0000005155.70870.bd","volume":"41","author":"A Geuna","year":"2003","unstructured":"Geuna, A., & Martin, B. R. (2003). University research evaluation and funding: an international comparison. Minerva, 41(4), 277\u2013304.","journal-title":"Minerva"},{"key":"373_CR10","unstructured":"Gl\u00e4ser, J. (2007). The social orders of research evaluation systems. In R. Whitley & J. Gl\u00e4ser (Eds.), The changing governance of sciences (pp. 245\u2013264). Dordrecht: Springer."},{"issue":"1","key":"373_CR11","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1007\/s11192-009-0408-0","volume":"79","author":"I G\u00f3mez","year":"2009","unstructured":"G\u00f3mez, I., Bordons, M., Fern\u00e1ndez, M. T., & Morillo, F. (2009). Structure and research performance of Spanish universities. Scientometrics, 79(1), 131\u2013146.","journal-title":"Scientometrics"},{"issue":"2","key":"373_CR12","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1007\/s11192-007-1906-6","volume":"78","author":"C Kao","year":"2009","unstructured":"Kao, C., & Pao, H. L. (2009). An evaluation of research performance in management of 168 Taiwan universities. Scientometrics, 78(2), 261\u2013277.","journal-title":"Scientometrics"},{"issue":"7","key":"373_CR13","doi-asserted-by":"crossref","first-page":"489","DOI":"10.3152\/147154306781778777","volume":"33","author":"G Laudel","year":"2006","unstructured":"Laudel, G. (2006). The art of getting funded: how scientists adapt to their funding conditions. Science and Public Policy, 33(7), 489\u2013504.","journal-title":"Science and Public Policy"},{"issue":"4","key":"373_CR14","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1023\/A:1027381906977","volume":"46","author":"I Liefner","year":"2003","unstructured":"Liefner, I. (2003). Funding, resource allocation, and performance in higher education systems. Higher Education, 46(4), 469\u2013489.","journal-title":"Higher Education"},{"issue":"1","key":"373_CR15","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/s11192-008-0108-1","volume":"74","author":"HF Moed","year":"2008","unstructured":"Moed, H. F. (2008). UK research assessment exercises: Informed judgments on research quality or quantity? Scientometrics, 74(1), 153\u2013161.","journal-title":"Scientometrics"},{"issue":"1","key":"373_CR16","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1007\/s00005-009-0001-5","volume":"57","author":"HF Moed","year":"2009","unstructured":"Moed, H. F. (2009). New developments in the use of citation analysis in research evaluation. Archivum Immunologiae et therapiae Experimentalis, 57(1), 13\u201318.","journal-title":"Archivum Immunologiae et therapiae Experimentalis"},{"issue":"6","key":"373_CR17","first-page":"709","volume":"56","author":"C Oppenheim","year":"2003","unstructured":"Oppenheim, C., & Norris, M. (2003). Citation counts and the research assessment exercise V: Archaeology and the 2001 RAE. Journal of Documentation, 56(6), 709\u2013730.","journal-title":"Journal of Documentation"},{"issue":"1","key":"373_CR18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00005-009-0008-y","volume":"57","author":"DA Pendlebury","year":"2009","unstructured":"Pendlebury, D. A. (2009). The use and misuse of journal metrics and other citation indicators. Archivum Immunologiae et therapiae Experimentalis, 57(1), 1\u201311.","journal-title":"Archivum Immunologiae et therapiae Experimentalis"},{"key":"373_CR19","unstructured":"RAE. (2008). Research Assessment Exercise, www.rae.ac.uk . Accessed 2 Feb 2011."},{"key":"373_CR20","unstructured":"REF. (2011). Research Excellence Framework. http:\/\/www.hefce.ac.uk\/Research\/ref\/ . Accessed 2 Feb 2011."},{"key":"373_CR21","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/S0048-7333(98)00026-2","volume":"27","author":"EJ Rinia","year":"1998","unstructured":"Rinia, E. J., van Leeuwen, Th. N., van Vuren, H. G., & Van Raan, A. F. J. (1998). Comparative analysis of a set of bibliometric indicators and central peer review criteria, Evaluation of condensed matter physics in the Netherlands. Research Policy, 27, 95\u2013107.","journal-title":"Research Policy"},{"issue":"2","key":"373_CR22","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1023\/A:1005651429368","volume":"47","author":"R Rousseau","year":"2000","unstructured":"Rousseau, R., & Smeyers, M. (2000). Output-financing at LUC. Scientometrics, 47(2), 379\u2013387.","journal-title":"Scientometrics"},{"key":"373_CR23","unstructured":"VQR. (2011). Linee guida VQR 2004\u20132008, http:\/\/www.civr.miur.it\/vqr_decreto.html . Accessed 2 Feb 2011."},{"key":"373_CR24","unstructured":"VTR (2006). Italian Triennial Research Evaluation. VTR 2001\u20132003. Risultati delle valutazioni dei Panel di Area. http:\/\/vtr2006.cineca.it\/ . Accessed 2 Feb 2011."}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-011-0373-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11192-011-0373-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-011-0373-2","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,6,1]],"date-time":"2019-06-01T09:34:15Z","timestamp":1559381655000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11192-011-0373-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,3,20]]},"references-count":24,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2011,7]]}},"alternative-id":["373"],"URL":"https:\/\/doi.org\/10.1007\/s11192-011-0373-2","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,3,20]]}}}