{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,5]],"date-time":"2026-04-05T09:31:48Z","timestamp":1775381508117,"version":"3.50.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T00:00:00Z","timestamp":1657843200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T00:00:00Z","timestamp":1657843200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["01PQ17001"],"award-info":[{"award-number":["01PQ17001"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Max Planck Institute for Solid State Research"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2022,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Although it is bibliometric standard to employ field normalization, the detailed procedure of field normalization is not standardized regarding the handling of the document types. All publications without filtering the document type can be used or only selected document types. Furthermore, the field-normalization procedure can be carried out with regard to the document type of publications or without. We studied if the field-normalized scores strongly depend on the choice of different document type handlings. In doing so, we used the publications from the Web of Science between 2000 and 2017 and compared different field-normalized scores. We compared the results on the individual publication level, the country level, and the institutional level. We found rather high correlations between the different scores but the concordance values provide a more differentiated conclusion: Rather different scores are produced on the individual publication level. As our results on the aggregated levels are not supported by our results on the level of individual publications, any comparison of normalized scores that result from different procedures should only be performed with caution.<\/jats:p>","DOI":"10.1007\/s11192-022-04446-y","type":"journal-article","created":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T12:03:58Z","timestamp":1657886638000},"page":"4419-4438","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Relevance of document types in the scores\u2019 calculation of a specific field-normalized indicator: Are the scores strongly dependent on or nearly independent of the document type handling?"],"prefix":"10.1007","volume":"127","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7025-7256","authenticated-orcid":false,"given":"Robin","family":"Haunschild","sequence":"first","affiliation":[]},{"given":"Lutz","family":"Bornmann","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,7,15]]},"reference":[{"issue":"1","key":"4446_CR1","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1162\/qss_a_00018","volume":"1","author":"C Birkle","year":"2020","unstructured":"Birkle, C., Pendlebury, D. A., Schnell, J., & Adams, J. (2020). Web of Science as a data source for research on scientific and scholarly activity. Quantitative Science Studies, 1(1), 363\u2013376. https:\/\/doi.org\/10.1162\/qss_a_00018","journal-title":"Quantitative Science Studies"},{"issue":"2","key":"4446_CR2","doi-asserted-by":"publisher","first-page":"408","DOI":"10.1016\/j.joi.2015.01.006","volume":"9","author":"L Bornmann","year":"2015","unstructured":"Bornmann, L., & Marx, W. (2015). Methods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgements of experts? Journal of Informetrics, 9(2), 408\u2013418.","journal-title":"Journal of Informetrics"},{"key":"4446_CR3","unstructured":"Clarivate Analytics. (2021). InCites indicators handbook. Retrieved 8 July 2022, from http:\/\/incites.help.clarivate.com\/Content\/Indicators-Handbook\/ih-about.htm"},{"key":"4446_CR4","unstructured":"Clarivate Analytics. (2021). The Clarivate Analytics Impact Factor. Retrieved 24 March 2021, from https:\/\/clarivate.com\/webofsciencegroup\/essays\/impact-factor\/"},{"key":"4446_CR5","unstructured":"CWTS. (2022). CWTS Leiden Ranking Indicators. Retrieved 2 May 2022, from https:\/\/www.leidenranking.com\/information\/indicators"},{"key":"4446_CR6","unstructured":"Elsevier. (2019). Research metrics guidebook. Elsevier, Amsterdam, the Netherlands."},{"issue":"2","key":"4446_CR7","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1023\/A:1014848323806","volume":"53","author":"W Gl\u00e4nzel","year":"2002","unstructured":"Gl\u00e4nzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171\u2013193. https:\/\/doi.org\/10.1023\/A:1014848323806","journal-title":"Scientometrics"},{"key":"4446_CR520","doi-asserted-by":"publisher","unstructured":"Haunschild, R., Daniels, A.D. and Bornmann, L. (2022). Scores of a specific field-normalized indicator calculated with different approaches of field-categorization: Are the scores different or similar? Journal of Informetrics, 16 (1), 101241, ISSN 1751-1577. https:\/\/doi.org\/10.1016\/j.joi.2021.101241. Accessed 17 May 2021","DOI":"10.1016\/j.joi.2021.101241"},{"key":"4446_CR8","unstructured":"Haunschild, R., Marx, W., French, B., & Bornmann, L. (2018). Relationship between field-normalized indicators calculated with different approaches of field-categorization. Paper presented at the 23rd International Conference on Science and Technology Indicators, Leiden (The Netherlands). https:\/\/scholarlypublications.universiteitleiden.nl\/handle\/1887\/65267. Accessed 17 May 2021"},{"issue":"1","key":"4446_CR9","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1162\/qss_a_00020","volume":"1","author":"C Herzog","year":"2020","unstructured":"Herzog, C., Hook, D., & Konkiel, S. (2020). Dimensions: Bringing down barriers between scientometricians and data. Quantitative Science Studies, 1(1), 387\u2013395. https:\/\/doi.org\/10.1162\/qss_a_00020","journal-title":"Quantitative Science Studies"},{"issue":"7548","key":"4446_CR10","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1038\/520429a","volume":"520","author":"D Hicks","year":"2015","unstructured":"Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429\u2013431.","journal-title":"Nature"},{"issue":"9","key":"4446_CR11","doi-asserted-by":"publisher","first-page":"e1002542","DOI":"10.1371\/journal.pbio.1002542","volume":"14","author":"JPA Ioannidis","year":"2016","unstructured":"Ioannidis, J. P. A., Boyack, K., & Wouters, P. F. (2016). Citation metrics: A primer on how (not) to normalize. PLoS Biology, 14(9), e1002542. https:\/\/doi.org\/10.1371\/journal.pbio.1002542","journal-title":"PLoS Biology"},{"issue":"1","key":"4446_CR12","doi-asserted-by":"publisher","first-page":"255","DOI":"10.2307\/2532051","volume":"45","author":"LI Lin","year":"1989","unstructured":"Lin, L. I. (1989). A concordance correlation-coefficient to evaluate reproducibility. Biometrics, 45(1), 255\u2013268. https:\/\/doi.org\/10.2307\/2532051","journal-title":"Biometrics"},{"issue":"1","key":"4446_CR13","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1111\/j.0006-341X.2000.00324.x","volume":"56","author":"LI Lin","year":"2000","unstructured":"Lin, L. I. (2000). A note on the concordance correlation coefficient. Biometrics, 56(1), 324\u2013325. https:\/\/doi.org\/10.1111\/j.0006-341X.2000.00324.x","journal-title":"Biometrics"},{"issue":"2","key":"4446_CR14","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1016\/j.joi.2006.09.007","volume":"1","author":"J Lundberg","year":"2007","unstructured":"Lundberg, J. (2007). Lifting the crown: Citation z-score. Journal of Informetrics, 1(2), 145\u2013154.","journal-title":"Journal of Informetrics"},{"key":"4446_CR15","unstructured":"McBride, G. B. (2005). A proposal for strength-of-agreement criteria for Lin\u2019s concordance correlation coefficient. National Institute of Water & Atmospheric Research: Hamilton, New Zealand"},{"issue":"6","key":"4446_CR16","doi-asserted-by":"publisher","first-page":"658","DOI":"10.1108\/00220410410568142","volume":"60","author":"AJ Nederhof","year":"2004","unstructured":"Nederhof, A. J., & Visser, M. S. (2004). Quantitative deconstruction of citation impact indicators. Journal of Documentation, 60(6), 658\u2013672. https:\/\/doi.org\/10.1108\/00220410410568142","journal-title":"Journal of Documentation"},{"key":"4446_CR17","unstructured":"OurResearch. (2021). We\u2019re building a replacement for Microsoft Academic Graph. Retrieved 11 April 2022, from https:\/\/blog.ourresearch.org\/were-building-a-replacement-for-microsoft-academic-graph\/"},{"key":"4446_CR18","unstructured":"Priem, J., Piwowar, H., & Orr, R. (2022). OpenAlex: A fully-open index of scholarly works, authors, venues, institutions. https:\/\/arxiv.org\/abs\/2205.01833. Accessed 17 May 2021"},{"key":"4446_CR19","unstructured":"R Core Team. (2019). R: A language and environment for statistical computing (Version 3.6.0). Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https:\/\/www.r-project.org\/. Accessed 17 May 2021"},{"key":"4446_CR20","volume-title":"Bibliometric indicators: Definitions and usage at Karolinska Institutet","author":"C Rehn","year":"2007","unstructured":"Rehn, C., Kronman, U., & Wadskog, D. (2007). Bibliometric indicators: Definitions and usage at Karolinska Institutet. Karolinska Institutet University Library."},{"key":"4446_CR21","unstructured":"Scheidsteger, T., Haunschild, R., Hug, S., & Bornmann, L. (2018). The concordance of field-normalized scores based on Web of Science and Microsoft Academic data: A case study in computer sciences Paper presented at the STI 2018 Conference Proceedings, Leiden, The Netherlands. https:\/\/scholarlypublications.universiteitleiden.nl\/handle\/1887\/65358?solr_nav%5Bid%5D=3add44046ea4b81cb330&solr_nav%5Bpage%5D=0&solr_nav%5Boffset%5D=0. Accessed 17 May 2021"},{"key":"4446_CR22","unstructured":"Signorell, A., Aho, K., Alfons, A., Anderegg, N., Aragon, T., Arachchige, C., Zeileis, A. (2020). DescTools: Tools for descriptive statistics. Retrieved 24 March 2021, from https:\/\/cran.r-project.org\/package=DescTools"},{"key":"4446_CR23","doi-asserted-by":"crossref","unstructured":"Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J. P., & Wang, K. (2015). An overview of Microsoft Academic Service (MAS) and applications. Paper presented at the 24th International Conference on World Wide Web (WWW \u201915 Companion), Florence, Italy.","DOI":"10.1145\/2740908.2742839"},{"issue":"4","key":"4446_CR24","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1108\/EUM0000000007175","volume":"54","author":"TN Van Leeuwen","year":"1998","unstructured":"Van Leeuwen, T. N., Moed, H. F., & Reedijk, J. (1998). A new classification system to describe the ageing of scientific journals and their impact factors. Journal of Documentation, 54(4), 387\u2013419. https:\/\/doi.org\/10.1108\/EUM0000000007175","journal-title":"Journal of Documentation"},{"issue":"2","key":"4446_CR25","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1016\/j.joi.2016.02.007","volume":"10","author":"L Waltman","year":"2016","unstructured":"Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365\u2013391.","journal-title":"Journal of Informetrics"},{"issue":"12","key":"4446_CR26","doi-asserted-by":"publisher","first-page":"2419","DOI":"10.1002\/asi.22708","volume":"63","author":"L Waltman","year":"2012","unstructured":"Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E. C., Tijssen, R. J., van Eck, N. J., van Leeuwen, T. N., van Raan, A. F., Visser, M. S., & Wouters, P. (2012). The Leiden ranking 2011\/2012: Data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology, 63(12), 2419\u20132432. https:\/\/doi.org\/10.1002\/asi.22708","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"1","key":"4446_CR27","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/j.joi.2010.08.001","volume":"5","author":"L Waltman","year":"2011","unstructured":"Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2011). Towards a new crown indicator: Some theoretical considerations. Journal of Informetrics, 5(1), 37\u201347. https:\/\/doi.org\/10.1016\/j.joi.2010.08.001","journal-title":"Journal of Informetrics"},{"issue":"3","key":"4446_CR28","doi-asserted-by":"publisher","first-page":"851","DOI":"10.1007\/s11192-012-0775-9","volume":"94","author":"J Wang","year":"2013","unstructured":"Wang, J. (2013). Citation time window choice for research impact evaluation. Scientometrics, 94(3), 851\u2013872. https:\/\/doi.org\/10.1007\/s11192-012-0775-9","journal-title":"Scientometrics"},{"issue":"1","key":"4446_CR29","doi-asserted-by":"publisher","first-page":"396","DOI":"10.1162\/qss_a_00021","volume":"1","author":"K Wang","year":"2020","unstructured":"Wang, K., Shen, Z., Huang, C., Wu, C.-H., Dong, Y., & Kanakia, A. (2020). Microsoft academic graph: When experts are not enough. Quantitative Science Studies, 1(1), 396\u2013413. https:\/\/doi.org\/10.1162\/qss_a_00021","journal-title":"Quantitative Science Studies"},{"key":"4446_CR30","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24277-4","volume-title":"ggplot2: Elegant graphics for data analysis","author":"H Wickham","year":"2016","unstructured":"Wickham, H. (2016). ggplot2: Elegant graphics for data analysis. Springer."},{"key":"4446_CR31","unstructured":"Wickham, H. (2017). Tidyverse: Easily install and load the 'Tidyverse'. R package version 1.2.1. Retrieved 22 June 2020, from https:\/\/CRAN.R-project.org\/package=tidyverse"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04446-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-022-04446-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04446-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,10]],"date-time":"2022-08-10T07:42:35Z","timestamp":1660117355000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-022-04446-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,15]]},"references-count":32,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2022,8]]}},"alternative-id":["4446"],"URL":"https:\/\/doi.org\/10.1007\/s11192-022-04446-y","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,15]]},"assertion":[{"value":"17 May 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 June 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 July 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}