{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T07:06:18Z","timestamp":1776927978372,"version":"3.51.2"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"12","license":[{"start":{"date-parts":[[2022,6,19]],"date-time":"2022-06-19T00:00:00Z","timestamp":1655596800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,6,19]],"date-time":"2022-06-19T00:00:00Z","timestamp":1655596800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004270","name":"Royal Institute of Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004270","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>While normalized bibliometric indicators are expected to resolve the subject-field differences between organizations in research evaluations, the identification of reference organizations working on similar research topics is still of importance. Research organizations, policymakers and research funders tend to use benchmark units as points of comparison for a certain research unit in order to understand and monitor its development and performance. In addition, benchmark organizations can also be used to pinpoint potential collaboration partners or competitors. Therefore, methods for identifying benchmark research units are of practical significance. Even so, few studies have further explored this problem. This study aims to propose a bibliometric approach for the identification of benchmark units. We define an appropriate benchmark as a well-connected research environment, in which researchers investigate similar topics and publish a similar number of publications compared to a given research organization during the same period. Four essential attributes for the evaluation of benchmarks are research topics, output, connectedness, and scientific impact. We apply this strategy to two research organizations in Sweden and examine the effectiveness of the proposed method. Identified benchmark units are evaluated by examining the research similarity and the robustness of various measures of connectivity.<\/jats:p>","DOI":"10.1007\/s11192-022-04413-7","type":"journal-article","created":{"date-parts":[[2022,6,19]],"date-time":"2022-06-19T15:02:47Z","timestamp":1655650967000},"page":"7557-7574","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Identifying benchmark units for research management and evaluation"],"prefix":"10.1007","volume":"127","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7817-5327","authenticated-orcid":false,"given":"Qi","family":"Wang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tobias","family":"Jeppsson","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,6,19]]},"reference":[{"issue":"4","key":"4413_CR1","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1108\/14635770910972441","volume":"16","author":"P Amaral","year":"2009","unstructured":"Amaral, P., & Sousa, R. (2009). Barriers to internal benchmarking initiatives: An empirical investigation. Benchmarking: An International Journal, 16(4), 523\u2013542.","journal-title":"Benchmarking: An International Journal"},{"issue":"3","key":"4413_CR2","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1108\/14635770810876593","volume":"15","author":"G Anand","year":"2008","unstructured":"Anand, G., & Kodali, R. (2008). Benchmarking the benchmarking models. Benchmarking: An International Journal, 15(3), 257\u2013291.","journal-title":"Benchmarking: An International Journal"},{"key":"4413_CR3","unstructured":"Andersen, J. P., Didegah, F., & Schneider, J. W. (2017). The necessity of comparing like with like in evaluative scientometrics: A first attempt to produce and test a generic approach to identifying relevant benchmark units. In STI conference science and technology indicators conference."},{"key":"4413_CR4","doi-asserted-by":"crossref","unstructured":"Bast, H., & Korzen, C. (2017). A benchmark and evaluation for text extraction from pdf. In 2017 ACM\/IEEE joint conference on digital libraries (JCDL).","DOI":"10.1109\/JCDL.2017.7991564"},{"issue":"4","key":"4413_CR5","doi-asserted-by":"publisher","first-page":"934","DOI":"10.1108\/BIJ-10-2015-0100","volume":"24","author":"HH Bi","year":"2017","unstructured":"Bi, H. H. (2017). Multi-criterion and multi-period performance benchmarking of products and services: Discovering hidden performance gaps. Benchmarking: An International Journal, 24(4), 934\u2013972.","journal-title":"Benchmarking: An International Journal"},{"key":"4413_CR47","unstructured":"Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan), 993\u20131022."},{"key":"4413_CR6","unstructured":"Camp, R. C. (1989). Benchmarking: The search for industry best practices that lead to superior performance. Quality Press."},{"issue":"2","key":"4413_CR46","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1007\/s11192-012-0672-2","volume":"93","author":"N Carayol","year":"2012","unstructured":"Carayol, N., Filliatreau, G., & Lahatte, A. (2012). Reference classes: a tool for benchmarking universities\u2019 research. Scientometrics, 93(2), 351\u2013371.","journal-title":"Scientometrics"},{"issue":"3","key":"4413_CR7","doi-asserted-by":"publisher","first-page":"244","DOI":"10.1108\/14635770210429009","volume":"9","author":"LC Carpinetti","year":"2002","unstructured":"Carpinetti, L. C., & De Melo, A. M. (2002). What to benchmark? Benchmarking: An International Journal, 9(3), 244\u2013255.","journal-title":"Benchmarking: An International Journal"},{"issue":"3","key":"4413_CR45","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1108\/14635770310477744","volume":"10","author":"R Dattakumar","year":"2003","unstructured":"Dattakumar, R., & Jagadeesh, R. (2003). A review of literature on benchmarking. Benchmarking: An International Journal, 10(3), 176\u2013209.","journal-title":"Benchmarking: An International Journal"},{"issue":"4","key":"4413_CR8","volume":"7","author":"A Ettorchi-Tardy","year":"2012","unstructured":"Ettorchi-Tardy, A., Levif, M., & Michel, P. (2012). Benchmarking: A method for continuous quality improvement in health. Healthcare Policy, 7(4), e101.","journal-title":"Healthcare Policy"},{"issue":"2","key":"4413_CR9","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1016\/S1084-8568(96)90012-7","volume":"1","author":"DB Fedor","year":"1996","unstructured":"Fedor, D. B., Parsons, C. K., & Shalley, C. E. (1996). Organizational comparison processes: Investigating the adoption and impact of benchmarking-related activities. Journal of Quality Management, 1(2), 161\u2013192.","journal-title":"Journal of Quality Management"},{"issue":"3","key":"4413_CR48","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1016\/j.joi.2017.06.006","volume":"11","author":"K Frenken","year":"2017","unstructured":"Frenken, K., Heimeriks, G. J., & Hoekman, J. (2017). What drives university research performance? An analysis using the CWTS Leiden Ranking data. Journal of informetrics, 11(3), 859\u2013872.","journal-title":"Journal of Informetrics"},{"issue":"2","key":"4413_CR10","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1007\/BF02018475","volume":"35","author":"W Gl\u00e4nzel","year":"1996","unstructured":"Gl\u00e4nzel, W. (1996). The need for standards in bibliometric research and technology. Scientometrics, 35(2), 167\u2013176.","journal-title":"Scientometrics"},{"issue":"3","key":"4413_CR11","doi-asserted-by":"publisher","first-page":"512","DOI":"10.1016\/S0377-2217(00)00195-8","volume":"133","author":"E Gonz\u00e1lez","year":"2001","unstructured":"Gonz\u00e1lez, E., & \u00c1lvarez, A. (2001). From efficiency measurement to efficiency improvement: The choice of a relevant benchmark. European Journal of Operational Research, 133(3), 512\u2013520.","journal-title":"European Journal of Operational Research"},{"issue":"2","key":"4413_CR12","doi-asserted-by":"publisher","first-page":"480","DOI":"10.1016\/j.tranpol.2010.10.012","volume":"18","author":"TF Henning","year":"2011","unstructured":"Henning, T. F., Muruvan, S., Feng, W. A., & Dunn, R. C. (2011). The development of a benchmarking tool for monitoring progress towards sustainable transportation in New Zealand. Transport Policy, 18(2), 480\u2013488.","journal-title":"Transport Policy"},{"issue":"5","key":"4413_CR13","volume":"9","author":"F Hoffmann","year":"2019","unstructured":"Hoffmann, F., Bertram, T., Mikut, R., Reischl, M., & Nelles, O. (2019). Benchmarking in classification and regression. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(5), e1318.","journal-title":"Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery"},{"issue":"13","key":"4413_CR14","first-page":"1","volume":"40","author":"K Hornik","year":"2011","unstructured":"Hornik, K., & Gr\u00fcn, B. (2011). Topicmodels: An R package for fitting topic models. Journal of Statistical Software, 40(13), 1\u201330.","journal-title":"Journal of Statistical Software"},{"issue":"3","key":"4413_CR15","doi-asserted-by":"publisher","first-page":"675","DOI":"10.1198\/106186005X59630","volume":"14","author":"T Hothorn","year":"2005","unstructured":"Hothorn, T., Leisch, F., Zeileis, A., & Hornik, K. (2005). The design and analysis of benchmark experiments. Journal of Computational and Graphical Statistics, 14(3), 675\u2013699.","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"1","key":"4413_CR16","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/S0167-6377(03)00073-7","volume":"32","author":"JL Hougaard","year":"2004","unstructured":"Hougaard, J. L., & Keiding, H. (2004). Continuous benchmark selections. Operations Research Letters, 32(1), 94\u201398.","journal-title":"Operations Research Letters"},{"issue":"1","key":"4413_CR17","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1016\/S0377-2217(01)00080-7","volume":"137","author":"JL Hougaard","year":"2002","unstructured":"Hougaard, J. L., & Tvede, M. (2002). Benchmark selection: An axiomatic approach. European Journal of Operational Research, 137(1), 218\u2013228.","journal-title":"European Journal of Operational Research"},{"key":"4413_CR18","doi-asserted-by":"crossref","unstructured":"Huppler, K. (2009). The art of building a good benchmark. In Technology conference on performance evaluation and benchmarking. Springer.","DOI":"10.1007\/978-3-642-10424-4_3"},{"key":"4413_CR19","unstructured":"Fitz-Enz, J. (1993). Benchmarking staff performance: How staff departments can enhance their value to the customer. Pfeiffer."},{"issue":"3","key":"4413_CR20","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1016\/j.giq.2010.02.003","volume":"27","author":"J Jansen","year":"2010","unstructured":"Jansen, J., de Vries, S., & van Schaik, P. (2010). The contextual benchmark method: Benchmarking e-government services. Government Information Quarterly, 27(3), 213\u2013219.","journal-title":"Government Information Quarterly"},{"issue":"1","key":"4413_CR21","doi-asserted-by":"publisher","first-page":"23","DOI":"10.3152\/147154300781782156","volume":"27","author":"JS Katz","year":"2000","unstructured":"Katz, J. S. (2000). Scale-independent indicators and research evaluation. Science and Public Policy, 27(1), 23\u201336.","journal-title":"Science and Public Policy"},{"key":"4413_CR22","doi-asserted-by":"crossref","unstructured":"V. Kistowski, J., Arnold, J. A., Huppler, K., Lange, K. D., Henning, J. L., & Cao, P. (2015). How to build a benchmark. In Proceedings of the 6th ACM\/SPEC international conference on performance engineering.","DOI":"10.1145\/2668930.2688819"},{"issue":"4","key":"4413_CR23","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1016\/j.giq.2012.04.006","volume":"29","author":"E Kromidha","year":"2012","unstructured":"Kromidha, E. (2012). Strategic e-government development and the role of benchmarking. Government Information Quarterly, 29(4), 573\u2013581.","journal-title":"Government Information Quarterly"},{"issue":"3","key":"4413_CR24","doi-asserted-by":"publisher","first-page":"290","DOI":"10.1108\/14635770610668794","volume":"13","author":"A Kumar","year":"2006","unstructured":"Kumar, A., Antony, J., & Dhakar, T. S. (2006). Integrating quality function deployment and benchmarking to achieve greater profitability. Benchmarking: An International Journal, 13(3), 290\u2013310.","journal-title":"Benchmarking: An International Journal"},{"key":"4413_CR25","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1016\/j.jnucmat.2015.03.041","volume":"462","author":"Y Li","year":"2015","unstructured":"Li, Y., & Korzhavyi, P. A. (2015). Interactions of point defects with stacking faults in oxygen-free phosphorus-containing copper. Journal of Nuclear Materials, 462, 160\u2013164.","journal-title":"Journal of Nuclear Materials"},{"key":"4413_CR26","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1016\/j.swevo.2018.04.005","volume":"44","author":"Y Lou","year":"2019","unstructured":"Lou, Y., & Yuen, S. Y. (2019). On constructing alternative benchmark suite for evolutionary algorithms. Swarm and Evolutionary Computation, 44, 287\u2013292.","journal-title":"Swarm and Evolutionary Computation"},{"issue":"1","key":"4413_CR27","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/0925-5273(95)99064-C","volume":"38","author":"M Lucertini","year":"1995","unstructured":"Lucertini, M., Nicol\u00f2, F., & Telmon, D. (1995). Integration of benchmarking and benchmarking of integration. International Journal of Production Economics, 38(1), 59\u201371.","journal-title":"International Journal of Production Economics"},{"issue":"3","key":"4413_CR28","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1007\/BF02129600","volume":"36","author":"G Melin","year":"1996","unstructured":"Melin, G., & Persson, O. (1996). Studying research collaboration using co-authorships. Scientometrics, 36(3), 363\u2013377.","journal-title":"Scientometrics"},{"issue":"4","key":"4413_CR29","doi-asserted-by":"publisher","first-page":"588","DOI":"10.1108\/14635771111147650","volume":"18","author":"JP Moriarty","year":"2011","unstructured":"Moriarty, J. P. (2011). A theory of benchmarking. Benchmarking: An International Journal, 18(4), 588\u2013612.","journal-title":"Benchmarking: An International Journal"},{"issue":"2","key":"4413_CR44","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.socnet.2009.02.002","volume":"31","author":"T Opsahl","year":"2009","unstructured":"Opsahl, T., & Panzarasa, P. (2009). Clustering in weighted networks. Social networks, 31(2), 155\u2013163.","journal-title":"Social networks"},{"issue":"2","key":"4413_CR30","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1002\/(SICI)1097-4571(1999)50:2<115::AID-ASI3>3.0.CO;2-J","volume":"50","author":"EC Noyons","year":"1999","unstructured":"Noyons, E. C., Moed, H. F., & Luwel, M. (1999). Combining mapping and citation analysis for evaluative bibliometric purposes: A bibliometric study. Journal of the American Society for Information Science, 50(2), 115\u2013131.","journal-title":"Journal of the American Society for Information Science"},{"issue":"4","key":"4413_CR31","doi-asserted-by":"publisher","first-page":"597","DOI":"10.1016\/j.giq.2012.05.008","volume":"29","author":"M Petrovi\u0107","year":"2012","unstructured":"Petrovi\u0107, M., Bojkovi\u0107, N., Ani\u0107, I., & Petrovi\u0107, D. (2012). Benchmarking the digital divide using a multi-level outranking framework: Evidence from EBRD countries of operation. Government Information Quarterly, 29(4), 597\u2013607.","journal-title":"Government Information Quarterly"},{"issue":"6","key":"4413_CR32","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1108\/eb039332","volume":"10","author":"LS Pryor","year":"1989","unstructured":"Pryor, L. S. (1989). Benchmarking: A self-improvement strategy. The Journal of Business Strategy, 10(6), 28.","journal-title":"The Journal of Business Strategy"},{"key":"4413_CR33","doi-asserted-by":"crossref","unstructured":"Rousseau, R. (2020). Benchmarkings and rankings. In R. Ball (Ed.), Handbook bibliometrics (pp. 299\u2013309). De Gruyter Saur.","DOI":"10.1515\/9783110646610-030"},{"issue":"1","key":"4413_CR34","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1016\/j.joi.2014.11.010","volume":"9","author":"J Ruiz-Castillo","year":"2015","unstructured":"Ruiz-Castillo, J., & Waltman, L. (2015). Field-normalized citation impact indicators using algorithmically constructed classification systems of science. Journal of Informetrics, 9(1), 102\u2013117.","journal-title":"Journal of Informetrics"},{"issue":"3","key":"4413_CR35","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1108\/IJQSS-04-2016-0033","volume":"8","author":"M Sankey","year":"2016","unstructured":"Sankey, M., & Padr\u00f3, F. F. (2016). ACODE Benchmarks for technology enhanced learning (TEL): Findings from a 24 university benchmarking exercise regarding the benchmarks\u2019 fitness for purpose. International Journal of Quality and Service Sciences, 8(3), 345\u2013362.","journal-title":"International Journal of Quality and Service Sciences"},{"key":"4413_CR36","doi-asserted-by":"publisher","DOI":"10.1016\/j.seps.2019.04.002","volume":"68","author":"G Tuzkaya","year":"2019","unstructured":"Tuzkaya, G., Sennaroglu, B., Kalender, Z. T., & Mutlu, M. (2019). Hospital service quality evaluation with IVIF-PROMETHEE and a case study. Socio-Economic Planning Sciences, 68, 100705.","journal-title":"Socio-Economic Planning Sciences"},{"issue":"2","key":"4413_CR37","doi-asserted-by":"publisher","first-page":"372","DOI":"10.1002\/asi.22775","volume":"64","author":"L Waltman","year":"2013","unstructured":"Waltman, L., & Schreiber, M. (2013). On the calculation of percentile-based bibliometric indicators. Journal of the American Society for Information Science and Technology, 64(2), 372\u2013379.","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"12","key":"4413_CR38","doi-asserted-by":"publisher","first-page":"2378","DOI":"10.1002\/asi.22748","volume":"63","author":"L Waltman","year":"2012","unstructured":"Waltman, L., & Van Eck, N. J. (2012). A new methodology for constructing a publication-level classification system of science. Journal of the American Society for Information Science and Technology, 63(12), 2378\u20132392.","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"11","key":"4413_CR39","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1140\/epjb\/e2013-40829-0","volume":"86","author":"L Waltman","year":"2013","unstructured":"Waltman, L., & Van Eck, N. J. (2013). A smart local moving algorithm for large-scale modularity-based community detection. The European Physical Journal B, 86(11), 471.","journal-title":"The European Physical Journal B"},{"issue":"2","key":"4413_CR40","doi-asserted-by":"publisher","first-page":"290","DOI":"10.1002\/asi.23930","volume":"69","author":"Q Wang","year":"2018","unstructured":"Wang, Q. (2018). A bibliometric model for identifying emerging research topics. Journal of the Association for Information Science and Technology, 69(2), 290\u2013304.","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"4413_CR41","unstructured":"Wang, Q. & Jeppsson, T. (2021). A bibliometric strategy for identifying benchmark research units. In 18th International Conference on Scientometrics & Informetrics (pp. 1229\u20131234)."},{"issue":"2","key":"4413_CR42","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1007\/s11192-009-0146-3","volume":"84","author":"NJ Van Eck","year":"2010","unstructured":"Van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84(2), 523\u2013538.","journal-title":"Scientometrics"},{"issue":"2","key":"4413_CR43","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1007\/s11192-005-0218-y","volume":"63","author":"M Zitt","year":"2005","unstructured":"Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2005). Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalisation. Scientometrics, 63(2), 373\u2013401.","journal-title":"Scientometrics"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04413-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-022-04413-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04413-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,5]],"date-time":"2022-12-05T05:19:19Z","timestamp":1670217559000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-022-04413-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,19]]},"references-count":48,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["4413"],"URL":"https:\/\/doi.org\/10.1007\/s11192-022-04413-7","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,19]]},"assertion":[{"value":"31 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 May 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 June 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}