{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T09:05:15Z","timestamp":1774947915005,"version":"3.50.1"},"reference-count":19,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T00:00:00Z","timestamp":1714348800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T00:00:00Z","timestamp":1714348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Max-Planck-Gesellschaft zur F\u00f6rderung der Wissenschaften e.V."}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In evaluative bibliometrics and higher education studies, one is frequently confronted with the task of comparing institutions with similar institutions. In this Letter to the Editor, a simple approach is discussed which applies ChatGPT. Although the approach seems to produce promising results (tested with an example at the level of research institute and of a university), it is necessary to investigate it systematically based on a sample including many institutions before it should be applied in research evaluation. The challenge in systematic investigations is that ChatGPT provides the user with different answers on the sane request (missing reliability).<\/jats:p>","DOI":"10.1007\/s11192-024-05039-7","type":"journal-article","created":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T11:03:29Z","timestamp":1714388609000},"page":"3593-3598","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["The use of ChatGPT to find similar institutions for institutional benchmarking"],"prefix":"10.1007","volume":"129","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0810-7091","authenticated-orcid":false,"given":"Lutz","family":"Bornmann","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benedetto","family":"Lepori","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,4,29]]},"reference":[{"key":"5039_CR1","unstructured":"Andersen, J. P., Didegah, F., & Schneider, J. W. (2017). The necessity of comparing like with like in evaluative scientometrics: A first attempt to produce and test a generic approach to identifying relevant benchmark units. In Proceedings of the science, technology, & innovation indicators conference \u201cOpen indicators: Innovation, participation and actor-based STI indicators\u201d. Paris, France."},{"issue":"11","key":"5039_CR2","doi-asserted-by":"publisher","first-page":"2310","DOI":"10.1002\/asi.22923","volume":"64","author":"L Bornmann","year":"2013","unstructured":"Bornmann, L., de Moya Aneg\u00f3n, F., & Mutz, R. (2013). Do universities or research institutions with a specific subject profile have an advantage or a disadvantage in institutional rankings? A latent class analysis with data from the SCImago ranking. Journal of the American Society for Information Science and Technology, 64(11), 2310\u20132316.","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"2","key":"5039_CR3","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1007\/s11192-012-0672-2","volume":"93","author":"N Carayol","year":"2012","unstructured":"Carayol, N., Filliatreau, G., & Lahatte, A. (2012). Reference classes: A tool for benchmarking universities\u2019 research. Scientometrics, 93(2), 351\u2013371. https:\/\/doi.org\/10.1007\/s11192-012-0672-2","journal-title":"Scientometrics"},{"key":"5039_CR4","volume-title":"Rankings and university performance: A conditional multidimensional approach","author":"C Daraio","year":"2014","unstructured":"Daraio, C., Bonaccorsi, A., & Simar, L. (2014). Rankings and university performance: A conditional multidimensional approach. University of Rome."},{"key":"5039_CR5","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2023.1270749","author":"F Farhat","year":"2024","unstructured":"Farhat, F., Silva, E. S., Hassani, H., Madsen, D. O., Sohail, S. S., Himeur, Y., & Zafar, A. (2024). The scholarly footprint of ChatGPT: A bibliometric analysis of the early outbreak phase. Frontiers in Artificial Intelligence. https:\/\/doi.org\/10.3389\/frai.2023.1270749","journal-title":"Frontiers in Artificial Intelligence"},{"key":"5039_CR6","volume-title":"Normierter Vergleich der im Pakt f\u00fcr Forschung und Innovation gef\u00f6rderten Organisationen mit internationalen Vergleichsorganisationen anhand ausgew\u00e4hlter Indikatoren","author":"R Frietsch","year":"2022","unstructured":"Frietsch, R., Gruber, S., & Rothengatter, O. (2022). Normierter Vergleich der im Pakt f\u00fcr Forschung und Innovation gef\u00f6rderten Organisationen mit internationalen Vergleichsorganisationen anhand ausgew\u00e4hlter Indikatoren. Fraunhofer ISI."},{"issue":"1","key":"5039_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11019-023-10136-0","volume":"26","author":"B Gordijn","year":"2023","unstructured":"Gordijn, B., & ten Have, H. (2023). ChatGPT: Evolution or revolution? Medicine Health Care and Philosophy, 26(1), 1\u20132. https:\/\/doi.org\/10.1007\/s11019-023-10136-0","journal-title":"Medicine Health Care and Philosophy"},{"issue":"1","key":"5039_CR8","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1108\/Cemj-02-2023-0091","volume":"31","author":"P Korzynski","year":"2023","unstructured":"Korzynski, P., Mazurek, G., Altmann, A., Ejdys, J., Kazlauskaite, R., Paliszkiewicz, J., Wach, K., & Ziemba, E. (2023). Generative artificial intelligence as a new context for management theories: Analysis of ChatGPT. Central European Management Journal, 31(1), 3\u201313. https:\/\/doi.org\/10.1108\/Cemj-02-2023-0091","journal-title":"Central European Management Journal"},{"key":"5039_CR9","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1080\/21568235.2022.2094817","volume":"12","author":"B Lepori","year":"2022","unstructured":"Lepori, B., Borden, V. M. H., & Coates, H. (2022). Opportunities and challenges for international institutional data comparisons. European Journal of Higher Education, 12, 373\u2013390. https:\/\/doi.org\/10.1080\/21568235.2022.2094817","journal-title":"European Journal of Higher Education"},{"key":"5039_CR10","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-023-02353-2","author":"B Lepori","year":"2023","unstructured":"Lepori, B., Lambrechts, A. A., Wagner-Schuster, D., & Zahradnik, G. (2023). The European Tertiary Education Register, the reference dataset on european higher education institutions. Scientific Data. https:\/\/doi.org\/10.1038\/s41597-023-02353-2","journal-title":"Scientific Data"},{"key":"5039_CR11","doi-asserted-by":"publisher","DOI":"10.1108\/LHTN-01-2023-0009","author":"B Lund","year":"2023","unstructured":"Lund, B., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News. https:\/\/doi.org\/10.1108\/LHTN-01-2023-0009","journal-title":"Library Hi Tech News"},{"issue":"2","key":"5039_CR12","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1002\/(sici)1097-4571(1999)50:2<115::Aid-asi3>3.3.Co;2-a","volume":"50","author":"ECM Noyons","year":"1999","unstructured":"Noyons, E. C. M., Moed, H. F., & Luwel, M. (1999). Combining mapping and citation analysis for evaluative bibliometric purposes: A bibliometric study. Journal of the American Society for Information Science, 50(2), 115\u2013131. https:\/\/doi.org\/10.1002\/(sici)1097-4571(1999)50:2%3c115::Aid-asi3%3e3.3.Co;2-a","journal-title":"Journal of the American Society for Information Science"},{"issue":"9","key":"5039_CR13","doi-asserted-by":"publisher","first-page":"5783","DOI":"10.3390\/app13095783","volume":"13","author":"MM Rahman","year":"2023","unstructured":"Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13(9), 5783. https:\/\/doi.org\/10.3390\/app13095783","journal-title":"Applied Sciences"},{"key":"5039_CR14","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1016\/j.iotcps.2023.04.003","volume":"3","author":"PP Ray","year":"2023","unstructured":"Ray, P. P. (2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3, 121\u2013154. https:\/\/doi.org\/10.1016\/j.iotcps.2023.04.003","journal-title":"Internet of Things and Cyber-Physical Systems"},{"issue":"1","key":"5039_CR15","doi-asserted-by":"publisher","first-page":"713","DOI":"10.1007\/s11192-023-04882-4","volume":"129","author":"FE Sandnes","year":"2024","unstructured":"Sandnes, F. E. (2024). Can we identify prominent scholars using ChatGPT? Scientometrics, 129(1), 713\u2013718. https:\/\/doi.org\/10.1007\/s11192-023-04882-4","journal-title":"Scientometrics"},{"key":"5039_CR16","unstructured":"Thelwall, M. (2024). Can ChatGPT evaluate research quality? Retrieved 3 Apr 2024, from https:\/\/arxiv.org\/abs\/2402.05519"},{"key":"5039_CR17","doi-asserted-by":"crossref","unstructured":"van Eck, N. J., & Waltman, L. (2024). An open approach for classifying research publications. Retrieved 22 Feb 2024, from https:\/\/www.leidenmadtrics.nl\/articles\/an-open-approach-for-classifying-research-publications","DOI":"10.59350\/qc0px-76778"},{"key":"5039_CR18","volume-title":"Multidimensional ranking: The design and development of U-Multirank","year":"2012","unstructured":"van Vught, F. A., & Ziegele, F. (Eds.). (2012). Multidimensional ranking: The design and development of U-Multirank. Springer."},{"issue":"12","key":"5039_CR19","doi-asserted-by":"publisher","first-page":"7557","DOI":"10.1007\/s11192-022-04413-7","volume":"127","author":"Q Wang","year":"2022","unstructured":"Wang, Q., & Jeppsson, T. (2022). Identifying benchmark units for research management and evaluation. Scientometrics, 127(12), 7557\u20137574. https:\/\/doi.org\/10.1007\/s11192-022-04413-7","journal-title":"Scientometrics"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-024-05039-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-024-05039-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-024-05039-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,3]],"date-time":"2024-07-03T02:04:28Z","timestamp":1719972268000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-024-05039-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,29]]},"references-count":19,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["5039"],"URL":"https:\/\/doi.org\/10.1007\/s11192-024-05039-7","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,29]]},"assertion":[{"value":"3 April 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 April 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 April 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Lutz Bornmann is Editorial Board Member of <i>Scientometrics<\/i>. No funding was received for writing this paper. The authors declare they have no financial interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interest"}}]}}