{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T07:32:08Z","timestamp":1775460728254,"version":"3.50.1"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2022,7,21]],"date-time":"2022-07-21T00:00:00Z","timestamp":1658361600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springer.com\/tdm"},{"start":{"date-parts":[[2022,7,21]],"date-time":"2022-07-21T00:00:00Z","timestamp":1658361600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2022,8]]},"DOI":"10.1007\/s11192-022-04469-5","type":"journal-article","created":{"date-parts":[[2022,7,21]],"date-time":"2022-07-21T21:02:37Z","timestamp":1658437357000},"page":"4609-4630","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Reasons and consequences of changes in Russian research assessment policies"],"prefix":"10.1007","volume":"127","author":[{"given":"Denis","family":"Kosyakov","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1028-9958","authenticated-orcid":false,"given":"Andrey","family":"Guskov","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,7,21]]},"reference":[{"issue":"7300","key":"4469_CR1","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1038\/465860a","volume":"465","author":"A Abbott","year":"2010","unstructured":"Abbott, A., Cyranoski, D., Jones, N., Maher, B., Schiermeier, Q., & Van Noorden, R. (2010). Metrics: Do metrics matter? Nature, 465(7300), 860\u2013862. https:\/\/doi.org\/10.1038\/465860a","journal-title":"Nature"},{"key":"4469_CR2","doi-asserted-by":"publisher","DOI":"10.1016\/j.joi.2017.09.007","author":"G Abramo","year":"2017","unstructured":"Abramo, G., D\u2019Angelo, A. C., & Murgia, G. (2017). The relationship among research productivity, research collaboration, and their determinants. Journal of Informetrics. https:\/\/doi.org\/10.1016\/j.joi.2017.09.007","journal-title":"Journal of Informetrics"},{"issue":"3","key":"4469_CR3","doi-asserted-by":"publisher","first-page":"779","DOI":"10.1007\/s11192-013-1013-9","volume":"97","author":"G Abramo","year":"2013","unstructured":"Abramo, G., D\u2019Angelo, C. A., & Rosati, F. (2013). Measuring institutional research productivity for the life sciences: The importance of accounting for the order of authors in the byline. Scientometrics, 97(3), 779\u2013795. https:\/\/doi.org\/10.1007\/s11192-013-1013-9","journal-title":"Scientometrics"},{"key":"4469_CR4","doi-asserted-by":"publisher","DOI":"10.1016\/j.joi.2010.11.002","author":"G Abramo","year":"2011","unstructured":"Abramo, G., D\u2019Angelo, C. A., & Solazzi, M. (2011). Are researchers that collaborate more at the international level top performers? An investigation on the Italian university system. Journal of Informetrics. https:\/\/doi.org\/10.1016\/j.joi.2010.11.002","journal-title":"Journal of Informetrics"},{"key":"4469_CR5","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-007-1686-z","author":"HA Abt","year":"2007","unstructured":"Abt, H. A. (2007). The frequencies of multinational papers in various sciences. Scientometrics. https:\/\/doi.org\/10.1007\/s11192-007-1686-z","journal-title":"Scientometrics"},{"issue":"2","key":"4469_CR6","doi-asserted-by":"publisher","first-page":"296","DOI":"10.4103\/1008-682X.171582","volume":"18","author":"A Agarwal","year":"2016","unstructured":"Agarwal, A., Durairajanayagam, D., Tatagari, S., Esteves, S. C., Harlev, A., Henkel, R., Roychoudhury, S., Homa, S., Puchalt, N. G., Ramasamy, R., Majzoub, A., Dao Ly, K., Tvrda, E., Assidi, M., Kesari, K., Sharma, R., Banihani, S., Ko, E., Abu-Elmagd, M., \u2026 Bashiri, A. (2016). Bibliometrics: Tracking research impact by selecting the appropriate metrics. Asian Journal of Andrology, 18(2), 296\u2013309. https:\/\/doi.org\/10.4103\/1008-682X.171582","journal-title":"Asian Journal of Andrology"},{"issue":"1","key":"4469_CR7","first-page":"95","volume":"31","author":"NA Ascheulova","year":"2010","unstructured":"Ascheulova, N. A., & Kolchinsky, E. I. (2010). The reform of science in Russia (a historico-sociological analysis). Voprosy Istorii Estestvoznaniia I Tekhniki [studies in the History of Science and Technology], 31(1), 95\u2013119.","journal-title":"Voprosy Istorii Estestvoznaniia I Tekhniki [studies in the History of Science and Technology]"},{"issue":"6061","key":"4469_CR8","doi-asserted-by":"publisher","first-page":"1344","DOI":"10.1126\/science.334.6061.1344","volume":"334","author":"Y Bhattacharjee","year":"2011","unstructured":"Bhattacharjee, Y. (2011). Saudi universities offer cash in exchange for academic prestige. Science, 334(6061), 1344\u20131345. https:\/\/doi.org\/10.1126\/science.334.6061.1344","journal-title":"Science"},{"key":"4469_CR9","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-09785-5_2","volume-title":"Incentives and performance: Governance of research organizations","author":"M Binswanger","year":"2015","unstructured":"Binswanger, M. (2015). How nonsense became excellence: Forcing professors to publish. In I. M. Welpe, J. Wollersheim, S. Ringelhan, & M. Osterloh (Eds.), Incentives and performance: Governance of research organizations. Cham: Springer International Publishing. https:\/\/doi.org\/10.1007\/978-3-319-09785-5_2"},{"issue":"2","key":"4469_CR10","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1108\/OIR-08-2015-0276","volume":"42","author":"L Bornmann","year":"2018","unstructured":"Bornmann, L. (2018). Which research institution performs better than average in a subject category or better than selected other institutions? Online Information Review, 42(2), 222\u2013237. https:\/\/doi.org\/10.1108\/OIR-08-2015-0276","journal-title":"Online Information Review"},{"issue":"10","key":"4469_CR11","doi-asserted-by":"publisher","first-page":"8701","DOI":"10.1007\/s11192-021-04124-5","volume":"126","author":"M Chankseliani","year":"2021","unstructured":"Chankseliani, M., Lovakov, A., & Pislyakov, V. (2021). A big picture: Bibliometric study of academic publications from post-Soviet countries. Scientometrics, 126(10), 8701\u20138730. https:\/\/doi.org\/10.1007\/s11192-021-04124-5","journal-title":"Scientometrics"},{"issue":"11","key":"4469_CR12","doi-asserted-by":"publisher","first-page":"1011","DOI":"10.1037\/h0024051","volume":"21","author":"DJ De Solla Price","year":"1966","unstructured":"De Solla Price, D. J., & Beaver, D. (1966). Collaboration in an invisible college. American Psychologist, 21(11), 1011\u20131018. https:\/\/doi.org\/10.1037\/h0024051","journal-title":"American Psychologist"},{"issue":"2","key":"4469_CR13","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1002\/(SICI)1097-4571(2000)51:2<145::AID-ASI6>3.0.CO;2-9","volume":"51","author":"L Egghe","year":"2000","unstructured":"Egghe, L., Rousseau, R., & Hooydonk, G. V. (2000). Methods for accrediting publications to authors or countries: Consequences for evaluation studies. Journal of the American Society for Information Science, 51(2), 145\u2013157. https:\/\/doi.org\/10.1002\/(SICI)1097-4571(2000)51:2%3c145::AID-ASI6%3e3.0.CO;2-9","journal-title":"Journal of the American Society for Information Science"},{"key":"4469_CR14","unstructured":"Erokhina, E. (2019). Russian Science in Scopus and WoS - Quality or Quantity? Indicator.ru.  Retrieved August 29, 2021, from https:\/\/indicator.ru\/engineering-science\/rossijskaya-nauka-v-scopus-i-wos-kolichestvo-ili-kachestvo.htm"},{"key":"4469_CR15","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0010271","author":"D Fanelli","year":"2010","unstructured":"Fanelli, D. (2010). Do pressures to publish increase scientists\u2019 bias? An empirical support from US states data. PLoS ONE. https:\/\/doi.org\/10.1371\/journal.pone.0010271","journal-title":"PLoS ONE"},{"key":"4469_CR16","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-016-1876-7","author":"A Guskov","year":"2016","unstructured":"Guskov, A., Kosyakov, D., & Selivanova, I. (2016). Scientometric research in Russia: Impact of science policy changes. Scientometrics. https:\/\/doi.org\/10.1007\/s11192-016-1876-7","journal-title":"Scientometrics"},{"issue":"2","key":"4469_CR17","doi-asserted-by":"publisher","first-page":"1053","DOI":"10.1007\/s11192-018-2890-8","volume":"117","author":"AE Guskov","year":"2018","unstructured":"Guskov, A. E., Kosyakov, D. V., & Selivanova, I. V. (2018). Boosting research productivity in top Russian universities: The circumstances of breakthrough. Scientometrics, 117(2), 1053\u20131080. https:\/\/doi.org\/10.1007\/s11192-018-2890-8","journal-title":"Scientometrics"},{"issue":"5","key":"4469_CR18","doi-asserted-by":"publisher","first-page":"470","DOI":"10.1097\/00001888-199905000-00009","volume":"74","author":"EC Halperin","year":"1999","unstructured":"Halperin, E. C. (1999). Publish or perish\u2014And bankrupt the medical library while we\u2019re at it. Academic Medicine, 74(5), 470\u2013472. https:\/\/doi.org\/10.1097\/00001888-199905000-00009","journal-title":"Academic Medicine"},{"key":"4469_CR19","doi-asserted-by":"publisher","DOI":"10.1002\/asi.24472","author":"H Hottenrott","year":"2021","unstructured":"Hottenrott, H., Rose, M. E., & Lawson, C. (2021). The rise of multiple institutional affiliations in academia. Journal of the Association for Information Science and Technology. https:\/\/doi.org\/10.1002\/asi.24472","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"12","key":"4469_CR20","doi-asserted-by":"publisher","first-page":"2427","DOI":"10.1002\/asi.21625","volume":"62","author":"MH Huang","year":"2011","unstructured":"Huang, M. H., Lin, C. S., & Chen, D. Z. (2011). Counting methods, country rank changes, and counting inflation in the assessment of national research productivity and impact. Journal of the American Society for Information Science and Technology, 62(12), 2427\u20132436. https:\/\/doi.org\/10.1002\/asi.21625","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"7722","key":"4469_CR21","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1038\/d41586-018-06185-8","volume":"561","author":"JPA Ioannidis","year":"2018","unstructured":"Ioannidis, J. P. A., Klavans, R., & Boyack, K. W. (2018). Thousands of scientists publish a paper every five days. Nature, 561(7722), 167\u2013169. https:\/\/doi.org\/10.1038\/d41586-018-06185-8","journal-title":"Nature"},{"key":"4469_CR22","unstructured":"Kosyakov, D., & Guskov, A. (2019c). Synchronous scientific mobility and international collaboration: Case of Russia. In 17th International Conference on Scientometrics and Informetrics, ISSI 2019c\u2014Proceedings (Vol. 1, pp. 1319\u20131328)."},{"key":"4469_CR23","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.procs.2019.01.080","volume":"146","author":"D Kosyakov","year":"2019","unstructured":"Kosyakov, D., & Guskov, A. (2019a). Impact of national science policy on academic migration and research productivity in Russia. Procedia Computer Science, 146, 60\u201371. https:\/\/doi.org\/10.1016\/j.procs.2019.01.080","journal-title":"Procedia Computer Science"},{"key":"4469_CR24","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.procs.2019.01.072","volume":"146","author":"D Kosyakov","year":"2019","unstructured":"Kosyakov, D., & Guskov, A. (2019b). Research assessment and evaluation in Russian fundamental science. Procedia Computer Science, 146, 11\u201319. https:\/\/doi.org\/10.1016\/j.procs.2019.01.072","journal-title":"Procedia Computer Science"},{"key":"4469_CR25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0162709","author":"V Larivi\u00e8re","year":"2016","unstructured":"Larivi\u00e8re, V., & Costas, R. (2016). How many is too many? On the relationship between research productivity and impact. PLoS ONE. https:\/\/doi.org\/10.1371\/journal.pone.0162709","journal-title":"PLoS ONE"},{"issue":"1","key":"4469_CR26","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1016\/j.joi.2016.11.007","volume":"11","author":"L Leydesdorff","year":"2017","unstructured":"Leydesdorff, L., & Park, H. W. (2017). Full and fractional counting in bibliometric networks. Journal of Informetrics, 11(1), 117\u2013120. https:\/\/doi.org\/10.1016\/j.joi.2016.11.007","journal-title":"Journal of Informetrics"},{"issue":"3","key":"4469_CR27","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1016\/j.joi.2013.03.007","volume":"7","author":"CS Lin","year":"2013","unstructured":"Lin, C. S., Huang, M. H., & Chen, D. Z. (2013). The influences of counting methods on university rankings based on paper count and citation count. Journal of Informetrics, 7(3), 611\u2013621. https:\/\/doi.org\/10.1016\/j.joi.2013.03.007","journal-title":"Journal of Informetrics"},{"key":"4469_CR28","unstructured":"Mach\u00e1\u010dek, V., & Srholec, M. (2019). Predatory publications in Scopus: Evidence on cross-country differences. 1, 351\u2013362. Scopus."},{"issue":"2","key":"4469_CR29","doi-asserted-by":"publisher","first-page":"813","DOI":"10.1007\/s11192-016-2048-5","volume":"109","author":"EJT Manganote","year":"2016","unstructured":"Manganote, E. J. T., Schulz, P. A., & de Brito Cruz, C. H. (2016). Effect of high energy physics large collaborations on higher education institutions citations and rankings. Scientometrics, 109(2), 813\u2013826. https:\/\/doi.org\/10.1007\/s11192-016-2048-5","journal-title":"Scientometrics"},{"issue":"6","key":"4469_CR30","doi-asserted-by":"publisher","first-page":"5019","DOI":"10.1007\/s11192-021-03899-x","volume":"126","author":"T Marina","year":"2021","unstructured":"Marina, T., & Sterligov, I. (2021). Prevalence of potentially predatory publishing in Scopus on the country level. Scientometrics, 126(6), 5019\u20135077.","journal-title":"Scientometrics"},{"issue":"1","key":"4469_CR31","doi-asserted-by":"publisher","first-page":"101110","DOI":"10.1016\/j.joi.2020.101110","volume":"15","author":"N Matveeva","year":"2021","unstructured":"Matveeva, N., Sterligov, I., & Yudkevich, M. (2021). The effect of Russian University Excellence Initiative on publications and collaboration patterns. Journal of Informetrics, 15(1), 101110. https:\/\/doi.org\/10.1016\/j.joi.2020.101110","journal-title":"Journal of Informetrics"},{"key":"4469_CR32","doi-asserted-by":"publisher","unstructured":"Moed, H. F. (2005). Citation analysis in research evaluation. In proceedings of ISSI 2005: 10th international conference of the international society for scientometrics and informetrics, (Vol. 2, pp. 437\u2013441). https:\/\/doi.org\/10.1007\/1-4020-3714-7","DOI":"10.1007\/1-4020-3714-7"},{"issue":"10","key":"4469_CR33","doi-asserted-by":"publisher","first-page":"1988","DOI":"10.1002\/asi.23314","volume":"66","author":"HF Moed","year":"2015","unstructured":"Moed, H. F., & Halevi, G. (2015). Multidimensional assessment of scholarly research impact. Journal of the Association for Information Science and Technology, 66(10), 1988\u20132002. https:\/\/doi.org\/10.1002\/asi.23314","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"2","key":"4469_CR34","doi-asserted-by":"publisher","first-page":"1153","DOI":"10.1007\/s11192-018-2769-8","volume":"116","author":"HF Moed","year":"2018","unstructured":"Moed, H. F., Markusova, V., & Akoev, M. (2018). Trends in Russian research output indexed in scopus and web of science. Scientometrics, 116(2), 1153\u20131180. https:\/\/doi.org\/10.1007\/s11192-018-2769-8","journal-title":"Scientometrics"},{"issue":"23","key":"4469_CR35","first-page":"53","volume":"1","author":"SL Parfenova","year":"2017","unstructured":"Parfenova, S. L., Grishakina, E. G., Zolotarev, D. V., & Bogatov, V. V. (2017). Publication landscape of the Russian science. Science Governance and Scientometrics, 1(23), 53\u201379.","journal-title":"Science Governance and Scientometrics"},{"issue":"4","key":"4469_CR36","doi-asserted-by":"publisher","first-page":"1178","DOI":"10.1016\/j.joi.2016.10.006","volume":"10","author":"A Perianes-Rodriguez","year":"2016","unstructured":"Perianes-Rodriguez, A., Waltman, L., & van Eck, N. J. (2016). Constructing bibliometric networks: A comparison between full and fractional counting. Journal of Informetrics, 10(4), 1178\u20131195. https:\/\/doi.org\/10.1016\/j.joi.2016.10.006","journal-title":"Journal of Informetrics"},{"issue":"3","key":"4469_CR37","doi-asserted-by":"publisher","first-page":"204","DOI":"10.3103\/S0147688219030109","volume":"46","author":"IV Selivanova","year":"2019","unstructured":"Selivanova, I. V., Kosyakov, D. V., & Guskov, A. E. (2019). The impact of errors in the scopus database on the research assessment. Scientific and Technical Information Processing, 46(3), 204\u2013212. https:\/\/doi.org\/10.3103\/S0147688219030109","journal-title":"Scientific and Technical Information Processing"},{"issue":"2","key":"4469_CR38","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1016\/j.joi.2019.03.010","volume":"13","author":"G Sivertsen","year":"2019","unstructured":"Sivertsen, G., Rousseau, R., & Zhang, L. (2019). Measuring scientific contributions with modified fractional counting. Journal of Informetrics, 13(2), 679\u2013694. https:\/\/doi.org\/10.1016\/j.joi.2019.03.010","journal-title":"Journal of Informetrics"},{"issue":"1","key":"4469_CR39","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1007\/s11192-020-03372-1","volume":"123","author":"S Tarkhan-Mouravi","year":"2020","unstructured":"Tarkhan-Mouravi, S. (2020). Traditional indicators inflate some countries\u2019 scientific impact over 10 times. Scientometrics, 123(1), 337\u2013356. https:\/\/doi.org\/10.1007\/s11192-020-03372-1","journal-title":"Scientometrics"},{"key":"4469_CR40","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.1154","author":"E Wager","year":"2015","unstructured":"Wager, E., Singhvi, S., & Kleinert, S. (2015). Too much of a good thing? An observational study of prolific authors. PeerJ. https:\/\/doi.org\/10.7717\/peerj.1154","journal-title":"PeerJ"},{"issue":"12","key":"4469_CR41","doi-asserted-by":"publisher","first-page":"2419","DOI":"10.1002\/asi.22708","volume":"63","author":"L Waltman","year":"2012","unstructured":"Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E. C. M., Tijssen, R. J. W., van Eck, N. J., van Leeuwen, T. N., van Raan, A. F. J., Visser, M. S., & Wouters, P. (2012). The Leiden ranking 2011\/2012: Data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology, 63(12), 2419\u20132432. https:\/\/doi.org\/10.1002\/asi.22708","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"4","key":"4469_CR42","doi-asserted-by":"publisher","first-page":"872","DOI":"10.1016\/j.joi.2015.08.001","volume":"9","author":"L Waltman","year":"2015","unstructured":"Waltman, L., & van Eck, N. J. (2015). Field-normalized citation impact indicators and the choice of an appropriate counting method. Journal of Informetrics, 9(4), 872\u2013894. https:\/\/doi.org\/10.1016\/j.joi.2015.08.001","journal-title":"Journal of Informetrics"},{"key":"4469_CR43","doi-asserted-by":"publisher","DOI":"10.1126\/science.1136099","author":"S Wuchty","year":"2007","unstructured":"Wuchty, S., Jones, B. F., & Uzzi, B. (2007). The increasing dominance of teams in production of knowledge. Science. https:\/\/doi.org\/10.1126\/science.1136099","journal-title":"Science"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04469-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-022-04469-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04469-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,10]],"date-time":"2022-08-10T07:48:18Z","timestamp":1660117698000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-022-04469-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,21]]},"references-count":43,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2022,8]]}},"alternative-id":["4469"],"URL":"https:\/\/doi.org\/10.1007\/s11192-022-04469-5","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,21]]},"assertion":[{"value":"31 August 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 July 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 July 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The research did not involve human participants.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}