{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T20:41:50Z","timestamp":1776112910474,"version":"3.50.1"},"reference-count":42,"publisher":"Emerald","issue":"1","license":[{"start":{"date-parts":[[2022,5,16]],"date-time":"2022-05-16T00:00:00Z","timestamp":1652659200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JD"],"published-print":{"date-parts":[[2023,1,10]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title><jats:p>This study aims to assess the validity of citation metrics based on the disciplinary representative survey.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title><jats:p>The present project compared citation rankings for individual scientists with expert judgments collected through a survey of 818 Russian sociologists. The Russian Index of Science Citation was used to construct the general population of 3,689 Russian sociologists, to whom the survey was sent by email. The regression analyses of bibliometric indicators and peer review scores for 723 names of scholars mentioned in the survey have been undertaken.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Findings<\/jats:title><jats:p>Findings suggest that scientometric indicators predict with significant accuracy the names of the most influential sociologists and those scholars who are not mentioned while they are less relevant for prediction names which received moderate attention in the survey.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title><jats:p>This study contributes to the research on the validity of citation metrics by focusing on scientometric indicators, not limited to traditional metrics but including non-standard publication metrics and indicators of potential metric abuse. Besides, the study presents the national bibliometric data source that is especially important for non-Western higher education systems, less presented in the Web of Science or Scopus.<\/jats:p><\/jats:sec>","DOI":"10.1108\/jd-02-2022-0039","type":"journal-article","created":{"date-parts":[[2022,5,13]],"date-time":"2022-05-13T10:53:32Z","timestamp":1652439212000},"page":"127-143","source":"Crossref","is-referenced-by-count":13,"title":["Expert judgments versus publication-based metrics: do\u00a0the\u00a0two methods produce identical results in measuring academic reputation?"],"prefix":"10.1108","volume":"79","author":[{"given":"Katerina","family":"Guba","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3261-7393","authenticated-orcid":false,"given":"Angelika","family":"Tsivinskaya","sequence":"additional","affiliation":[]}],"member":"140","published-online":{"date-parts":[[2022,5,16]]},"reference":[{"issue":"2","key":"key2023012512322640500_ref001","doi-asserted-by":"publisher","first-page":"646","DOI":"10.1016\/j.joi.2016.04.006","article-title":"A farewell to the MNCS and like size-independent indicators","volume":"10","year":"2016","journal-title":"Journal of Informetrics"},{"issue":"1","key":"key2023012512322640500_ref002","doi-asserted-by":"publisher","first-page":"33","DOI":"10.3152\/147154404781776563","article-title":"Peer reviews and bibliometric indicators: a comparative study at Norwegian university","volume":"13","year":"2004","journal-title":"Research Evaluation"},{"issue":"1","key":"key2023012512322640500_ref003","doi-asserted-by":"publisher","DOI":"10.1177\/2158244019829575","article-title":"Citations, citation indicators, and research quality: an overview of basic concepts and theories","volume":"9","year":"2019","journal-title":"Sage Open"},{"issue":"6","key":"key2023012512322640500_ref004","doi-asserted-by":"publisher","first-page":"822","DOI":"10.1016\/j.respol.2010.03.003","article-title":"University research funding and publication performance \u2014 an international comparison","volume":"39","year":"2010","journal-title":"Research Policy"},{"issue":"1","key":"key2023012512322640500_ref005","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1108\/00220410810844150","article-title":"What do citation counts measure? A review of studies on citing behavior","volume":"64","year":"2008","journal-title":"Journal of Documentation"},{"issue":"2","key":"key2023012512322640500_ref006","doi-asserted-by":"publisher","first-page":"408","DOI":"10.1016\/j.joi.2015.01.006","article-title":"Methods for the generation of normalized citation impact scores in bibliometrics: which method best reflects the judgements of experts?","volume":"9","year":"2015","journal-title":"Journal of Informetrics"},{"issue":"1","key":"key2023012512322640500_ref007","first-page":"28","article-title":"The Nature of scientific communication and influence","volume":"12","year":"1970","journal-title":"International Social Science Journal"},{"key":"key2023012512322640500_ref008","doi-asserted-by":"publisher","first-page":"231","DOI":"10.3152\/095820207X257030","article-title":"Testing novel quantitative indicators of research \u2018quality\u2019, esteem and \u2018user engagement\u2019: an economics pilot study","volume":"164","year":"2007","journal-title":"Research Evaluation"},{"issue":"3","key":"key2023012512322640500_ref009","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/0148-6195(91)90024-Q","article-title":"Why economists rank their journals the way they do","volume":"43","year":"1991","journal-title":"Journal of Economics and Business"},{"key":"key2023012512322640500_ref010","volume-title":"Engines of Anxiety: Academic Rankings, Reputation, and Accountability","year":"2016"},{"issue":"2","key":"key2023012512322640500_ref011","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1016\/j.joi.2010.12.002","article-title":"The first Italian research assessment exercise: a\u00a0bibliometric perspective","volume":"5","year":"2011","journal-title":"Journal of Informetrics"},{"issue":"3","key":"key2023012512322640500_ref012","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1177\/1440783304046370","article-title":"Why are the most influential books in Australian sociology not necessarily the most highly cited ones?","volume":"40","year":"2004","journal-title":"Journal of Sociology"},{"key":"key2023012512322640500_ref013","first-page":"489","article-title":"Peer Review versus bibliometrics. Do the two methods produce identical results in measuring academic reputation?","year":"2021"},{"issue":"2","key":"key2023012512322640500_ref014","doi-asserted-by":"publisher","first-page":"1053","DOI":"10.1007\/s11192-018-2890-8","article-title":"Boosting research productivity in top Russian universities: the circumstances of breakthrough","volume":"117","year":"2018","journal-title":"Scientometrics"},{"issue":"3","key":"key2023012512322640500_ref015","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1093\/reseval\/rvx018","article-title":"Indicators as judgment devices. An empirical study of citizen bibliometrics in research evaluation","volume":"26","year":"2017","journal-title":"Research Evaluation"},{"issue":"2","key":"key2023012512322640500_ref016","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1016\/j.respol.2011.09.007","article-title":"Performance-based university research funding systems","volume":"41","year":"2012","journal-title":"Research Policy"},{"issue":"5","key":"key2023012512322640500_ref017","doi-asserted-by":"publisher","first-page":"1162","DOI":"10.1108\/JD-02-2019-0033","article-title":"Russian science citation index on the WoS platform: a critical assessment","volume":"75","year":"2019","journal-title":"Journal of Documentation"},{"issue":"1","key":"key2023012512322640500_ref018","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1093\/reseval\/rvaa032","article-title":"The role of metrics in peer assessments","volume":"30","year":"2021","journal-title":"Research Evaluation"},{"issue":"4","key":"key2023012512322640500_ref019","doi-asserted-by":"publisher","first-page":"554","DOI":"10.1016\/j.joi.2010.06.005","article-title":"Ranking of library and information science researchers. Comparison of data sources for correlating citation data, and expert judgments","volume":"4","year":"2010","journal-title":"Journal of Informetrics"},{"issue":"2","key":"key2023012512322640500_ref020","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1080\/00987913.2017.1290483","article-title":"Correlation between perception-based journal rankings and the Journal Impact Factor (JIF). A systematic review and meta-analysis","volume":"43","year":"2017","journal-title":"Serials Review"},{"issue":"2","key":"key2023012512322640500_ref021","doi-asserted-by":"publisher","first-page":"1153","DOI":"10.1007\/s11192-018-2769-8","article-title":"Trends in Russian research output indexed in Scopus and Web of science","volume":"116","year":"2018","journal-title":"Scientometrics"},{"issue":"1","key":"key2023012512322640500_ref022","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1007\/s11192-015-1765-5","article-title":"The journal coverage of Web of Science and Scopus: a comparative analysis","volume":"106","year":"2016","journal-title":"Scientometrics"},{"issue":"5","key":"key2023012512322640500_ref023","doi-asserted-by":"publisher","first-page":"626","DOI":"10.1177\/0011392113498866","article-title":"The globalization of social sciences? Evidence from a quantitative analysis of 30 years of production, collaboration and citations in the social sciences (1980-2009)","volume":"62","year":"2014","journal-title":"Current Sociology"},{"issue":"1","key":"key2023012512322640500_ref024","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-018-2758-y","article-title":"Russian index of science CitationOverview and review","volume":"116","year":"2018","journal-title":"Scientometrics"},{"key":"key2023012512322640500_ref025","volume-title":"The Tyranny of Metrics","year":"2018"},{"issue":"1","key":"key2023012512322640500_ref026","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1177\/144078330303900106","article-title":"The validity of publication and citation counts for sociology and other selected disciplines","volume":"39","year":"2003","journal-title":"Journal of Sociology"},{"issue":"3","key":"key2023012512322640500_ref027","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1016\/j.joi.2009.11.001","article-title":"Peer review and the h-index: two studies","volume":"4","year":"2010","journal-title":"Journal of Informetrics"},{"issue":"2","key":"key2023012512322640500_ref028","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1093\/reseval\/rvx006","article-title":"Opportunities and opportunism: publication outlet selection under pressure to increase research productivity","volume":"26","year":"2017","journal-title":"Research Evaluation"},{"issue":"1","key":"key2023012512322640500_ref029","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1108\/eb026940","article-title":"The correlation between citation counts and the 1992 Research Assessment Exercise ratings for British library and information science university departments","volume":"51","year":"1995","journal-title":"Journal of Documentation"},{"issue":"5","key":"key2023012512322640500_ref030","doi-asserted-by":"publisher","first-page":"477","DOI":"10.1108\/EUM000000000720","article-title":"The correlation between citation counts and the 1992 research assessment exercise ratings for British research in genetics, anatomy and archaeology","volume":"53","year":"1997","journal-title":"Journal of Documentation"},{"issue":"3","key":"key2023012512322640500_ref031","doi-asserted-by":"publisher","first-page":"2131","DOI":"10.1007\/s11192-014-1510-5","article-title":"Globalization of the social sciences in Eastern Europe: genuine breakthrough or a slippery slope of the research evaluation practice?","volume":"102","year":"2015","journal-title":"Scientometrics"},{"issue":"2","key":"key2023012512322640500_ref032","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1093\/reseval\/rvv038","article-title":"Evaluation practices and effects of indicator use\u2014a literature review","volume":"25","year":"2016","journal-title":"Research Evaluation"},{"key":"key2023012512322640500_ref033","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1017\/S1049096500059916","article-title":"Reputations vs citations. Who are the top scholars in political science?","volume":"15","year":"1982","journal-title":"APSC"},{"key":"key2023012512322640500_ref034","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1017\/S1049096500003826","article-title":"Strata and stability. Reputations of American political scientists","volume":"11","year":"1978","journal-title":"APSC"},{"issue":"2","key":"key2023012512322640500_ref035","doi-asserted-by":"publisher","first-page":"478","DOI":"10.1016\/j.respol.2017.12.004","article-title":"Self-citations as strategic response to the use of metrics for career decisions","volume":"48","year":"2019","journal-title":"Research Policy"},{"issue":"4","key":"key2023012512322640500_ref036","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1016\/j.joi.2011.06.002","article-title":"Comparing the expert survey and citation impact journal ranking methods. Example from the field of Artificial Intelligence","volume":"5","year":"2011","journal-title":"Journal of Informetrics"},{"issue":"3","key":"key2023012512322640500_ref037","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1007\/bf02459049","article-title":"Citation ranking versus expert judgment in evaluating communication scholars. Effects of research specialty size and individual prominence","volume":"41","year":"1998","journal-title":"Scientometrics"},{"issue":"6","key":"key2023012512322640500_ref038","doi-asserted-by":"publisher","first-page":"818","DOI":"10.1177\/0011392118811392","article-title":"The sources of academic localism and globalism in Russian sociology: the choice of professional ideologies and occupational niches among social scientists","volume":"67","year":"2019","journal-title":"Current Sociology"},{"issue":"6","key":"key2023012512322640500_ref039","doi-asserted-by":"publisher","first-page":"989","DOI":"10.1080\/09668136.2021.1902945","article-title":"Can the Russian research policy be called neo-liberal? A study in the comparative sociology of quantification","volume":"73","year":"2021","journal-title":"Europe-Asia Studies"},{"issue":"3","key":"key2023012512322640500_ref040","doi-asserted-by":"publisher","first-page":"1635","DOI":"10.1007\/s11192-019-03243-4","article-title":"What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018","volume":"121","year":"2019","journal-title":"Scientometrics"},{"issue":"1","key":"key2023012512322640500_ref041","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1007\/BF02458401","article-title":"Matters of quantitative studies of science the fault of theorists is offering too little and asking too much","volume":"43","year":"1998","journal-title":"Scientometrics"},{"issue":"2","key":"key2023012512322640500_ref043","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1007\/s11192-013-0969-9","article-title":"Correlations between bibliometrics and peer evaluation for all disciplines. The evaluation of Brazilian scientists","volume":"96","year":"2013","journal-title":"Scientometrics"}],"container-title":["Journal of Documentation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/JD-02-2022-0039\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/JD-02-2022-0039\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T22:33:25Z","timestamp":1753396405000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jd\/article\/79\/1\/127-143\/203050"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,16]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,5,16]]},"published-print":{"date-parts":[[2023,1,10]]}},"alternative-id":["10.1108\/JD-02-2022-0039"],"URL":"https:\/\/doi.org\/10.1108\/jd-02-2022-0039","relation":{},"ISSN":["0022-0418"],"issn-type":[{"value":"0022-0418","type":"print"}],"subject":[],"published":{"date-parts":[[2022,5,16]]}}}