{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T07:43:04Z","timestamp":1784619784029,"version":"3.55.0"},"reference-count":73,"publisher":"MIT Press","issue":"4","license":[{"start":{"date-parts":[[2023,12,7]],"date-time":"2023-12-07T00:00:00Z","timestamp":1701907200000},"content-version":"vor","delay-in-days":340,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Recent years have seen a rise in awareness around \u201cresponsible metrics\u201d and calls for research assessment reforms internationally. Yet within the field of quantitative science studies and in research policy contexts, concerns about the limitations of evaluative bibliometrics are almost as old as the tools themselves. Given that many of the concerns articulated in recent reform movements go back decades, why has momentum for change grown only in the past 10 years? In this paper, we draw on analytical insights from the sociology of social movements on collective action frames to chart the emergence, development, and expansion of \u201cresponsible metrics\u201d as a professional reform movement. Through reviewing important texts that have shaped reform efforts, we argue that hitherto, three framings have underpinned the responsible metrics reform agenda: the metrics skepticism framing, the professional-expert framing, and the reflexivity framing. We suggest that although these three framings have coexisted within the responsible metrics movement to date, cohabitation between these framings may not last indefinitely, especially as the responsible metrics movement extends into wider research assessment reform movements.<\/jats:p>","DOI":"10.1162\/qss_a_00280","type":"journal-article","created":{"date-parts":[[2023,12,7]],"date-time":"2023-12-07T10:37:57Z","timestamp":1701945477000},"page":"879-897","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":33,"title":["The rise of responsible metrics as a professional reform movement: A collective action frames account"],"prefix":"10.1162","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3352-943X","authenticated-orcid":true,"given":"Alexander","family":"Rushforth","sequence":"first","affiliation":[{"name":"Centre for Science and Technology Studies (CWTS), Leiden University, Leiden, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1504-8395","authenticated-orcid":true,"given":"Bj\u00f6rn","family":"Hammarfelt","sequence":"additional","affiliation":[{"name":"Swedish School of Library and Information Science, University of Bor\u00e5s, Bor\u00e5s, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2023,11,1]]},"reference":[{"issue":"1","key":"2024022218552644800_bib1","doi-asserted-by":"publisher","DOI":"10.1177\/2158244019829575","article-title":"Citations, citation indicators, and research quality: An overview of basic concepts and theories","volume":"9","author":"Aksnes","year":"2019","journal-title":"Sage Open"},{"issue":"16","key":"2024022218552644800_bib2","doi-asserted-by":"publisher","first-page":"5773","DOI":"10.1073\/pnas.1404402111","article-title":"Rescuing US biomedical research from its systemic flaws","volume":"111","author":"Alberts","year":"2014","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2024022218552644800_bib3","article-title":"Does DORA need to attack the impact factor to reform how it is used in academia?","author":"Anderson","year":"2013","journal-title":"Scholarly Kitchen"},{"key":"2024022218552644800_bib4","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1146\/annurev.soc.26.1.611","article-title":"Framing processes and social movements: An overview and assessment","volume":"26","author":"Benford","year":"2000","journal-title":"Annual Review of Sociology"},{"issue":"1","key":"2024022218552644800_bib5","doi-asserted-by":"publisher","first-page":"12","DOI":"10.17705\/1CAIS.03412","article-title":"A hermeneutic approach for conducting literature reviews and literature searches","volume":"34","author":"Boell","year":"2014","journal-title":"Communications of the Association for Information Systems"},{"key":"2024022218552644800_bib6","article-title":"Metrics have their merits","author":"Bornmann","year":"2023","journal-title":"Research Professional"},{"key":"2024022218552644800_bib7","doi-asserted-by":"publisher","first-page":"102","DOI":"10.4337\/9781784718862.00014","article-title":"Understanding the movement(s) for responsible innovation","volume-title":"International handbook on responsible innovation","author":"Brundage","year":"2019"},{"issue":"2","key":"2024022218552644800_bib8","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1111\/j.1467-954X.2012.02077.x","article-title":"Living with the h-index? Metric assemblages in the contemporary academy","volume":"60","author":"Burrows","year":"2012","journal-title":"Sociological Review"},{"issue":"8","key":"2024022218552644800_bib9","doi-asserted-by":"publisher","first-page":"565","DOI":"10.3152\/030234207X254404","article-title":"Assessing university research: A plea for a balanced approach","volume":"34","author":"Butler","year":"2007","journal-title":"Science and Public Policy"},{"key":"2024022218552644800_bib10","doi-asserted-by":"publisher","first-page":"111","DOI":"10.6120\/JoEMLS.2017.541\/0015.OR.AM","article-title":"What have we learned from San Francisco Declaration on Research Assessment and Leiden Manifesto?","volume":"54","author":"Chen","year":"2017","journal-title":"Journal of Educational Media and Library Sciences"},{"key":"2024022218552644800_bib11","volume-title":"Peerless science: Peer review and US science policy","author":"Chubin","year":"1990"},{"key":"2024022218552644800_bib12","volume-title":"Agreement on reforming research assessment","author":"CoARA","year":"2022"},{"issue":"3","key":"2024022218552644800_bib13","doi-asserted-by":"publisher","first-page":"554","DOI":"10.1177\/030631285015003009","article-title":"The possibilities of science policy","volume":"15","author":"Collins","year":"1985","journal-title":"Social Studies of Science"},{"issue":"1","key":"2024022218552644800_bib14","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1086\/725131","article-title":"Provincializing impact: From imperial anxiety to algorithmic universalism","volume":"38","author":"Csiszar","year":"2023","journal-title":"Osiris"},{"key":"2024022218552644800_bib15","doi-asserted-by":"publisher","DOI":"10.6084\/m9.figshare.13227914.v2","article-title":"The changing role of funders in responsible research assessment: Progress, obstacles and the way ahead","author":"Curry","year":"2020","journal-title":"RoRI Working Paper No. 3"},{"key":"2024022218552644800_bib16","volume-title":"Harnessing the Metric Tide: Indicators, infrastructures & priorities for UK responsible research assessment. Report of The Metric Tide Revisited panel","author":"Curry","year":"2022"},{"key":"2024022218552644800_bib17","doi-asserted-by":"publisher","DOI":"10.1515\/9780804778121","volume-title":"The evaluation society","author":"Dahler-Larsen","year":"2012"},{"issue":"3","key":"2024022218552644800_bib18","doi-asserted-by":"publisher","first-page":"2255","DOI":"10.1007\/s11192-015-1634-2","article-title":"The lost paradise, the original sin, and the Dodo bird: A scientometrics Sapere Aude manifesto as a reply to the Leiden manifesto on scientometrics","volume":"105","author":"David","year":"2015","journal-title":"Scientometrics"},{"key":"2024022218552644800_bib19","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1007\/978-3-319-17308-5_3","article-title":"Responsible innovation in the US, UK and Denmark: Governance landscapes","volume-title":"Responsible innovation 2","author":"Davies","year":"2015"},{"key":"2024022218552644800_bib20","volume-title":"Social problems and social movements","author":"DeFronzo","year":"2019"},{"issue":"9","key":"2024022218552644800_bib21","doi-asserted-by":"publisher","first-page":"1954","DOI":"10.1002\/asi.23382","article-title":"To intervene or not to intervene; is that the question? On the role of scientometrics in research evaluation","volume":"66","author":"de Rijcke","year":"2015","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"2","key":"2024022218552644800_bib22","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1093\/reseval\/rvv038","article-title":"Evaluation practices and effects of indicator use\u2014A literature review","volume":"25","author":"de Rijcke","year":"2016","journal-title":"Research Evaluation"},{"issue":"2","key":"2024022218552644800_bib23","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1177\/10892680211015636","article-title":"The tone debate: Knowledge, self, and social order","volume":"26","author":"Derksen","year":"2022","journal-title":"Review of General Psychology"},{"key":"2024022218552644800_bib24","volume-title":"The politics of large numbers: A history of statistical reasoning","author":"Desrosi\u00e8res","year":"1998"},{"key":"2024022218552644800_bib25","volume-title":"The Declaration","author":"DORA","year":"2013"},{"issue":"1","key":"2024022218552644800_bib26","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1007\/s11569-013-0172-8","article-title":"Meta-regulation and nanotechnologies: The challenge of responsibilisation within the European Commission\u2019s code of conduct for responsible nanosciences and nanotechnologies research","volume":"7","author":"Dorbeck-Jung","year":"2013","journal-title":"Nanoethics"},{"key":"2024022218552644800_bib27","doi-asserted-by":"publisher","DOI":"10.2777\/75255","volume-title":"Evaluation of research careers fully acknowledging Open Science practices","author":"EC","year":"2017"},{"key":"2024022218552644800_bib28","volume-title":"Research assessment and implementation of Open Science\u2014Council conclusions","author":"EU","year":"2022"},{"key":"2024022218552644800_bib29","volume-title":"Cultivation devices: Sustainability as a quality","author":"Franssen","year":"2022"},{"issue":"10","key":"2024022218552644800_bib30","doi-asserted-by":"publisher","first-page":"1124","DOI":"10.1002\/asi.24206","article-title":"Science and its significant other: Representing the humanities in bibliometric scholarship","volume":"70","author":"Franssen","year":"2019","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"2","key":"2024022218552644800_bib31","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1525\/sp.2004.51.2.269","article-title":"Building an interdiscipline: Collective action framing and the rise of genetic toxicology","volume":"51","author":"Frickel","year":"2004","journal-title":"Social Problems"},{"issue":"4","key":"2024022218552644800_bib32","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1080\/0950543042000311814","article-title":"Just science? Organizing scientist activism in the US environmental justice movement","volume":"13","author":"Frickel","year":"2004","journal-title":"Science as Culture"},{"issue":"3","key":"2024022218552644800_bib33","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1093\/reseval\/rvx018","article-title":"Indicators as judgment devices: An empirical study of citizen bibliometrics in research evaluation","volume":"26","author":"Hammarfelt","year":"2017","journal-title":"Research Evaluation"},{"issue":"7548","key":"2024022218552644800_bib34","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1038\/520429a","article-title":"Bibliometrics: The Leiden Manifesto for research metrics","volume":"520","author":"Hicks","year":"2015","journal-title":"Nature"},{"issue":"3","key":"2024022218552644800_bib35","doi-asserted-by":"publisher","first-page":"930","DOI":"10.1162\/qss_a_00056","article-title":"Against method: Exploding the boundary between qualitative and quantitative studies of science","volume":"1","author":"Kang","year":"2020","journal-title":"Quantitative Science Studies"},{"issue":"12","key":"2024022218552644800_bib36","doi-asserted-by":"publisher","first-page":"9869","DOI":"10.1007\/s11192-021-04039-1","article-title":"(E-)valuative metrics as a contested field: A comparative analysis of the altmetrics\u2014and the Leiden Manifesto","volume":"126","author":"Leckert","year":"2021","journal-title":"Scientometrics"},{"key":"2024022218552644800_bib37","volume-title":"A pathway towards multidimensional academic careers: A LERU framework for the assessment of researchers","author":"LERU","year":"2022"},{"issue":"3","key":"2024022218552644800_bib38","doi-asserted-by":"publisher","first-page":"2129","DOI":"10.1007\/s11192-016-2150-8","article-title":"Professional and citizen bibliometrics: Complementarities and ambivalences in the development and use of indicators\u2014A state-of-the-art report","volume":"109","author":"Leydesdorff","year":"2016","journal-title":"Scientometrics"},{"issue":"8","key":"2024022218552644800_bib39","doi-asserted-by":"publisher","first-page":"575","DOI":"10.3152\/030234207X255179","article-title":"The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review","volume":"34","author":"Moed","year":"2007","journal-title":"Science and Public Policy"},{"issue":"7","key":"2024022218552644800_bib40","doi-asserted-by":"publisher","first-page":"e3000737","DOI":"10.1371\/journal.pbio.3000737","article-title":"The Hong Kong Principles for assessing researchers: Fostering research integrity","volume":"18","author":"Moher","year":"2020","journal-title":"PLOS Biology"},{"issue":"2","key":"2024022218552644800_bib41","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1007\/s11024-020-09425-2","article-title":"Navigating uncertainty: Early career academics and practices of appraisal devices","volume":"59","author":"N\u00e4stesj\u00f6","year":"2021","journal-title":"Minerva"},{"key":"2024022218552644800_bib42","doi-asserted-by":"publisher","first-page":"117","DOI":"10.4324\/9781003074960-11","article-title":"Responsible research and innovation: From science in society to science for society, with society","volume-title":"Emerging technologies: Ethics, law and governance","author":"Owen","year":"2020"},{"issue":"3","key":"2024022218552644800_bib43","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1080\/0964401042000229034","article-title":"Responsibility and environmental governance","volume":"13","author":"Pellizzoni","year":"2004","journal-title":"Environmental Politics"},{"issue":"4","key":"2024022218552644800_bib44","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1177\/02704676221126388","article-title":"Process and bureaucracy: Scientific reform as civilisation","volume":"42","author":"Penders","year":"2022","journal-title":"Bulletin of Science, Technology & Society"},{"key":"2024022218552644800_bib45","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1515\/9783110646610-044","article-title":"The competent bibliometrician\u2014A guided tour through the scholarly and practitioner literature","volume-title":"Handbook of bibliometrics","author":"Petersohn","year":"2021"},{"key":"2024022218552644800_bib46","doi-asserted-by":"publisher","first-page":"54","DOI":"10.4018\/978-1-7998-2181-6.ch003","article-title":"Investigating assessment standards in the Netherlands, Italy, and the United Kingdom: Challenges for responsible research evaluation","volume-title":"Shaping the future through standardization","author":"Petersohn","year":"2020"},{"issue":"4","key":"2024022218552644800_bib47","doi-asserted-by":"publisher","first-page":"888","DOI":"10.1162\/qss_a_00224","article-title":"Indicators of research quality, quantity, openness, and responsibility in institutional review, promotion, and tenure policies across seven countries","volume":"3","author":"Pontika","year":"2022","journal-title":"Quantitative Science Studies"},{"key":"2024022218552644800_bib48","doi-asserted-by":"publisher","DOI":"10.7208\/chicago\/9780226675183.001.0001","volume-title":"A history of the modern fact: Problems of knowledge in the sciences of wealth and society","author":"Poovey","year":"1998"},{"key":"2024022218552644800_bib49","doi-asserted-by":"publisher","DOI":"10.1515\/9781400821617","volume-title":"Trust in numbers: The pursuit of objectivity in science and public life","author":"Porter","year":"1996"},{"key":"2024022218552644800_bib50","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780198296034.001.0001","volume-title":"The audit society: Rituals of verification","author":"Power","year":"1997"},{"issue":"1","key":"2024022218552644800_bib51","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1093\/reseval\/rvy030","article-title":"S&T indicators in the wild: Contextualization and participation for responsible metrics","volume":"28","author":"R\u00e0fols","year":"2019","journal-title":"Research Evaluation"},{"issue":"1","key":"2024022218552644800_bib52","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1007\/s11024-020-09419-0","article-title":"Bibliometrics in academic recruitment: A screening tool rather than a game changer","volume":"59","author":"Reymert","year":"2021","journal-title":"Minerva"},{"key":"2024022218552644800_bib53","doi-asserted-by":"publisher","first-page":"m2081","DOI":"10.1136\/bmj.m2081","article-title":"Academic criteria for promotion and tenure in biomedical sciences faculties: Cross sectional analysis of international sample of universities","volume":"369","author":"Rice","year":"2020","journal-title":"British Medical Journal"},{"key":"2024022218552644800_bib54","doi-asserted-by":"publisher","DOI":"10.1016\/C2017-0-01828-1","volume-title":"Becoming metric-wise: A bibliometric guide for researchers","author":"Rousseau","year":"2018"},{"issue":"2","key":"2024022218552644800_bib55","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1007\/s11024-015-9274-5","article-title":"Accounting for impact? The journal impact factor and the making of biomedical research in the Netherlands","volume":"53","author":"Rushforth","year":"2015","journal-title":"Minerva"},{"key":"2024022218552644800_bib56","article-title":"Nobel winner declares boycott of top science journals","author":"Sample","year":"2013","journal-title":"Guardian"},{"issue":"1","key":"2024022218552644800_bib57","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1111\/1469-8676.12098","article-title":"Governing by numbers: Audit culture, rankings and the new world order","volume":"23","author":"Shore","year":"2015","journal-title":"Social Anthropology\/Anthropologie Sociale"},{"issue":"3","key":"2024022218552644800_bib58","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1177\/030631277800800305","article-title":"Cited documents as concept symbols","volume":"8","author":"Small","year":"1978","journal-title":"Social Studies of Science"},{"key":"2024022218552644800_bib59","doi-asserted-by":"publisher","first-page":"13","DOI":"10.4324\/9780203449721-3","article-title":"Introduction: New accountabilities: Anthropological studies in audit, ethics and the academy","volume-title":"Audit cultures","author":"Strathern","year":"2003"},{"key":"2024022218552644800_bib60","doi-asserted-by":"publisher","first-page":"59","DOI":"10.5281\/zenodo.3607319","article-title":"Re-valuing research excellence: From excellentism to responsible assessment","volume-title":"Transforming research excellence: New ideas from the Global South","author":"Tijssen","year":"2020"},{"key":"2024022218552644800_bib61","volume-title":"Good practice in researcher evaluation. Recommendation for the responsible evaluation of a researcher in Finland","author":"TJNK","year":"2020"},{"key":"2024022218552644800_bib62","volume-title":"NOR-CAM: A toolbox for recognition and rewards in academic careers","author":"UiR","year":"2021"},{"key":"2024022218552644800_bib63","volume-title":"Recommendation on Open Science","author":"UNESCO","year":"2021"},{"key":"2024022218552644800_bib64","volume-title":"Statement on responsible research assessment","author":"University of Bristol","year":".."},{"issue":"3","key":"2024022218552644800_bib65","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1007\/BF02129602","article-title":"Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises","volume":"36","author":"van Raan","year":"1996","journal-title":"Scientometrics"},{"issue":"1","key":"2024022218552644800_bib66","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1007\/BF02458401","article-title":"In matters of quantitative studies of science the fault of theorists is offering too little and asking too much","volume":"43","author":"van Raan","year":"1998","journal-title":"Scientometrics"},{"key":"2024022218552644800_bib67","volume-title":"Position paper \u2018Room for everyone\u2019s talent\u2019","author":"VSNU","year":"2019"},{"issue":"1","key":"2024022218552644800_bib68","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1007\/s11192-005-0007-7","article-title":"Impact of bibliometrics upon the science system: Inadvertent consequences?","volume":"62","author":"Weingart","year":"2005","journal-title":"Scientometrics"},{"key":"2024022218552644800_bib69","volume-title":"Guidance for research organisations on how to implement responsible and fair approaches for research assessment","author":"Wellcome Trust","year":"2018"},{"key":"2024022218552644800_bib70","doi-asserted-by":"publisher","DOI":"10.4135\/9781473978782","volume-title":"The Metric Tide: Independent review of the role of metrics in research assessment and management","author":"Wilsdon","year":"2016"},{"key":"2024022218552644800_bib71","doi-asserted-by":"publisher","first-page":"247","DOI":"10.4324\/9781315206455-15","article-title":"Responsible metrics","volume-title":"Strategy and planning in higher education","author":"Wilsdon","year":"2017"},{"key":"2024022218552644800_bib72","volume-title":"From responsible metrics to responsible research assessment (RRA)","author":"Wilsdon","year":"2021"},{"key":"2024022218552644800_bib73","volume-title":"The citation culture","author":"Wouters","year":"1999"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/4\/4\/879\/2339464\/qss_a_00280.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/4\/4\/879\/2339464\/qss_a_00280.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,22]],"date-time":"2024-02-22T13:55:41Z","timestamp":1708610141000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/4\/4\/879\/118385\/The-rise-of-responsible-metrics-as-a-professional"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":73,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,11,1]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00280","relation":{"has-review":[{"id-type":"doi","id":"10.1162\/QSS_A_00280\/v2\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00280\/v1\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00280\/v2\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00280\/v1\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00280\/v1\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00280\/v2\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00280\/v2\/decision1","asserted-by":"object"}]},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}