{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T07:43:48Z","timestamp":1773042228003,"version":"3.50.1"},"reference-count":124,"publisher":"MIT Press","issue":"2","license":[{"start":{"date-parts":[[2023,4,7]],"date-time":"2023-04-07T00:00:00Z","timestamp":1680825600000},"content-version":"vor","delay-in-days":96,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Research England, Scottish Funding Council, Higher Education Funding Council for Wales, and Department for the Economy, Northern Ireland"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,5,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Although funding is essential for some types of research and beneficial for others, it may constrain academic choice and creativity. Thus, it is important to check whether it ever seems unnecessary. Here we investigate whether funded U.K. research tends to be higher quality in all fields and for all major research funders. Based on peer review quality scores for 113,877 articles from all fields in the U.K.\u2019s Research Excellence Framework (REF) 2021, we estimate that there are substantial disciplinary differences in the proportion of funded journal articles, from Theology and Religious Studies (16%+) to Biological Sciences (91%+). The results suggest that funded research is likely to be of higher quality overall, for all the largest research funders, and for 30 out of 34 REF Units of Assessment (disciplines or sets of disciplines), even after factoring out research team size. There are differences between funders in the average quality of the research supported, however. Funding seems particularly associated with higher research quality in health-related fields. The results do not show cause and effect and do not take into account the amount of funding received but are consistent with funding either improving research quality or being won by high-quality researchers or projects.<\/jats:p>","DOI":"10.1162\/qss_a_00254","type":"journal-article","created":{"date-parts":[[2023,4,7]],"date-time":"2023-04-07T10:50:37Z","timestamp":1680864637000},"page":"501-534","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":18,"title":["Is research funding always beneficial? A cross-disciplinary analysis of U.K. research 2014\u201320"],"prefix":"10.1162","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6065-205X","authenticated-orcid":true,"given":"Mike","family":"Thelwall","sequence":"first","affiliation":[{"name":"Statistical Cybermetrics and Research Evaluation Group, University of Wolverhampton, Wolverhampton, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4827-971X","authenticated-orcid":true,"given":"Kayvan","family":"Kousha","sequence":"additional","affiliation":[{"name":"Statistical Cybermetrics and Research Evaluation Group, University of Wolverhampton, Wolverhampton, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9251-5391","authenticated-orcid":true,"given":"Mahshid","family":"Abdoli","sequence":"additional","affiliation":[{"name":"Statistical Cybermetrics and Research Evaluation Group, University of Wolverhampton, Wolverhampton, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4807-7659","authenticated-orcid":true,"given":"Emma","family":"Stuart","sequence":"additional","affiliation":[{"name":"Statistical Cybermetrics and Research Evaluation Group, University of Wolverhampton, Wolverhampton, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2284-0161","authenticated-orcid":true,"given":"Meiko","family":"Makita","sequence":"additional","affiliation":[{"name":"Statistical Cybermetrics and Research Evaluation Group, University of Wolverhampton, Wolverhampton, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2351-4816","authenticated-orcid":true,"given":"Cristina I.","family":"Font-Juli\u00e1n","sequence":"additional","affiliation":[{"name":"Department of Audiovisual Communication, Documentation and History of Art, Universitat Polit\u00e8cnica de Val\u00e8ncia, Val\u00e8ncia, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1265-543X","authenticated-orcid":true,"given":"Paul","family":"Wilson","sequence":"additional","affiliation":[{"name":"Statistical Cybermetrics and Research Evaluation Group, University of Wolverhampton, Wolverhampton, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4386-3813","authenticated-orcid":true,"given":"Jonathan","family":"Levitt","sequence":"additional","affiliation":[{"name":"Statistical Cybermetrics and Research Evaluation Group, University of Wolverhampton, Wolverhampton, UK"}]}],"member":"281","published-online":{"date-parts":[[2023,5,1]]},"reference":[{"issue":"5","key":"2023062118182779600_bib1","doi-asserted-by":"publisher","first-page":"e0251488","DOI":"10.1371\/journal.pone.0251488","article-title":"Getting to the bottom of research funding: Acknowledging the complexity of funding dynamics","volume":"16","author":"Aagaard","year":"2021","journal-title":"PLOS ONE"},{"issue":"2","key":"2023062118182779600_bib2","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1080\/00221546.2010.11779047","article-title":"The effects of scholarly productivity and institutional characteristics on the distribution of federal research grants","volume":"81","author":"Ali","year":"2010","journal-title":"Journal of Higher Education"},{"issue":"5","key":"2023062118182779600_bib3","doi-asserted-by":"publisher","first-page":"1286","DOI":"10.2214\/AJR.20.22786","article-title":"Funding of radiology research: Frequency and association with citation rate","volume":"215","author":"Alkhawtani","year":"2020","journal-title":"American Journal of Roentgenology"},{"issue":"2","key":"2023062118182779600_bib4","doi-asserted-by":"publisher","first-page":"805","DOI":"10.1007\/s11192-019-03066-3","article-title":"What characterises funded biomedical research? Evidence from a basic and a clinical domain","volume":"119","author":"\u00c1lvarez-Bornstein","year":"2019","journal-title":"Scientometrics"},{"issue":"2","key":"2023062118182779600_bib5","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1016\/j.spinee.2013.10.047","article-title":"Does source of funding and conflict of interest influence the outcome and quality of spinal research?","volume":"14","author":"Amiri","year":"2014","journal-title":"Spine Journal"},{"key":"2023062118182779600_bib6","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1007\/978-94-007-5977-0_2","article-title":"Academic career paths","volume-title":"The work situation of the academic profession in Europe: Findings of a survey in twelve countries","author":"Ates","year":"2013"},{"issue":"1","key":"2023062118182779600_bib7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1111\/1468-5965.00341","article-title":"Institutions, inertia and European Union research policy","volume":"40","author":"Banchoff","year":"2002","journal-title":"Journal of Common Market Studies"},{"key":"2023062118182779600_bib8","article-title":"New on Scopus: Link to datasets, search funding acknowledgements and find more CiteScore transparency","author":"Beatty","year":"2017","journal-title":"Scopus Blog"},{"issue":"2","key":"2023062118182779600_bib9","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1016\/S0048-7333(99)00112-2","article-title":"Unintended consequences of cooperative research: Impact of industry sponsorship on climate for academic freedom and other graduate student outcome","volume":"30","author":"Behrens","year":"2001","journal-title":"Research Policy"},{"issue":"4","key":"2023062118182779600_bib10","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1007\/s10734-011-9457-4","article-title":"Academic work from a comparative perspective: A survey of faculty working time across 13 countries","volume":"63","author":"Bentley","year":"2012","journal-title":"Higher Education"},{"issue":"3","key":"2023062118182779600_bib11","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1097\/00007611-199503000-00009","article-title":"Impact of unfunded research in medicine, pathology, and surgery","volume":"88","author":"Berman","year":"1995","journal-title":"Southern Medical Journal"},{"issue":"1","key":"2023062118182779600_bib12","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1007\/s11024-014-9247-0","article-title":"Competitive research grants and their impact on career performance","volume":"52","author":"Bloch","year":"2014","journal-title":"Minerva"},{"issue":"2","key":"2023062118182779600_bib13","doi-asserted-by":"publisher","first-page":"e0147726","DOI":"10.1371\/journal.pone.0147726","article-title":"Size, accumulation and performance for research grants: Examining the role of size for centres of excellence","volume":"11","author":"Bloch","year":"2016","journal-title":"PLOS ONE"},{"issue":"1","key":"2023062118182779600_bib14","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1093\/scipol\/scu019","article-title":"The size of research funding: Trends and implications","volume":"42","author":"Bloch","year":"2015","journal-title":"Science and Public Policy"},{"issue":"3","key":"2023062118182779600_bib15","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1177\/088307389501000313","article-title":"The saga of the septum pellucidum: A tale of unfunded clinical investigations","volume":"10","author":"Bodensteiner","year":"1995","journal-title":"Journal of Child Neurology"},{"issue":"5","key":"2023062118182779600_bib16","first-page":"577","article-title":"Research by pathologists not funded by external grant agencies: A success story","volume":"5","author":"Borkowski","year":"1992","journal-title":"Modern Pathology"},{"issue":"3","key":"2023062118182779600_bib17","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1007\/s11192-006-0121-1","article-title":"Selecting scientific excellence through committee peer review\u2014A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants","volume":"68","author":"Bornmann","year":"2006","journal-title":"Scientometrics"},{"issue":"5","key":"2023062118182779600_bib18","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1136\/flgastro-2020-101506","article-title":"Management of iron deficiency anaemia in secondary care across England between 2012 and 2018: A real-world analysis of hospital episode statistics","volume":"12","author":"Brookes","year":"2021","journal-title":"Frontline Gastroenterology"},{"key":"2023062118182779600_bib19","first-page":"45","article-title":"The practice and politics of funded qualitative research","volume-title":"Strategies of qualitative inquiry","author":"Cheek","year":"2008"},{"issue":"1","key":"2023062118182779600_bib20","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1177\/0193841X9401800103","article-title":"Grants peer review in theory and practice","volume":"18","author":"Chubin","year":"1994","journal-title":"Evaluation Review"},{"issue":"1","key":"2023062118182779600_bib21","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1111\/j.1475-5890.2008.00069.x","article-title":"Money for science? The impact of research grants on academic output","volume":"29","author":"Chudnovsky","year":"2008","journal-title":"Fiscal Studies"},{"issue":"1","key":"2023062118182779600_bib22","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1177\/030631289019001002","article-title":"The emergence of research grants within the prize system of the French Academy of Sciences, 1795\u20131914","volume":"19","author":"Crosland","year":"1989","journal-title":"Social Studies of Science"},{"key":"2023062118182779600_bib23","doi-asserted-by":"publisher","DOI":"10.3386\/w30107","volume-title":"Gender and underrepresented minority differences in research funding","author":"Cruz-Castro","year":"2022"},{"key":"2023062118182779600_bib24","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780198742937.001.0001","volume-title":"Research for the developing world: Public funding from Australia, Canada, and the UK","author":"Currie-Alder","year":"2015"},{"issue":"2","key":"2023062118182779600_bib25","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1016\/j.respol.2008.11.008","article-title":"Funding incentives, collaborative dynamics and scientific productivity: Evidence from the EU framework program","volume":"38","author":"Defazio","year":"2009","journal-title":"Research Policy"},{"issue":"3","key":"2023062118182779600_bib26","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1111\/1467-8330.00137","article-title":"The new social contract for science: Accountability, relevance, and value in US and UK science and research policy","volume":"32","author":"Demeritt","year":"2000","journal-title":"Antipode"},{"key":"2023062118182779600_bib27","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1007\/978-94-007-5977-0_6","article-title":"The research function of the academic profession in Europe","volume-title":"The work situation of the academic profession in Europe: Findings of a survey in twelve countries","author":"Drennan","year":"2013"},{"issue":"3","key":"2023062118182779600_bib28","doi-asserted-by":"publisher","first-page":"1093","DOI":"10.1007\/s11192-015-1825-x","article-title":"How to boost scientific production? A statistical analysis of research funding and other influencing factors","volume":"106","author":"Ebadi","year":"2016","journal-title":"Scientometrics"},{"issue":"4","key":"2023062118182779600_bib29","doi-asserted-by":"publisher","first-page":"904","DOI":"10.1080\/03075079.2020.1817891","article-title":"Why do academics do unfunded research? Resistance, compliance and identity in the UK neo-liberal university","volume":"47","author":"Edwards","year":"2022","journal-title":"Studies in Higher Education"},{"issue":"Suppl. 10","key":"2023062118182779600_bib30","doi-asserted-by":"publisher","first-page":"S105","DOI":"10.1097\/ACM.0b013e3181b3707d","article-title":"A small grants program improves medical education research productivity","volume":"84","author":"El-Sawi","year":"2009","journal-title":"Academic Medicine"},{"issue":"3","key":"2023062118182779600_bib31","doi-asserted-by":"publisher","first-page":"1611","DOI":"10.1007\/s11192-016-2145-5","article-title":"Who gets Horizon 2020 research grants? Propensity to apply and probability to succeed in a two-step analysis","volume":"109","author":"Enger","year":"2016","journal-title":"Scientometrics"},{"issue":"3","key":"2023062118182779600_bib32","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1016\/S0735-6757(97)90011-5","article-title":"Research funding in the four major emergency medicine journals","volume":"15","author":"Ernst","year":"1997","journal-title":"American Journal of Emergency Medicine"},{"key":"2023062118182779600_bib33","doi-asserted-by":"publisher","first-page":"e13323","DOI":"10.7554\/eLife.13323","article-title":"NIH peer review percentile scores are poorly predictive of grant productivity","volume":"5","author":"Fang","year":"2016","journal-title":"eLife"},{"issue":"1","key":"2023062118182779600_bib34","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1007\/s11024-017-9338-9","article-title":"The drawbacks of project funding for epistemic innovation: Comparing institutional affordances and constraints of different types of research funding","volume":"56","author":"Franssen","year":"2018","journal-title":"Minerva"},{"issue":"4","key":"2023062118182779600_bib35","doi-asserted-by":"publisher","first-page":"444","DOI":"10.1177\/0162243909345836","article-title":"Undone science: Charting social movement and civil society challenges to research agenda setting","volume":"35","author":"Frickel","year":"2010","journal-title":"Science, Technology, & Human Values"},{"key":"2023062118182779600_bib36","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1007\/978-3-319-10720-2_10","article-title":"Academic careers and work in Europe: Trends, challenges, perspectives","volume-title":"Academic work and careers in Europe: Trends, challenges, perspectives","author":"Fumasoli","year":"2015"},{"issue":"9","key":"2023062118182779600_bib37","doi-asserted-by":"publisher","first-page":"e106474","DOI":"10.1371\/journal.pone.0106474","article-title":"The validation of peer review through research impact measures and the implications for funding strategies","volume":"9","author":"Gallo","year":"2014","journal-title":"PLOS ONE"},{"issue":"1","key":"2023062118182779600_bib38","doi-asserted-by":"publisher","first-page":"17","DOI":"10.3152\/147154402781776952","article-title":"Using curriculum vitae to compare some impacts of NSF research grants with research center funding","volume":"11","author":"Gaughan","year":"2002","journal-title":"Research Evaluation"},{"issue":"10","key":"2023062118182779600_bib39","doi-asserted-by":"publisher","first-page":"1309","DOI":"10.1136\/gut.2010.228874","article-title":"Guidelines for the management of iron deficiency anaemia","volume":"60","author":"Goddard","year":"2011","journal-title":"Gut"},{"key":"2023062118182779600_bib40","volume-title":"The impact of research grants on the productivity and quality of scientific research","author":"Godin","year":"2003"},{"issue":"2","key":"2023062118182779600_bib41","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1080\/00779954.2017.1325921","article-title":"The effect of public funding on research output: The New Zealand Marsden Fund","volume":"52","author":"Gush","year":"2018","journal-title":"New Zealand Economic Papers"},{"issue":"1","key":"2023062118182779600_bib42","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1109\/TKDE.2015.2457911","article-title":"Ordinal regression methods: Survey and experimental study","volume":"28","author":"Guti\u00e9rrez","year":"2015","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"issue":"3","key":"2023062118182779600_bib43","doi-asserted-by":"publisher","first-page":"101050","DOI":"10.1016\/j.joi.2020.101050","article-title":"Research funding: Past performance is a stronger predictor of future scientific output than reviewer scores","volume":"14","author":"Gy\u0151rffy","year":"2020","journal-title":"Journal of Informetrics"},{"issue":"7578","key":"2023062118182779600_bib44","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1038\/527286a","article-title":"Racial bias continues to haunt NIH grants","volume":"527","author":"Hayden","year":"2015","journal-title":"Nature"},{"issue":"5","key":"2023062118182779600_bib45","doi-asserted-by":"publisher","first-page":"e002800","DOI":"10.1136\/bmjopen-2013-002800","article-title":"On the time spent preparing grant proposals: An observational study of Australian researchers","volume":"3","author":"Herbert","year":"2013","journal-title":"BMJ Open"},{"issue":"3","key":"2023062118182779600_bib46","doi-asserted-by":"publisher","first-page":"e004462","DOI":"10.1136\/bmjopen-2013-004462","article-title":"The impact of funding deadlines on personal workloads, stress and family relationships: A qualitative study of Australian researchers","volume":"4","author":"Herbert","year":"2014","journal-title":"BMJ Open"},{"issue":"1","key":"2023062118182779600_bib47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1057\/s41599-021-00891-x","article-title":"The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants","volume":"8","author":"Heyard","year":"2021","journal-title":"Humanities and Social Sciences Communications"},{"issue":"2","key":"2023062118182779600_bib48","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1016\/j.respol.2011.09.007","article-title":"Performance-based university research funding systems","volume":"41","author":"Hicks","year":"2012","journal-title":"Research Policy"},{"issue":"4","key":"2023062118182779600_bib49","doi-asserted-by":"publisher","first-page":"840","DOI":"10.1111\/jonm.12747","article-title":"Evidence-based practice among epilepsy specialist nurses in the Republic of Ireland: Findings from the SENsE study","volume":"27","author":"Higgins","year":"2019","journal-title":"Journal of Nursing Management"},{"issue":"3","key":"2023062118182779600_bib50","doi-asserted-by":"publisher","first-page":"146","DOI":"10.3152\/030234208X299044","article-title":"Does competitive research funding encourage diversity in higher education?","volume":"35","author":"Horta","year":"2008","journal-title":"Science and Public Policy"},{"issue":"12","key":"2023062118182779600_bib51","doi-asserted-by":"publisher","first-page":"2382","DOI":"10.1080\/03075079.2019.1612351","article-title":"Organisational factors and academic research agendas: An analysis of academics in the social sciences","volume":"45","author":"Horta","year":"2020","journal-title":"Studies in Higher Education"},{"issue":"1","key":"2023062118182779600_bib53","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ijindorg.2016.12.004","article-title":"Fishing for complementarities: Research grants and research productivity","volume":"51","author":"Hottenrott","year":"2017","journal-title":"International Journal of Industrial Organization"},{"issue":"4","key":"2023062118182779600_bib54","doi-asserted-by":"publisher","first-page":"534","DOI":"10.1111\/j.1467-6435.2011.00519.x","article-title":"Industry funding of university research and scientific productivity","volume":"64","author":"Hottenrott","year":"2011","journal-title":"Kyklos"},{"issue":"4","key":"2023062118182779600_bib55","doi-asserted-by":"publisher","first-page":"463","DOI":"10.1080\/13662716.2021.1990023","article-title":"The long-term effect of research grants on the scientific output of university professors","volume":"29","author":"Hussinger","year":"2022","journal-title":"Industry and Innovation"},{"issue":"5","key":"2023062118182779600_bib56","doi-asserted-by":"publisher","first-page":"1106","DOI":"10.12669\/pjms.36.5.3088","article-title":"Psychological burden of quarantine in children and adolescents: A rapid systematic review and proposed solutions","volume":"36","author":"Imran","year":"2020","journal-title":"Pakistan Journal of Medical Sciences"},{"key":"2023062118182779600_bib57","doi-asserted-by":"publisher","first-page":"203","DOI":"10.4337\/9781800883086.00020","article-title":"Research funding in the context of high institutional stratification. Policy scenarios for Europe based on insights from the United States","volume-title":"Handbook of public research funding","author":"Jappe","year":"2023"},{"issue":"7","key":"2023062118182779600_bib58","doi-asserted-by":"publisher","first-page":"810","DOI":"10.1016\/S1076-6332(03)80353-7","article-title":"American Society of Neuroradiology research survey 2001","volume":"9","author":"Johnson","year":"2002","journal-title":"Academic Radiology"},{"issue":"4","key":"2023062118182779600_bib59","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1007\/s10734-010-9342-6","article-title":"Reshaping academic capitalism to meet development priorities: The case of public universities in Kenya","volume":"61","author":"Johnson","year":"2011","journal-title":"Higher Education"},{"issue":"3","key":"2023062118182779600_bib60","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1016\/j.radi.2010.12.004","article-title":"Considerations, clues and challenges: Gaining ethical and trust research approval when using the NHS as a research setting","volume":"17","author":"Jonker","year":"2011","journal-title":"Radiography"},{"issue":"6","key":"2023062118182779600_bib61","doi-asserted-by":"publisher","first-page":"593","DOI":"10.1108\/00012531111187243","article-title":"The effect of funding on academic research impact: A case study of Iranian publications","volume":"63","author":"Jowkar","year":"2011","journal-title":"Aslib Proceedings"},{"key":"2023062118182779600_bib62","doi-asserted-by":"publisher","DOI":"10.1016\/S1479-3628(2007)4","volume-title":"Autonomy in social science research, volume 4: The view from United Kingdom and Australian universities","author":"Kayrooz","year":"2007"},{"issue":"7","key":"2023062118182779600_bib63","doi-asserted-by":"publisher","first-page":"1011","DOI":"10.1007\/s00192-017-3465-6","article-title":"Quality of randomized controlled trials published in the International Urogynecology Journal 2007\u20132016","volume":"29","author":"Kim","year":"2018","journal-title":"International Urogynecology Journal"},{"issue":"3","key":"2023062118182779600_bib64","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1093\/reseval\/rvv012","article-title":"Options and limitations in measuring the impact of research grants\u2014Evidence from Denmark and Norway","volume":"24","author":"Langfeldt","year":"2015","journal-title":"Research Evaluation"},{"issue":"1","key":"2023062118182779600_bib65","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/s11024-019-09385-2","article-title":"Co-existing notions of research quality: A framework to study context-specific understandings of good research","volume":"58","author":"Langfeldt","year":"2020","journal-title":"Minerva"},{"issue":"1","key":"2023062118182779600_bib66","doi-asserted-by":"publisher","first-page":"27","DOI":"10.3152\/147154405781776300","article-title":"Is external research funding a valid indicator for research performance?","volume":"14","author":"Laudel","year":"2005","journal-title":"Research Evaluation"},{"issue":"6","key":"2023062118182779600_bib67","doi-asserted-by":"publisher","first-page":"372","DOI":"10.3152\/030234207X234578","article-title":"Comparing the evolution of national research policies: What patterns of change?","volume":"34","author":"Lepori","year":"2007","journal-title":"Science and Public Policy"},{"key":"2023062118182779600_bib68","first-page":"1013","article-title":"Are funded articles more highly cited than unfunded articles? A preliminary investigation","volume-title":"Proceedings of ISSI 2011","author":"Levitt","year":"2011"},{"issue":"25","key":"2023062118182779600_bib69","doi-asserted-by":"publisher","first-page":"6498","DOI":"10.1073\/pnas.1609996114","article-title":"Future of fundamental discovery in US biomedical research","volume":"114","author":"Levitt","year":"2017","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"1\u20132","key":"2023062118182779600_bib70","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1007\/BF02457963","article-title":"The effect of funding on the outputs of biomedical research","volume":"41","author":"Lewison","year":"1998","journal-title":"Scientometrics"},{"issue":"4","key":"2023062118182779600_bib71","doi-asserted-by":"publisher","first-page":"e9","DOI":"10.1161\/RES.0000000000000158","article-title":"Peer review practices for evaluating biomedical research grants: A scientific statement from the American Heart Association","volume":"121","author":"Liaw","year":"2017","journal-title":"Circulation Research"},{"issue":"3","key":"2023062118182779600_bib72","doi-asserted-by":"publisher","first-page":"796","DOI":"10.1148\/radiol.12111976","article-title":"Characteristics and trends of radiology research: A survey of original articles published in AJR and Radiology between 2001 and 2010","volume":"264","author":"Lim","year":"2012","journal-title":"Radiology"},{"issue":"3","key":"2023062118182779600_bib74","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1016\/j.jacc.2012.10.013","article-title":"The magnitude and nature of unfunded published cardiovascular research","volume":"61","author":"Mai","year":"2013","journal-title":"Journal of the American College of Cardiology"},{"issue":"1","key":"2023062118182779600_bib75","doi-asserted-by":"publisher","first-page":"871","DOI":"10.1007\/s11192-020-03690-4","article-title":"Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations\u2019 COCI: A multidisciplinary comparison of coverage via citations","volume":"126","author":"Mart\u00edn-Mart\u00edn","year":"2021","journal-title":"Scientometrics"},{"key":"2023062118182779600_bib76","article-title":"Improvements to funding data in Scopus: Now 16.5M articles with funding information and easier to identify funded research","author":"McCullough","year":"2021","journal-title":"Scopus Blog"},{"issue":"4","key":"2023062118182779600_bib77","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1080\/00221546.2010.11779062","article-title":"Revisiting academic capitalism in Canada: No longer the exception","volume":"81","author":"Metcalfe","year":"2010","journal-title":"Journal of Higher Education"},{"issue":"2","key":"2023062118182779600_bib79","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1093\/scipol\/scw057","article-title":"The complex relationship between academic engagement and research output: Evidence from Italy","volume":"44","author":"Muscio","year":"2017","journal-title":"Science and Public Policy"},{"issue":"1","key":"2023062118182779600_bib80","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1016\/S0048-7333(01)00107-X","article-title":"Innovation and research policy in France (1980\u20132000) or the disappearance of the Colbertist state","volume":"31","author":"Mustar","year":"2002","journal-title":"Research Policy"},{"issue":"1","key":"2023062118182779600_bib81","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1093\/reseval\/rvv029","article-title":"Determining effects of individual research grants on publication output and impact: The case of the Emmy Noether Programme (German Research Foundation)","volume":"25","author":"Neufeld","year":"2016","journal-title":"Research Evaluation"},{"key":"2023062118182779600_bib82","volume-title":"Teaching-only academics in a research intensive university: From an undesirable to a desirable academic identity","author":"Nyamapfene","year":"2018"},{"key":"2023062118182779600_bib83","doi-asserted-by":"publisher","DOI":"10.1787\/9789264207462-en","volume-title":"Promoting research excellence: New approaches to funding","author":"OECD","year":"2014"},{"key":"2023062118182779600_bib84","volume-title":"How much is too much? Cross-subsidies from teaching to research in British universities","author":"Olive","year":"2017"},{"issue":"10","key":"2023062118182779600_bib85","doi-asserted-by":"publisher","first-page":"e0185578","DOI":"10.1371\/journal.pone.0185578","article-title":"Beyond funding: Acknowledgement patterns in biomedical, natural and social sciences","volume":"12","author":"Paul-Hus","year":"2017","journal-title":"PLOS ONE"},{"issue":"3\u20134","key":"2023062118182779600_bib86","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1007\/BF02095347","article-title":"The citation impact of funded and unfunded research in economics","volume":"19","author":"Peritz","year":"1990","journal-title":"Scientometrics"},{"issue":"7139","key":"2023062118182779600_bib87","doi-asserted-by":"publisher","first-page":"1236","DOI":"10.1136\/bmj.316.7139.1236","article-title":"What\u2019s wrong with Bonferroni adjustments","volume":"316","author":"Perneger","year":"1998","journal-title":"British Medical Journal"},{"issue":"5","key":"2023062118182779600_bib89","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1007\/s10734-005-1118-z","article-title":"The nature and implications of the growing importance of research grants to Canadian universities and academics","volume":"53","author":"Polster","year":"2007","journal-title":"Higher Education"},{"issue":"10","key":"2023062118182779600_bib90","doi-asserted-by":"publisher","first-page":"1714","DOI":"10.1177\/1420326X20961495","article-title":"Performance evaluation of climate-adaptive natural ventilation design: A case study of semi-open public cultural building","volume":"30","author":"Qi","year":"2021","journal-title":"Indoor and Built Environment"},{"key":"2023062118182779600_bib91","volume-title":"Panel criteria and working methods","author":"REF","year":"2020"},{"key":"2023062118182779600_bib92","volume-title":"Guidance on the REF 2021 results","author":"REF","year":"2022"},{"issue":"5","key":"2023062118182779600_bib93","doi-asserted-by":"publisher","first-page":"365","DOI":"10.3152\/095820211X13164389670392","article-title":"Systematic grant and funding body acknowledgement data for publications: New dimensions and new controversies for research policy and evaluation","volume":"20","author":"Rigby","year":"2011","journal-title":"Research Evaluation"},{"issue":"20","key":"2023062118182779600_bib94","doi-asserted-by":"publisher","first-page":"2208","DOI":"10.1097\/BRS.0b013e31817bd8f8","article-title":"Is research presented at the scoliosis research society annual meeting influenced by industry funding?","volume":"33","author":"Roach","year":"2008","journal-title":"Spine"},{"issue":"9","key":"2023062118182779600_bib95","doi-asserted-by":"publisher","first-page":"7859","DOI":"10.1007\/s11192-021-04077-9","article-title":"What is the relationship between research funding and citation-based performance? A comparative analysis between critical disciplines","volume":"126","author":"Roshani","year":"2021","journal-title":"Scientometrics"},{"issue":"2","key":"2023062118182779600_bib96","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1007\/s11192-009-0422-2","article-title":"Research quality and diversity of funding: A model for relating research money to output of research","volume":"79","author":"Sandstr\u00f6m","year":"2009","journal-title":"Scientometrics"},{"key":"2023062118182779600_bib97","doi-asserted-by":"publisher","DOI":"10.1101\/354662","article-title":"The impact of grant funding on the publication activity of awarded applicants: A systematic review of comparative studies and meta-analytical estimates","author":"Saygitov","year":"2018","journal-title":"bioRxiv"},{"issue":"4","key":"2023062118182779600_bib98","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1093\/reseval\/rvu016","article-title":"Analysing robustness and uncertainty levels of bibliometric performance statistics supporting science policy. A case study evaluating Danish postdoctoral funding","volume":"23","author":"Schneider","year":"2014","journal-title":"Research Evaluation"},{"issue":"4","key":"2023062118182779600_bib99","doi-asserted-by":"publisher","first-page":"639","DOI":"10.3171\/2015.7.SPINE15153","article-title":"Fracture of fusion mass after hardware removal in patients with high sagittal imbalance","volume":"24","author":"Sedney","year":"2016","journal-title":"Journal of Neurosurgery: Spine"},{"issue":"8","key":"2023062118182779600_bib100","doi-asserted-by":"publisher","first-page":"1106","DOI":"10.1002\/asi.24617","article-title":"Conditions that do or do not disadvantage interdisciplinary research proposals in project evaluation","volume":"73","author":"Seeber","year":"2022","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"9","key":"2023062118182779600_bib101","doi-asserted-by":"publisher","first-page":"e29875","DOI":"10.2196\/29875","article-title":"Recent academic research on clinically relevant digital measures: Systematic review","volume":"23","author":"Shandhi","year":"2021","journal-title":"Journal of Medical Internet Research"},{"issue":"3","key":"2023062118182779600_bib102","doi-asserted-by":"publisher","first-page":"1415","DOI":"10.1007\/s11192-016-2224-7","article-title":"Promoting diversity in science in Japan through mission-oriented research grants","volume":"110","author":"Shimada","year":"2017","journal-title":"Scientometrics"},{"issue":"8","key":"2023062118182779600_bib103","doi-asserted-by":"publisher","first-page":"1159","DOI":"10.1176\/ajp.154.8.1159","article-title":"Research without external funding in North American psychiatry","volume":"154","author":"Silberman","year":"1997","journal-title":"American Journal of Psychiatry"},{"issue":"7","key":"2023062118182779600_bib104","doi-asserted-by":"publisher","first-page":"e820","DOI":"10.1097\/GOX.0000000000000834","article-title":"A bibliometric analysis of the 100 most-cited articles in rhinoplasty","volume":"4","author":"Sinha","year":"2016","journal-title":"Plastic and Reconstructive Surgery Global Open"},{"issue":"2","key":"2023062118182779600_bib105","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1177\/1350508401082003","article-title":"Expanding and elaborating the concept of academic capitalism","volume":"8","author":"Slaughter","year":"2001","journal-title":"Organization"},{"issue":"6","key":"2023062118182779600_bib106","doi-asserted-by":"publisher","first-page":"781","DOI":"10.1001\/jama.1993.03500060081035","article-title":"Who pays for published research?","volume":"269","author":"Stein","year":"1993","journal-title":"JAMA"},{"issue":"1","key":"2023062118182779600_bib107","first-page":"2","article-title":"Health research policy and systems in Japan: A review focused on the Health, Labour and Welfare Sciences Research Grants","volume":"70","author":"Takemura","year":"2021","journal-title":"Journal of the National Institute of Public Health"},{"issue":"5","key":"2023062118182779600_bib108","doi-asserted-by":"publisher","first-page":"1335","DOI":"10.1096\/fj.09-148239","article-title":"Sources of funding for Nobel Prize-winning work: Public or private?","volume":"24","author":"Tatsioni","year":"2010","journal-title":"FASEB Journal"},{"issue":"2","key":"2023062118182779600_bib109","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1007\/s11024-022-09460-1","article-title":"The societal territory of academic disciplines: How disciplines matter to society","volume":"60","author":"Tellmann","year":"2022","journal-title":"Minerva"},{"issue":"1","key":"2023062118182779600_bib110","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1016\/j.joi.2016.12.002","article-title":"Three practical field normalised alternative indicator formulae for research evaluation","volume":"11","author":"Thelwall","year":"2017","journal-title":"Journal of Informetrics"},{"issue":"2","key":"2023062118182779600_bib111","doi-asserted-by":"publisher","first-page":"530","DOI":"10.1016\/j.joi.2017.03.004","article-title":"The accuracy of confidence intervals for field normalised indicators","volume":"11","author":"Thelwall","year":"2017","journal-title":"Journal of Informetrics"},{"key":"2023062118182779600_bib112","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.06571","article-title":"Why are co-authored academic articles more cited: Higher quality or larger audience?","author":"Thelwall","year":"2022","journal-title":"arXiv"},{"key":"2023062118182779600_bib113","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.08041","article-title":"Can REF output quality scores be assigned by AI? Experimental evidence","author":"Thelwall","year":"2022","journal-title":"arXiv"},{"issue":"1","key":"2023062118182779600_bib114","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1108\/AJIM-09-2015-0146","article-title":"Alternative metric indicators for funding scheme evaluations","volume":"68","author":"Thelwall","year":"2016","journal-title":"Aslib Journal of Information Management"},{"issue":"8","key":"2023062118182779600_bib115","doi-asserted-by":"publisher","first-page":"968","DOI":"10.1002\/asi.24315","article-title":"Academic collaboration rates and citation associations vary substantially between countries and fields","volume":"71","author":"Thelwall","year":"2020","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"1","key":"2023062118182779600_bib116","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1093\/swr\/35.1.3","article-title":"Harmful effects of federal research grants","volume":"35","author":"Thyer","year":"2011","journal-title":"Social Work Research"},{"issue":"1","key":"2023062118182779600_bib117","doi-asserted-by":"publisher","first-page":"e0169718","DOI":"10.1371\/journal.pone.0169718","article-title":"Strategies to prevent or reduce gender bias in peer review of research grants: A rapid scoping review","volume":"12","author":"Tricco","year":"2017","journal-title":"PLOS ONE"},{"issue":"3","key":"2023062118182779600_bib118","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/s41370-019-0197-3","article-title":"Quantifying the impact of housing interventions on indoor air quality and energy consumption using coupled simulation models","volume":"30","author":"Underhill","year":"2020","journal-title":"Journal of Exposure Science & Environmental Epidemiology"},{"issue":"9","key":"2023062118182779600_bib119","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1161\/CIRCULATIONAHA.117.031883","article-title":"Availability and use of shared data from cardiometabolic clinical trials","volume":"137","author":"Vaduganathan","year":"2018","journal-title":"Circulation"},{"issue":"3","key":"2023062118182779600_bib120","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1057\/hep.2012.15","article-title":"The selection of scientific talent in the allocation of research grants","volume":"25","author":"van Arensbergen","year":"2012","journal-title":"Higher Education Policy"},{"issue":"4","key":"2023062118182779600_bib121","doi-asserted-by":"publisher","first-page":"273","DOI":"10.3152\/095820209X475360","article-title":"Past performance, peer review and project selection: A case study in the social and behavioral sciences","volume":"18","author":"van den Besselaar","year":"2009","journal-title":"Research Evaluation"},{"issue":"10","key":"2023062118182779600_bib122","first-page":"757","article-title":"Growing and retaining general practice research leaders in Australia: How can we do better?","volume":"46","author":"van Driel","year":"2017","journal-title":"Australian Family Physician"},{"issue":"3","key":"2023062118182779600_bib123","doi-asserted-by":"publisher","first-page":"851","DOI":"10.1007\/s11192-012-0775-9","article-title":"Citation time window choice for research impact evaluation","volume":"94","author":"Wang","year":"2013","journal-title":"Scientometrics"},{"issue":"4","key":"2023062118182779600_bib124","doi-asserted-by":"publisher","first-page":"418","DOI":"10.1093\/reseval\/rvaa023","article-title":"The effect of competitive public funding on scientific output: A comparison between China and the EU","volume":"29","author":"Wang","year":"2020","journal-title":"Research Evaluation"},{"key":"2023062118182779600_bib125","volume-title":"The intellectual and social organization of the sciences","author":"Whitley","year":"2000"},{"issue":"11","key":"2023062118182779600_bib126","doi-asserted-by":"publisher","first-page":"e111458","DOI":"10.1371\/journal.pone.0111458","article-title":"NSFC health research funding and burden of disease in China","volume":"9","author":"Xu","year":"2014","journal-title":"PLOS ONE"},{"issue":"1","key":"2023062118182779600_bib127","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1007\/s11192-017-2583-8","article-title":"The funding factor: A cross-disciplinary examination of the association between research funding and citation impact","volume":"115","author":"Yan","year":"2018","journal-title":"Scientometrics"},{"issue":"2","key":"2023062118182779600_bib128","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1007\/s11192-010-0191-y","article-title":"Characteristics and impact of grant-funded research: A case study of the library and information science field","volume":"84","author":"Zhao","year":"2010","journal-title":"Scientometrics"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/4\/2\/501\/2136457\/qss_a_00254.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/qss\/article-pdf\/4\/2\/501\/2136457\/qss_a_00254.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,21]],"date-time":"2023-06-21T15:38:59Z","timestamp":1687361939000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/4\/2\/501\/115607\/Is-research-funding-always-beneficial-A-cross"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":124,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,5,1]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00254","relation":{"has-review":[{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v1\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v1\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v2\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v2\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v2\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v2\/review3","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v1\/review3","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v1\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v3\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v3\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1162\/QSS_A_00254\/v2\/decision1","asserted-by":"object"}]},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}