{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T03:07:30Z","timestamp":1776049650489,"version":"3.50.1"},"reference-count":40,"publisher":"eLife Sciences Publications, Ltd","license":[{"start":{"date-parts":[[2021,12,7]],"date-time":"2021-12-07T00:00:00Z","timestamp":1638835200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2021,12,7]],"date-time":"2021-12-07T00:00:00Z","timestamp":1638835200000},"content-version":"am","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2021,12,7]],"date-time":"2021-12-07T00:00:00Z","timestamp":1638835200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Genome Quebec, Genome Canada"}],"content-domain":{"domain":["elifesciences.org"],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2021,8,27]]},"abstract":"<jats:p>The Reproducibility Project: Cancer Biology (RPCB) was established to provide evidence about reproducibility in basic and preclinical cancer research, and to identify the factors that influence reproducibility more generally. In this commentary we address some of the scientific, ethical and policy implications of the project. We liken the basic and preclinical cancer research enterprise to a vast 'diagnostic machine' that is used to determine which clinical hypotheses should be advanced for further development, including clinical trials. The results of the RPCB suggest that this diagnostic machine currently recommends advancing many findings that are not reproducible. While concerning, we believe that more work needs to be done to evaluate the performance of the diagnostic machine. Specifically, we believe three questions remain unanswered: how often does the diagnostic machine correctly recommend against advancing real effects to clinical testing?; what are the relative costs to society of false positive and false negatives?; and how well do scientists and others interpret the outputs of the machine?<\/jats:p>","DOI":"10.7554\/elife.67527","type":"journal-article","created":{"date-parts":[[2021,12,7]],"date-time":"2021-12-07T00:00:21Z","timestamp":1638835221000},"update-policy":"https:\/\/doi.org\/10.7554\/elife.67527","source":"Crossref","is-referenced-by-count":15,"title":["Is preclinical research in cancer biology reproducible enough?"],"prefix":"10.7554","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1050-570X","authenticated-orcid":true,"given":"Patrick Bodilly","family":"Kane","sequence":"first","affiliation":[{"name":"Studies in Translation, Ethics and Medicine, Biomedical Ethics Unit, McGill University","place":["Montr\u00e9al, Canada"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1614-6779","authenticated-orcid":true,"given":"Jonathan","family":"Kimmelman","sequence":"additional","affiliation":[{"name":"Studies in Translation, Ethics and Medicine, Biomedical Ethics Unit, McGill University","place":["Montr\u00e9al, Canada"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"4374","published-online":{"date-parts":[[2021,12,7]]},"reference":[{"key":"bib1","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1038\/483531a","article-title":"Drug development: Raise standards for preclinical cancer research","volume":"483","author":"Begley","year":"2012","journal-title":"Nature"},{"key":"bib2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.2002212","article-title":"Can cancer researchers accurately judge whether preclinical reports will reproduce?","volume":"15","author":"Benjamin","year":"2017","journal-title":"PLOS Biology"},{"key":"bib3","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1038\/s41562-017-0189-z","article-title":"Redefine statistical significance","volume":"2","author":"Benjamin","year":"2018","journal-title":"Nature Human Behaviour"},{"key":"bib4","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1634\/theoncologist.2020-0054","article-title":"Can oncologists predict the efficacy of treatments in randomized trials?","volume":"26","author":"Benjamin","year":"2021","journal-title":"The Oncologist"},{"key":"bib5","volume-title":"On Exactitude in Science","author":"Borges","year":"1946"},{"key":"bib6","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1038\/nrn3475","article-title":"Power failure: Why small sample size undermines the reliability of neuroscience","volume":"14","author":"Button","year":"2013","journal-title":"Nature Reviews Neuroscience"},{"key":"bib7","doi-asserted-by":"publisher","first-page":"254","DOI":"10.1038\/nrc.2016.140","article-title":"Interrogating open issues in cancer precision medicine with patient-derived xenografts","volume":"17","author":"Byrne","year":"2017","journal-title":"Nature Review Cancer"},{"key":"bib8","doi-asserted-by":"publisher","first-page":"1433","DOI":"10.1126\/science.aaf0918","article-title":"Evaluating replicability of laboratory experiments in economics","volume":"351","author":"Camerer","year":"2016","journal-title":"Science"},{"key":"bib9","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1038\/s41562-018-0399-z","article-title":"Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015","volume":"2","author":"Camerer","year":"2018","journal-title":"Nature Human Behaviour"},{"key":"bib10","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.140216","article-title":"An investigation of the false discovery rate and the misinterpretation of p-values","volume":"1","author":"Colquhoun","year":"2014","journal-title":"Royal Society Open Science"},{"key":"bib11","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.j4530","article-title":"Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13","volume":"359","author":"Davis","year":"2017","journal-title":"BMJ"},{"key":"bib12","doi-asserted-by":"publisher","DOI":"10.15252\/embj.2018101117","article-title":"Rethinking research reproducibility","volume":"38","author":"Dirnagl","year":"2019","journal-title":"EMBO Journal"},{"key":"bib13","doi-asserted-by":"publisher","first-page":"15343","DOI":"10.1073\/pnas.1516179112","article-title":"Using prediction markets to estimate the reproducibility of scientific research","volume":"112","author":"Dreber","year":"2015","journal-title":"PNAS"},{"key":"bib14","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.62101","article-title":"Improving preclinical studies through replications","volume":"10","author":"Drude","year":"2021","journal-title":"eLife"},{"key":"bib15","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1038\/s41573-020-0087-3","article-title":"Improving target assessment in biomedical research: the GOT-IT recommendations","volume":"20","author":"Emmerich","year":"2021","journal-title":"Nature Reviews Drug Discovery"},{"key":"bib16","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1027\/1618-3169\/a000402","article-title":"Do researchers anchor their beliefs on the outcome of an initial study? Testing the time-reversal heuristic","volume":"65","author":"Ernst","year":"2018","journal-title":"Experimental Psychology"},{"key":"bib17","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.04333","article-title":"An open investigation of the reproducibility of cancer biology research","volume":"3","author":"Errington","year":"2014","journal-title":"eLife"},{"key":"bib18","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.73430","article-title":"Experiments from unfinished Registered Reports in the Reproducibility Project: Cancer Biology","volume":"10","author":"Errington","year":"2021","journal-title":"eLife"},{"key":"bib19","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.71601","article-title":"Investigating the replicability of preclinical cancer biology","volume":"10","author":"Errington","year":"2021","journal-title":"eLife"},{"key":"bib20","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.67995","article-title":"Challenges for assessing replicability in preclinical cancer biology","volume":"10","author":"Errington","year":"2021","journal-title":"eLife"},{"key":"bib21","doi-asserted-by":"publisher","first-page":"1005","DOI":"10.7326\/0003-4819-130-12-199906150-00019","article-title":"Toward evidence-based medical statistics. 2: The Bayes factor","volume":"130","author":"Goodman","year":"1999","journal-title":"Annals of Internal Medicine"},{"key":"bib22","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1038\/nbt.2786","article-title":"Clinical development success rates for investigational drugs","volume":"32","author":"Hay","year":"2014","journal-title":"Nature Biotechnology"},{"key":"bib23","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.08351","article-title":"A meta-analysis of threats to valid clinical inference in preclinical research of sunitinib","volume":"4","author":"Henderson","year":"2015","journal-title":"eLife"},{"key":"bib24","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1038\/bjc.2012.504","article-title":"Systematic review and meta-analysis of temozolomide in animal models of glioma: was clinical efficacy predicted?","volume":"108","author":"Hirst","year":"2013","journal-title":"British Journal of Cancer"},{"key":"bib25","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511642364","volume-title":"Gene Transfer and the Ethics of First-in-Human Research","author":"Kimmelman","year":"2010"},{"key":"bib26","doi-asserted-by":"publisher","first-page":"488","DOI":"10.1038\/nbt.2261","article-title":"Should preclinical studies be registered?","volume":"30","author":"Kimmelman","year":"2012","journal-title":"Nature Biotechnology"},{"key":"bib27","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1038\/s41571-018-0016-0","article-title":"The paradox of precision medicine","volume":"15","author":"Kimmelman","year":"2018","journal-title":"Nature Reviews Clinical Oncology"},{"key":"bib28","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1038\/nrd1470","article-title":"Can the pharmaceutical industry reduce attrition rates?","volume":"3","author":"Kola","year":"2004","journal-title":"Nature Reviews Drug Discovery"},{"key":"bib29","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.12844","article-title":"Why clinical translation cannot succeed without failure","volume":"4","author":"London","year":"2015","journal-title":"eLife"},{"key":"bib30","doi-asserted-by":"publisher","first-page":"476","DOI":"10.1126\/science.abc1731","article-title":"Against pandemic research exceptionalism","volume":"368","author":"London","year":"2020","journal-title":"Science"},{"key":"bib31","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0063221","article-title":"A survey on data reproducibility in cancer research provides insights into our limited ability to translate findings from the laboratory to the clinic","volume":"8","author":"Mobley","year":"2013","journal-title":"PLOS ONE"},{"key":"bib32","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1038\/542409a","article-title":"No publication without confirmation","volume":"542","author":"Mogil","year":"2017","journal-title":"Nature"},{"key":"bib33","volume-title":"First Course in Probability and Statistics","author":"Neyman","year":"1950"},{"key":"bib34","doi-asserted-by":"publisher","DOI":"10.1126\/science.aac4716","article-title":"Estimating the reproducibility of psychological science","volume":"349","author":"Open Science Collaboration","year":"2015","journal-title":"Science"},{"key":"bib35","doi-asserted-by":"publisher","DOI":"10.1101\/2021.11.30.470372","article-title":"Replication Study: RAF Inhibitors Prime Wild-Type RAF to Activate the MAPK Pathway and Enhance Growth","volume-title":"bioRxiv","author":"Pelech","year":"2021"},{"key":"bib36","doi-asserted-by":"publisher","DOI":"10.1038\/nrd3439-c1","article-title":"Believe it or not: How much can we rely on published data on potential drug targets?","volume":"10","author":"Prinz","year":"2011","journal-title":"Nature Reviews Drug Discovery"},{"key":"bib37","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.1000344","article-title":"Publication bias in reports of animal stroke studies leads to major overstatement of efficacy","volume":"8","author":"Sena","year":"2010","journal-title":"PLOS Biology"},{"key":"bib38","doi-asserted-by":"publisher","first-page":"1359","DOI":"10.1177\/0956797611417632","article-title":"False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant","volume":"22","author":"Simmons","year":"2011","journal-title":"Psychological Science"},{"key":"bib39","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0223758","article-title":"Publication rates in animal research. Extent and characteristics of published and non-published animal studies followed up at two German university medical centres","volume":"14","author":"Wieschowski","year":"2019","journal-title":"PLOS ONE"},{"key":"bib40","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1016\/j.jclinepi.2020.12.006","article-title":"Systematic review and narrative review lead experts to different cancer trial predictions: a randomized trial","volume":"132","author":"Yu","year":"2021","journal-title":"Journal of Clinical Epidemiology"}],"container-title":["eLife"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/cdn.elifesciences.org\/articles\/67527\/elife-67527-v1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/cdn.elifesciences.org\/articles\/67527\/elife-67527-v1.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/elifesciences.org\/articles\/67527","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T21:13:10Z","timestamp":1697058790000},"score":1,"resource":{"primary":{"URL":"https:\/\/elifesciences.org\/articles\/67527"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,7]]},"references-count":40,"alternative-id":["10.7554\/eLife.67527"],"URL":"https:\/\/doi.org\/10.7554\/elife.67527","archive":["CLOCKSS"],"relation":{"has-review":[{"id-type":"doi","id":"10.7554\/eLife.67527.sa1","asserted-by":"object"},{"id-type":"doi","id":"10.7554\/eLife.67527.sa2","asserted-by":"object"}]},"ISSN":["2050-084X"],"issn-type":[{"value":"2050-084X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,7]]},"article-number":"e67527"}}