{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,26]],"date-time":"2026-07-26T06:29:02Z","timestamp":1785047342522,"version":"3.55.0"},"reference-count":18,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2020,8,22]],"date-time":"2020-08-22T00:00:00Z","timestamp":1598054400000},"content-version":"vor","delay-in-days":21,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS 1251151"],"award-info":[{"award-number":["IIS 1251151"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01 LM006910"],"award-info":[{"award-number":["R01 LM006910"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Australian National Health and Medical Research Council","award":["GNT1157506"],"award-info":[{"award-number":["GNT1157506"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Evidence derived from existing health-care data, such as administrative claims and electronic health records, can fill evidence gaps in medicine. However, many claim such data cannot be used to estimate causal treatment effects because of the potential for observational study bias; for example, due to residual confounding. Other concerns include P hacking and publication bias.<\/jats:p>\n               <jats:p>In response, the Observational Health Data Sciences and Informatics international collaborative launched the Large-scale Evidence Generation and Evaluation across a Network of Databases (LEGEND) research initiative. Its mission is to generate evidence on the effects of medical interventions using observational health-care databases while addressing the aforementioned concerns by following a recently proposed paradigm. We define 10 principles of LEGEND that enshrine this new paradigm, prescribing the generation and dissemination of evidence on many research questions at once; for example, comparing all treatments for a disease for many outcomes, thus preventing publication bias. These questions are answered using a prespecified and systematic approach, avoiding P hacking. Best-practice statistical methods address measured confounding, and control questions (research questions where the answer is known) quantify potential residual bias. Finally, the evidence is generated in a network of databases to assess consistency by sharing open-source analytics code to enhance transparency and reproducibility, but without sharing patient-level information.<\/jats:p>\n               <jats:p>Here we detail the LEGEND principles and provide a generic overview of a LEGEND study. Our companion paper highlights an example study on the effects of hypertension treatments, and evaluates the internal and external validity of the evidence we generate.<\/jats:p>","DOI":"10.1093\/jamia\/ocaa103","type":"journal-article","created":{"date-parts":[[2020,5,14]],"date-time":"2020-05-14T22:08:59Z","timestamp":1589494139000},"page":"1331-1337","source":"Crossref","is-referenced-by-count":47,"title":["Principles of Large-scale Evidence Generation and Evaluation across a Network of Databases (LEGEND)"],"prefix":"10.1093","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0817-5361","authenticated-orcid":false,"given":"Martijn J","family":"Schuemie","sequence":"first","affiliation":[{"name":"Epidemiology Analytics, Janssen Research and Development, Titusville, New Jersey, USA"},{"name":"Department of Biostatistics, Fielding School of Public Health, University of California, Los Angeles, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Patrick B","family":"Ryan","sequence":"additional","affiliation":[{"name":"Epidemiology Analytics, Janssen Research and Development, Titusville, New Jersey, USA"},{"name":"Department of Biomedical Informatics, Columbia University Medical Center, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicole","family":"Pratt","sequence":"additional","affiliation":[{"name":"Quality Use of Medicines and Pharmacy Research Centre, University of South Australia, Adelaide, South Australia, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5281-4143","authenticated-orcid":false,"given":"RuiJun","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University Medical Center, New York, New York, USA"},{"name":"Department of Medicine, Weill Cornell Medical College, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seng Chan","family":"You","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Harlan M","family":"Krumholz","sequence":"additional","affiliation":[{"name":"Department of Medicine, Yale University School of Medicine, New Haven, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Madigan","sequence":"additional","affiliation":[{"name":"Department of Statistics, Columbia University, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"George","family":"Hripcsak","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University Medical Center, New York, New York, USA"},{"name":"Medical Informatics Services, New York-Presbyterian Hospital, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc A","family":"Suchard","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Fielding School of Public Health, University of California, Los Angeles, California, USA"},{"name":"Department of Biomathematics, David Geffen School of Medicine at UCLA, University of California, Los Angeles, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,9,10]]},"reference":[{"key":"2020110613105226300_ocaa103-B1","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/S0140-6736(18)32840-X","article-title":"Real-world studies no substitute for RCTs in establishing efficacy","volume":"393","author":"Gerstein","year":"2019","journal-title":"Lancet"},{"key":"2020110613105226300_ocaa103-B2","doi-asserted-by":"crossref","first-page":"3417","DOI":"10.1093\/eurheartj\/ehy407","article-title":"Association is not causation: treatment effects cannot be estimated from observational data in heart failure","volume":"39","author":"Rush","year":"2018","journal-title":"Eur Heart J"},{"issue":"8","key":"2020110613105226300_ocaa103-B3","doi-asserted-by":"crossref","first-page":"e124","DOI":"10.1371\/journal.pmed.0020124","article-title":"Why most published research findings are false","volume":"2","author":"Ioannidis","year":"2005","journal-title":"PLOS Med"},{"key":"2020110613105226300_ocaa103-B4","first-page":"574","article-title":"Observational health data sciences and informatics (OHDSI): opportunities for observational researchers","volume":"216","author":"Hripcsak","year":"2015","journal-title":"Stud Health Technol Inform"},{"key":"2020110613105226300_ocaa103-B5","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2017.0356","article-title":"Improving reproducibility by using high-throughput observational studies with empirical calibration","volume":"376","author":"Schuemie","year":"2018","journal-title":"Philos Trans A Math Phys Eng Sci"},{"key":"2020110613105226300_ocaa103-B6","article-title":"Large-Scale Evidence Generation and Evaluation across a Network of Databases (LEGEND): Assessing Validity Using Hypertension as a Case Study","author":"Schuemie","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613105226300_ocaa103-B7","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1002\/sim.5925","article-title":"Interpreting observational studies: why empirical calibration is needed to correct P-values","volume":"33","author":"Schuemie","year":"2014","journal-title":"Stat Med"},{"key":"2020110613105226300_ocaa103-B8","doi-asserted-by":"crossref","first-page":"2571","DOI":"10.1073\/pnas.1708282114","article-title":"Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data","volume":"115","author":"Schuemie","year":"2018","journal-title":"Proc Natl Acad Sci USA"},{"key":"2020110613105226300_ocaa103-B9","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1001\/jama.2012.96867","article-title":"Prespecified falsification end points: can they validate true observational associations?","volume":"309","author":"Prasad","year":"2013","journal-title":"JAMA"},{"key":"2020110613105226300_ocaa103-B10","doi-asserted-by":"crossref","first-page":"1432","DOI":"10.1111\/1475-6773.12279","article-title":"Control outcomes and exposures for improving internal validity of nonrandomized studies","volume":"50","author":"Dusetzina","year":"2015","journal-title":"Health Serv Res"},{"key":"2020110613105226300_ocaa103-B11","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1097\/EDE.0b013e3181d61eeb","article-title":"Negative controls: a tool for detecting confounding and bias in observational studies","volume":"21","author":"Lipsitch","year":"2010","journal-title":"Epidemiology"},{"key":"2020110613105226300_ocaa103-B12","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.jbi.2016.12.005","article-title":"Accuracy of an automated knowledge base for identifying drug adverse reactions","volume":"66","author":"Voss","year":"2017","journal-title":"J Biomed Inform"},{"key":"2020110613105226300_ocaa103-B13","doi-asserted-by":"crossref","first-page":"758","DOI":"10.1093\/aje\/kwv254","article-title":"Using big data to emulate a target trial when a randomized trial is not available","volume":"183","author":"Hern\u00e1n","year":"2016","journal-title":"Am J Epidemiol"},{"key":"2020110613105226300_ocaa103-B14","doi-asserted-by":"crossref","first-page":"2005","DOI":"10.1093\/ije\/dyy120","article-title":"Evaluating large-scale propensity score performance through real-world and synthetic data experiments","volume":"47","author":"Tian","year":"2018","journal-title":"Int J Epidemiol"},{"key":"2020110613105226300_ocaa103-B15","doi-asserted-by":"publisher","author":"Schuemie","year":"2020","DOI":"10.1162\/99608f92.147cc28e"},{"key":"2020110613105226300_ocaa103-B16","doi-asserted-by":"crossref","first-page":"1768","DOI":"10.1002\/sim.2302","article-title":"Tutorial in biostatistics: the self-controlled case series method","volume":"25","author":"Whitaker","year":"2006","journal-title":"Stat Med"},{"key":"2020110613105226300_ocaa103-B17","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1136\/bmj.39490.551019.BE","article-title":"What is \u2018quality of evidence\u2019 and why is it important to clinicians?","volume":"336","author":"Guyatt","year":"2008","journal-title":"BMJ"},{"key":"2020110613105226300_ocaa103-B18","first-page":"1816","article-title":"Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis","author":"Suchard","journal-title":"Lancet"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/8\/1331\/34152911\/ocaa103.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/8\/1331\/34152911\/ocaa103.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T19:35:34Z","timestamp":1604691334000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/8\/1331\/5895561"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,1]]},"references-count":18,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,9,10]]},"published-print":{"date-parts":[[2020,8,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa103","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,8]]},"published":{"date-parts":[[2020,8,1]]}}}