{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T06:49:55Z","timestamp":1774939795314,"version":"3.50.1"},"reference-count":23,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2020,8,22]],"date-time":"2020-08-22T00:00:00Z","timestamp":1598054400000},"content-version":"vor","delay-in-days":21,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS 1251151"],"award-info":[{"award-number":["IIS 1251151"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01 LM006910"],"award-info":[{"award-number":["R01 LM006910"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Australian National Health and Medical Research Council","award":["GNT1157506"],"award-info":[{"award-number":["GNT1157506"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>To demonstrate the application of the Large-scale Evidence Generation and Evaluation across a Network of Databases (LEGEND) principles described in our companion article to hypertension treatments and assess internal and external validity of the generated evidence.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>LEGEND defines a process for high-quality observational research based on 10 guiding principles. We demonstrate how this process, here implemented through large-scale propensity score modeling, negative and positive control questions, empirical calibration, and full transparency, can be applied to compare antihypertensive drug therapies. We assess internal validity through covariate balance, confidence-interval coverage, between-database heterogeneity, and transitivity of results. We assess external validity through comparison to direct meta-analyses of randomized controlled trials (RCTs).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>From 21.6 million unique antihypertensive new users, we generate 6\u00a0076\u00a0775 effect size estimates for 699\u00a0872 research questions on 12\u00a0946 treatment comparisons. Through propensity score matching, we achieve balance on all baseline patient characteristics for 75% of estimates, observe 95.7% coverage in our effect-estimate 95% confidence intervals, find high between-database consistency, and achieve transitivity in 84.8% of triplet hypotheses. Compared with meta-analyses of RCTs, our results are consistent with 28 of 30 comparisons while providing narrower confidence intervals.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>We find that these LEGEND results show high internal validity and are congruent with meta-analyses of RCTs. For these reasons we believe that evidence generated by LEGEND is of high quality and can inform medical decision-making where evidence is currently lacking. Subsequent publications will explore the clinical interpretations of this evidence.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaa124","type":"journal-article","created":{"date-parts":[[2020,6,1]],"date-time":"2020-06-01T19:15:35Z","timestamp":1591038935000},"page":"1268-1277","source":"Crossref","is-referenced-by-count":27,"title":["Large-scale evidence generation and evaluation across a network of databases (LEGEND): assessing validity using hypertension as a case study"],"prefix":"10.1093","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0817-5361","authenticated-orcid":false,"given":"Martijn J","family":"Schuemie","sequence":"first","affiliation":[{"name":"Epidemiology Analytics, Janssen Research and Development, Titusville, New Jersey, USA"},{"name":"Department of Biostatistics, Fielding School of Public Health, University of California, Los Angeles, Los Angeles, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Patrick B","family":"Ryan","sequence":"additional","affiliation":[{"name":"Epidemiology Analytics, Janssen Research and Development, Titusville, New Jersey, USA"},{"name":"Department of Biomedical Informatics, Columbia University Medical Center, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicole","family":"Pratt","sequence":"additional","affiliation":[{"name":"Quality Use of Medicines and Pharmacy Research Centre, University of South Australia, Adelaide, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5281-4143","authenticated-orcid":false,"given":"RuiJun","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University Medical Center, New York, New York, USA"},{"name":"Department of Medicine, Weill Cornell Medical College, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seng Chan","family":"You","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Harlan M","family":"Krumholz","sequence":"additional","affiliation":[{"name":"Department of Medicine, Yale University School of Medicine, New Haven, Connecticut, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Madigan","sequence":"additional","affiliation":[{"name":"Department of Statistics, Columbia University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"George","family":"Hripcsak","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University Medical Center, New York, New York, USA"},{"name":"Medical Informatics Services, New York-Presbyterian Hospital, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marc A","family":"Suchard","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Fielding School of Public Health, University of California, Los Angeles, Los Angeles, California, USA"},{"name":"Department of Biomathematics, David Geffen School of Medicine at UCLA, University of California, Los Angeles, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2020,9,10]]},"reference":[{"key":"2021042918155157100_ocaa124-B1","first-page":"574","article-title":"Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers","volume":"216","author":"Hripcsak.","year":"2015","journal-title":"Stud Health Technol Inform"},{"key":"2021042918155157100_ocaa124-B2","article-title":"Principles of Large-Scale Evidence Generation and Evaluation across a Network of Databases (LEGEND)","author":"Schuemie","journal-title":"J\u00a0Am Med Inform Association"},{"key":"2021042918155157100_ocaa124-B3","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1080\/09332480.2019.1579573","article-title":"Why Most Published Research Findings Are False","volume":"32","author":"Ioannidis","year":"2019","journal-title":"CHANCE"},{"key":"2021042918155157100_ocaa124-B4","doi-asserted-by":"crossref","first-page":"3417","DOI":"10.1093\/eurheartj\/ehy407","article-title":"Association is not causation: treatment effects cannot be estimated from observational data in heart failure","volume":"39","author":"Rush","year":"2018","journal-title":"Eur Heart J"},{"key":"2021042918155157100_ocaa124-B5","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/S0140-6736(18)32840-X","article-title":"Real-world studies no substitute for RCTs in establishing efficacy","volume":"393","author":"","year":"2019","journal-title":"Lancet"},{"key":"2021042918155157100_ocaa124-B6","article-title":"Improving reproducibility by using high-throughput observational studies with empirical calibration","volume":"376","author":"Schuemie","year":"2018","journal-title":"Philos Trans A Math Phys Eng Sci"},{"issue":"6","key":"2021042918155157100_ocaa124-B7","doi-asserted-by":"crossref","first-page":"2005","DOI":"10.1093\/ije\/dyy120","article-title":"Evaluating large-scale propensity score performance through real-world and synthetic data experiments","volume":"47","author":"Tian","year":"2018","journal-title":"Int J Epidemiol"},{"issue":"11","key":"2021042918155157100_ocaa124-B8","doi-asserted-by":"crossref","first-page":"2571","DOI":"10.1073\/pnas.1708282114","article-title":"Empirical CI calibration for population-level effect estimation studies in observational healthcare data","volume":"115","author":"Schuemie","year":"2018","journal-title":"Proc Natl Acad Sci USA"},{"issue":"2","key":"2021042918155157100_ocaa124-B9","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1002\/sim.5925","article-title":"Interpreting observational studies: why empirical calibration is needed to correct p-values","volume":"33","author":"Schuemie","year":"2014","journal-title":"Stat Med"},{"key":"2021042918155157100_ocaa124-B10","doi-asserted-by":"crossref","first-page":"e595","DOI":"10.1161\/CIR.0000000000000601","article-title":"Systematic review for the 2017 ACC\/AHA\/AAPA\/ABC\/ACPM\/AGS\/APhA\/ASH\/ASPC\/NMA\/PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: a report of the American College of Cardiology\/American Heart Association Task Force on Clinical Practice Guidelines","volume":"138","author":"Reboussin","year":"2018","journal-title":"Circulation"},{"issue":"6","key":"2021042918155157100_ocaa124-B11","doi-asserted-by":"crossref","first-page":"1269","DOI":"10.1161\/HYP.0000000000000066","article-title":"2017 ACC\/AHA\/AAPA\/ABC\/ACPM\/AGS\/APhA\/ASH\/ASPC\/NMA\/PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: executive summary: a report of the American College of Cardiology\/American Heart Association Task Force on Clinical Practice Guidelines","volume":"71","author":"Whelton","year":"2018","journal-title":"Hypertension"},{"issue":"10","key":"2021042918155157100_ocaa124-B12","doi-asserted-by":"crossref","first-page":"1953","DOI":"10.1097\/HJH.0000000000001940","article-title":"ESC\/ESH Guidelines for the management of arterial hypertension: The Task Force for the management of arterial hypertension of the European Society of Cardiology and the European Society of Hypertension: The Task Force for the management of arterial hypertension of the European Society of Cardiology and the European Society of","volume":"36","author":"Williams","year":"2018","journal-title":"Hypertension. J Hypertens"},{"key":"2021042918155157100_ocaa124-B13","first-page":"1816","article-title":"Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis","author":"Suchard","journal-title":"Lancet"},{"key":"2021042918155157100_ocaa124-B14","author":"Schuemie","year":"2018"},{"issue":"1","key":"2021042918155157100_ocaa124-B15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2414416.2414791","article-title":"Massive parallelization of serial inference algorithms for a complex generalized linear model","volume":"23","author":"Suchard","year":"2013","journal-title":"ACM Trans Model Comput Simul"},{"key":"2021042918155157100_ocaa124-B16","author":"Rosenbaum","journal-title":"The Central Role of the Propensity Score in Observational Studies for Causal Effects"},{"key":"2021042918155157100_ocaa124-B17","doi-asserted-by":"crossref","first-page":"11","DOI":"10.2147\/CER.S40357","article-title":"A tool for assessing the feasibility of comparative effectiveness research","volume":"3","author":"Walker","year":"2013","journal-title":"Comparative Effect Res"},{"key":"2021042918155157100_ocaa124-B18","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.jbi.2016.12.005","article-title":"Accuracy of an automated knowledge base for identifying drug adverse reactions","volume":"66","author":"Voss","year":"2017","journal-title":"J Biomed Inform"},{"issue":"3","key":"2021042918155157100_ocaa124-B19","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/0197-2456(86)90046-2","article-title":"Meta-analysis in clinical trials","volume":"7","author":"DerSimonian","year":"1986","journal-title":"Control Clin Trials"},{"issue":"7414","key":"2021042918155157100_ocaa124-B20","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1136\/bmj.327.7414.557","article-title":"Measuring inconsistency in meta-analyses","volume":"327","author":"Higgins","year":"2003","journal-title":"BMJ"},{"key":"2021042918155157100_ocaa124-B21","author":"Schuemie","year":"2020","journal-title":"How Confident Are We about Observational Findings in Health Care: A Benchmark Study. 2.1"},{"issue":"25","key":"2021042918155157100_ocaa124-B22","doi-asserted-by":"crossref","first-page":"3083","DOI":"10.1002\/sim.3697","article-title":"Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples","volume":"28","author":"Austin","year":"2009","journal-title":"Stat Med"},{"issue":"2","key":"2021042918155157100_ocaa124-B23","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1093\/biomet\/asn004","article-title":"The prognostic analogue of the propensity score","volume":"95","author":"Hansen","year":"2008","journal-title":"Biometrika"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/8\/1268\/37606722\/ocaa124.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/8\/1268\/37606722\/ocaa124.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T18:16:30Z","timestamp":1619720190000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/8\/1268\/5895564"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,1]]},"references-count":23,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,9,10]]},"published-print":{"date-parts":[[2020,8,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa124","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,8]]},"published":{"date-parts":[[2020,8,1]]}}}