{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T05:47:43Z","timestamp":1773467263513,"version":"3.50.1"},"reference-count":54,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2023,2,21]],"date-time":"2023-02-21T00:00:00Z","timestamp":1676937600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["RM1HG009034"],"award-info":[{"award-number":["RM1HG009034"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["U2COD023196"],"award-info":[{"award-number":["U2COD023196"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,4,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>The All of Us Research Program makes individual-level data available to researchers while protecting the participants\u2019 privacy. This article describes the protections embedded in the multistep access process, with a particular focus on how the data was transformed to meet generally accepted re-identification risk levels.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>At the time of the study, the resource consisted of 329\u00a0084 participants. Systematic amendments were applied to the data to mitigate re-identification risk (eg, generalization of geographic regions, suppression of public events, and randomization of dates). We computed the re-identification risk for each participant using a state-of-the-art adversarial model specifically assuming that it is known that someone is a participant in the program. We confirmed the expected risk is no greater than 0.09, a threshold that is consistent with guidelines from various US state and federal agencies. We further investigated how risk varied as a function of participant demographics.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The results indicated that 95th percentile of the re-identification risk of all the participants is below current thresholds. At the same time, we observed that risk levels were higher for certain race, ethnic, and genders.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>While the re-identification risk was sufficiently low, this does not imply that the system is devoid of risk. Rather, All of Us uses a multipronged data protection strategy that includes strong authentication practices, active monitoring of data misuse, and penalization mechanisms for users who violate terms of service.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocad021","type":"journal-article","created":{"date-parts":[[2023,2,21]],"date-time":"2023-02-21T19:31:22Z","timestamp":1677007882000},"page":"907-914","source":"Crossref","is-referenced-by-count":11,"title":["Managing re-identification risks while providing access to the <i>All of Us<\/i> research program"],"prefix":"10.1093","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0406-4944","authenticated-orcid":false,"given":"Weiyi","family":"Xia","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"given":"Melissa","family":"Basford","sequence":"additional","affiliation":[{"name":"Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"given":"Robert","family":"Carroll","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0308-4110","authenticated-orcid":false,"given":"Ellen Wright","family":"Clayton","sequence":"additional","affiliation":[{"name":"Law School, Vanderbilt University , Nashville, Tennessee, USA"},{"name":"Department of Pediatrics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"},{"name":"Department of Health Policy, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"given":"Paul","family":"Harris","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"},{"name":"Department of Biomedical Engineering, Vanderbilt University , Nashville, Tennessee, USA"}]},{"given":"Murat","family":"Kantacioglu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Texas at Dallas , Dallas, Texas, USA"}]},{"given":"Yongtai","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Vanderbilt University , Nashville, Tennessee, USA"}]},{"given":"Steve","family":"Nyemba","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"given":"Yevgeniy","family":"Vorobeychik","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Washington University in St. Louis , St. Louis, Missouri, USA"}]},{"given":"Zhiyu","family":"Wan","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]},{"given":"Bradley A","family":"Malin","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"},{"name":"Department of Computer Science, Vanderbilt University , Nashville, Tennessee, USA"},{"name":"Department of Biostatistics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}]}],"member":"286","published-online":{"date-parts":[[2023,2,21]]},"reference":[{"issue":"7","key":"2023041909001672600_ocad021-B1","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1056\/NEJMsr1809937","article-title":"The \u201cAll of Us\u201d Research Program","volume":"381","author":"All of Us Research Program Investigators.","year":"2019","journal-title":"N Engl J Med"},{"key":"2023041909001672600_ocad021-B2","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1038\/gim.2016.183","article-title":"The Precision Medicine Initiative\u2019s All of Us Research Program: an agenda for research on its ethical, legal, and social issues","volume":"19","author":"Sankar","year":"2016","journal-title":"Genet Med"},{"issue":"5","key":"2023041909001672600_ocad021-B3","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1377\/hlthaff.2017.1624","article-title":"Precision medicine: from science to value","volume":"37","author":"Ginsburg","year":"2018","journal-title":"Health Affairs (Project Hope)"},{"issue":"5","key":"2023041909001672600_ocad021-B4","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1002\/humu.22080","article-title":"Deep phenotyping for precision medicine","volume":"33","author":"Robinson","year":"2012","journal-title":"Hum Mutat"},{"issue":"2","key":"2023041909001672600_ocad021-B5","doi-asserted-by":"crossref","first-page":"e16","DOI":"10.2196\/mental.5165","article-title":"New tools for new research in psychiatry: a scalable and customizable platform to empower data driven smartphone research","volume":"3","author":"Torous","year":"2016","journal-title":"JMIR Ment Health"},{"issue":"1","key":"2023041909001672600_ocad021-B6","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1186\/s12911-018-0719-2","article-title":"Big data hurdles in precision medicine and precision public health","volume":"18","author":"Prosperi","year":"2018","journal-title":"BMC Med Inform Decis Mak"},{"issue":"6990","key":"2023041909001672600_ocad021-B7","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1038\/nature02628","article-title":"The case for a US prospective cohort study of genes and environment","volume":"429","author":"Collins","year":"2004","journal-title":"Nature"},{"issue":"7624","key":"2023041909001672600_ocad021-B8","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1038\/538161a","article-title":"Genomics is failing on diversity","volume":"538","author":"Popejoy","year":"2016","journal-title":"Nature"},{"key":"2023041909001672600_ocad021-B9","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.ajo.2021.01.008","article-title":"Predictive analytics for glaucoma using data from the All of Us Research Program","volume":"227","author":"Baxter","year":"2021","journal-title":"Am J Ophthalmol"},{"issue":"1","key":"2023041909001672600_ocad021-B10","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1186\/s12967-018-1585-5","article-title":"The new era of precision population health: insights for the All of Us Research Program and beyond","volume":"16","author":"Lyles","year":"2018","journal-title":"J Transl Med"},{"key":"2023041909001672600_ocad021-B11","author":"Bohnert","year":"2019"},{"issue":"6045","key":"2023041909001672600_ocad021-B12","doi-asserted-by":"crossref","first-page":"940","DOI":"10.1126\/science.1211704","article-title":"Weaving a richer tapestry in biomedical","volume":"333","author":"Tabak","year":"2011","journal-title":"Science"},{"issue":"12","key":"2023041909001672600_ocad021-B13","doi-asserted-by":"crossref","first-page":"e1001918","DOI":"10.1371\/journal.pmed.1001918","article-title":"Diversity in clinical and biomedical research: a promise yet to be fulfilled","volume":"12","author":"Oh","year":"2015","journal-title":"PLoS Med"},{"issue":"00","key":"2023041909001672600_ocad021-B14","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1111\/joim.12955","article-title":"The advantages of UK Biobank\u2019s open-access strategy for health research","volume":"286","author":"Conroy","year":"2019","journal-title":"J Inter Med"},{"key":"2023041909001672600_ocad021-B15"},{"key":"2023041909001672600_ocad021-B16"},{"key":"2023041909001672600_ocad021-B17"},{"key":"2023041909001672600_ocad021-B18"},{"key":"2023041909001672600_ocad021-B19","first-page":"607","article-title":"Biomedical research cohort membership disclosure on social media","volume":"2019","author":"Liu","year":"2019","journal-title":"AMIA Annu Symp Proc"},{"key":"2023041909001672600_ocad021-B20","year":"2021"},{"key":"2023041909001672600_ocad021-B21"},{"key":"2023041909001672600_ocad021-B22"},{"key":"2023041909001672600_ocad021-B23"},{"issue":"3","key":"2023041909001672600_ocad021-B24","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1136\/jamia.2009.002725","article-title":"The disclosure of diagnosis codes can breach research participants\u2019 privacy","volume":"17","author":"Loukides","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2023041909001672600_ocad021-B25","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1038\/clpt.2008.89","article-title":"Development of a large-scale de-identified DNA Biobank to enable personalized medicine","volume":"84","author":"Roden","year":"2008","journal-title":"Clin Pharmacol Ther"},{"key":"2023041909001672600_ocad021-B26"},{"issue":"4","key":"2023041909001672600_ocad021-B27","first-page":"744","article-title":"Enabling realistic health data re-identification risk assessment through adversarial modeling","volume":"28","author":"Xia","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"50","key":"2023041909001672600_ocad021-B28","doi-asserted-by":"crossref","first-page":"eabe9986","DOI":"10.1126\/sciadv.abe9986","article-title":"Using game theory to thwart multistage privacy intrusions when sharing data","volume":"7","author":"Wan","year":"2021","journal-title":"Sci Adv"},{"key":"2023041909001672600_ocad021-B29","author":"Sweeney","year":"2000"},{"issue":"1","key":"2023041909001672600_ocad021-B30","doi-asserted-by":"crossref","first-page":"e33","DOI":"10.2196\/jmir.2001","article-title":"De-identification methods for open health data: the case of the Heritage Health Prize claims dataset","volume":"14","author":"El Emam","year":"2012","journal-title":"J Med Internet Res"},{"issue":"12","key":"2023041909001672600_ocad021-B31","doi-asserted-by":"crossref","first-page":"e28071","DOI":"10.1371\/journal.pone.0028071","article-title":"A systematic review of re-identification attacks on health data","volume":"6","author":"Emam","year":"2011","journal-title":"PLoS One"},{"key":"2023041909001672600_ocad021-B32","author":"Sweeney"},{"issue":"1","key":"2023041909001672600_ocad021-B33","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1186\/s13063-020-4120-y","article-title":"Evaluating the re-identification risk of a clinical study report anonymized under EMA Policy 0070 and Health Canada Regulations","volume":"21","author":"Branson","year":"2020","journal-title":"Trials"},{"key":"2023041909001672600_ocad021-B34","article-title":"Re-identification risks in HIPAA Safe Harbor data: a study of data from one environmental health study","author":"Sweeney","journal-title":"Technol Sci"},{"issue":"2","key":"2023041909001672600_ocad021-B35","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1136\/jamia.2009.000026","article-title":"Evaluating re-identification risks with respect to the HIPAA privacy rule","volume":"17","author":"Benitez","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2023041909001672600_ocad021-B36","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1093\/jamia\/ocv004","article-title":"R-U policy frontiers for health data de-identification","volume":"22","author":"Xia","year":"2015","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2023041909001672600_ocad021-B37","doi-asserted-by":"crossref","first-page":"e0120592","DOI":"10.1371\/journal.pone.0120592","article-title":"A game theoretic framework for analyzing re-identification risk","volume":"10","author":"Wan","year":"2015","journal-title":"PLoS One"},{"key":"2023041909001672600_ocad021-B38","first-page":"59","author":"Xia","year":"2013"},{"key":"2023041909001672600_ocad021-B39","first-page":"1021","author":"Xia","year":"2015"},{"key":"2023041909001672600_ocad021-B40","year":"2020"},{"key":"2023041909001672600_ocad021-B41"},{"key":"2023041909001672600_ocad021-B42","author":"Sayce","year":"2021"},{"key":"2023041909001672600_ocad021-B43","first-page":"476","author":"Zhang","year":"2016"},{"key":"2023041909001672600_ocad021-B44","first-page":"10","author":"Liu","year":"2013"},{"key":"2023041909001672600_ocad021-B45","first-page":"590","author":"Chen","year":"2015"},{"key":"2023041909001672600_ocad021-B46","first-page":"20","author":"Aletras","year":"2018"},{"key":"2023041909001672600_ocad021-B47","first-page":"759","author":"Cheng","year":"2010"},{"key":"2023041909001672600_ocad021-B48","first-page":"83","author":"Peddinti","year":"2014"},{"key":"2023041909001672600_ocad021-B49"},{"key":"2023041909001672600_ocad021-B50","year":"2022"},{"key":"2023041909001672600_ocad021-B51","year":"2016"},{"key":"2023041909001672600_ocad021-B52","year":"2018"},{"key":"2023041909001672600_ocad021-B53"},{"key":"2023041909001672600_ocad021-B54","year":"2017"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/5\/907\/49872984\/ocad021.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/5\/907\/49872984\/ocad021.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,19]],"date-time":"2023-04-19T09:18:19Z","timestamp":1681895899000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/30\/5\/907\/7049587"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,21]]},"references-count":54,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,2,21]]},"published-print":{"date-parts":[[2023,4,19]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocad021","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,5,1]]},"published":{"date-parts":[[2023,2,21]]}}}