{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T08:00:20Z","timestamp":1773475220900,"version":"3.50.1"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T00:00:00Z","timestamp":1639958400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T00:00:00Z","timestamp":1639958400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000038","name":"U.S. Department of Health & Human Services | U.S. Food and Drug Administration","doi-asserted-by":"publisher","award":["75F40119D10037"],"award-info":[{"award-number":["75F40119D10037"]}],"id":[{"id":"10.13039\/100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The Sentinel System is a major component of the United States Food and Drug Administration\u2019s (FDA) approach to active medical product safety surveillance. While Sentinel has historically relied on large quantities of health insurance claims data, leveraging longitudinal electronic health records (EHRs) that contain more detailed clinical information, as structured and unstructured features, may address some of the current gaps in capabilities. We identify key challenges when using EHR data to investigate medical product safety in a scalable and accelerated way, outline potential solutions, and describe the Sentinel Innovation Center\u2019s initiatives to put solutions into practice by expanding and strengthening the existing system with a query-ready, large-scale data infrastructure of linked EHR and claims data. We describe our initiatives in four strategic priority areas: (1) data infrastructure, (2) feature engineering, (3) causal inference, and (4) detection analytics, with the goal of incorporating emerging data science innovations to maximize the utility of EHR data for medical product safety surveillance.<\/jats:p>","DOI":"10.1038\/s41746-021-00542-0","type":"journal-article","created":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T11:13:41Z","timestamp":1639998821000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":54,"title":["Broadening the reach of the FDA Sentinel system: A roadmap for integrating electronic health record data in a causal analysis framework"],"prefix":"10.1038","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0299-7273","authenticated-orcid":false,"given":"Rishi J.","family":"Desai","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3217-4147","authenticated-orcid":false,"given":"Michael E.","family":"Matheny","sequence":"additional","affiliation":[]},{"given":"Kevin","family":"Johnson","sequence":"additional","affiliation":[]},{"given":"Keith","family":"Marsolo","sequence":"additional","affiliation":[]},{"given":"Lesley H.","family":"Curtis","sequence":"additional","affiliation":[]},{"given":"Jennifer C.","family":"Nelson","sequence":"additional","affiliation":[]},{"given":"Patrick J.","family":"Heagerty","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9900-2142","authenticated-orcid":false,"given":"Judith","family":"Maro","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9340-7189","authenticated-orcid":false,"given":"Jeffery","family":"Brown","sequence":"additional","affiliation":[]},{"given":"Sengwee","family":"Toh","sequence":"additional","affiliation":[]},{"given":"Michael","family":"Nguyen","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1609-7420","authenticated-orcid":false,"given":"Robert","family":"Ball","sequence":"additional","affiliation":[]},{"given":"Gerald","family":"Dal Pan","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7761-7090","authenticated-orcid":false,"given":"Shirley V.","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Joshua J.","family":"Gagne","sequence":"additional","affiliation":[]},{"given":"Sebastian","family":"Schneeweiss","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,12,20]]},"reference":[{"key":"542_CR1","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1002\/cpt.320","volume":"99","author":"R Ball","year":"2016","unstructured":"Ball, R., Robb, M., Anderson, S. & Dal Pan, G. The FDA\u2019s sentinel initiative\u2014a comprehensive approach to medical product surveillance. Clin. Pharmacol. Therapeutics. 99, 265\u2013268 (2016).","journal-title":"Clin. Pharmacol. Therapeutics."},{"key":"542_CR2","doi-asserted-by":"publisher","first-page":"2091","DOI":"10.1056\/NEJMp1809643","volume":"379","author":"R Platt","year":"2018","unstructured":"Platt, R. et al. The FDA Sentinel Initiative\u2014an evolving national resource. N. Engl. J. Med. 379, 2091\u20132093 (2018).","journal-title":"N. Engl. J. Med."},{"key":"542_CR3","unstructured":"Sentinel Initiative [Internet]. FDA Advisory Committee Meetings; 2021. https:\/\/www.sentinelinitiative.org\/communications\/fda-advisory-committee-meetings. Accessed March 1, 2021."},{"key":"542_CR4","unstructured":"Sentinel Initiative [Internet]. FDA Safety Communications; 2021. https:\/\/www.sentinelinitiative.org\/communications\/fda-safety-communications. Accessed March 1, 2021"},{"key":"542_CR5","doi-asserted-by":"publisher","first-page":"793","DOI":"10.1093\/jamia\/ocaa028","volume":"27","author":"JS Brown","year":"2020","unstructured":"Brown, J. S., Maro, J. C., Nguyen, M. & Ball, R. Using and improving distributed data networks to generate actionable evidence: the case of real-world outcomes in the Food and Drug Administration\u2019s Sentinel system. J. Am. Med. Inform. Assoc. 27, 793\u2013797 (2020).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"542_CR6","unstructured":"Gottlieb, S. FDA budget matters: a cross-cutting data enterprise for real world evidence. Food and Drug Administration. https:\/\/www.fda.gov\/news-events\/fda-voices\/fda-budget-matters-cross-cutting-data-enterprise-real-world-evidence (2018)."},{"key":"542_CR7","unstructured":"Sentinel System Five-Year Strategy 2019-2023. Food and Drug Administration, 2019. Available at https:\/\/www.fda.gov\/media\/120333\/download. Accessed February 2, 2021."},{"key":"542_CR8","unstructured":"Pray, L. & Robinson, S. Challenges for the FDA: The Future of Drug Safety: Workshop Summary (National Academies, 2007)."},{"key":"542_CR9","doi-asserted-by":"publisher","first-page":"658","DOI":"10.1210\/endrev\/bnab007","volume":"42","author":"S Schneeweiss","year":"2021","unstructured":"Schneeweiss, S. & Patorno, E. Conducting real-world evidence studies on the clinical outcomes of diabetes treatments. Endocr. Rev. 42, 658\u2013690 (2021).","journal-title":"Endocr. Rev."},{"key":"542_CR10","doi-asserted-by":"publisher","first-page":"895","DOI":"10.1097\/EDE.0000000000000907","volume":"29","author":"SV Wang","year":"2018","unstructured":"Wang, S. V. et al. Data mining for adverse drug events with a propensity score matched tree-based scan statistic. Epidemiol. 29, 895 (2018).","journal-title":"Epidemiol."},{"key":"542_CR11","doi-asserted-by":"publisher","first-page":"1424","DOI":"10.1093\/aje\/kwab034","volume":"190","author":"SV Wang","year":"2021","unstructured":"Wang, S. V. et al. A general propensity score for signal identification using tree-based scan statistics. Am. J. Epidemiol. 190, 1424\u20131433 (2021).","journal-title":"Am. J. Epidemiol."},{"key":"542_CR12","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1002\/sam.11232","volume":"7","author":"JC Nelson","year":"2014","unstructured":"Nelson, J. C. et al. Integrating database knowledge and epidemiological design to improve the implementation of data mining methods that evaluate vaccine safety in large healthcare databases. Stat. Anal. Data Min. 7, 337\u2013351 (2014).","journal-title":"Stat. Anal. Data Min."},{"key":"542_CR13","doi-asserted-by":"publisher","first-page":"758","DOI":"10.1093\/aje\/kwv254","volume":"183","author":"MA Hernan","year":"2016","unstructured":"Hernan, M. A. & Robins, J. M. Using big data to emulate a target trial when a randomized trial is not available. Am. J. Epidemiol. 183, 758\u2013764 (2016).","journal-title":"Am. J. Epidemiol."},{"key":"542_CR14","doi-asserted-by":"publisher","first-page":"1002","DOI":"10.1161\/CIRCULATIONAHA.120.051718","volume":"143","author":"JM Franklin","year":"2020","unstructured":"Franklin, J. M. et al. Emulating randomized clinical trials with nonrandomized real-world evidence studies: first results from the RCT DUPLICATE Initiative. Circulation. 143, 1002\u20131013 (2020).","journal-title":"Circulation."},{"key":"542_CR15","doi-asserted-by":"publisher","first-page":"771","DOI":"10.2147\/CLEP.S166545","volume":"10","author":"S Schneeweiss","year":"2018","unstructured":"Schneeweiss, S. Automated data-adaptive analytics for electronic healthcare data to study causal treatment effects. Clin. Epidemiol. 10, 771 (2018).","journal-title":"Clin. Epidemiol."},{"key":"542_CR16","doi-asserted-by":"publisher","first-page":"827","DOI":"10.1002\/cpt.1577","volume":"107","author":"S Schneeweiss","year":"2020","unstructured":"Schneeweiss, S., Brown, J. S., Bate, A., Trifir\u00f2, G. & Bartels, D. B. Choosing among common data models for real\u2010world data analyses fit for making decisions about the effectiveness of medical products. Clin. Pharmacol. Therapeutics. 107, 827\u2013833 (2020).","journal-title":"Clin. Pharmacol. Therapeutics."},{"key":"542_CR17","doi-asserted-by":"publisher","first-page":"1077","DOI":"10.1002\/pds.4645","volume":"27","author":"R Ball","year":"2018","unstructured":"Ball, R. et al. Evaluating automated approaches to anaphylaxis case classification using unstructured data from the FDA Sentinel System. Pharmacoepidemiology Drug Saf. 27, 1077\u20131084 (2018).","journal-title":"Pharmacoepidemiology Drug Saf."},{"key":"542_CR18","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1097\/EDE.0000000000001330","volume":"32","author":"MA Bann","year":"2021","unstructured":"Bann, M. A. et al. Identification and validation of anaphylaxis using electronic health data in a population-based setting. Epidemiology 32, 439\u2013443 (2021).","journal-title":"Epidemiology"},{"key":"542_CR19","doi-asserted-by":"publisher","first-page":"1507","DOI":"10.1093\/jamia\/ocab036","volume":"28","author":"TB Gibson","year":"2021","unstructured":"Gibson, T. B. et al. Electronic phenotyping of health outcomes of interest using a linked claims-electronic health record database: findings from a machine learning pilot project. J. Am. Med. Inform. Assoc. 28, 1507\u20131517 (2021).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"542_CR20","unstructured":"Shi, X., Li, X. & Cai, T. Spherical regression under mismatch corruption with application to automated knowledge translation. J. Am. Stat. Assoc. 1\u201312 (2020)."},{"key":"542_CR21","doi-asserted-by":"publisher","first-page":"459","DOI":"10.1002\/pds.4107","volume":"26","author":"S Suissa","year":"2017","unstructured":"Suissa, S., Moodie, E. E. & Dell\u2019Aniello, S. Prevalent new-user cohort designs for comparative drug effect studies by time-conditional propensity scores. Pharmacoepidemiol Drug Saf. 26, 459\u2013468 (2017).","journal-title":"Pharmacoepidemiol Drug Saf."},{"key":"542_CR22","doi-asserted-by":"publisher","first-page":"S91","DOI":"10.1016\/j.jclinepi.2013.02.017","volume":"66","author":"SD Lendle","year":"2013","unstructured":"Lendle, S. D., Fireman, B. & van der Laan, M. J. Targeted maximum likelihood estimation in safety analysis. J. Clin. Epidemiol. 66, S91\u2013S98 (2013).","journal-title":"J. Clin. Epidemiol."},{"key":"542_CR23","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1097\/EDE.0000000000000762","volume":"29","author":"R Wyss","year":"2018","unstructured":"Wyss, R. et al. Using super learner prediction modeling to improve high-dimensional propensity score estimation. Epidemiology 29, 96\u2013106 (2018).","journal-title":"Epidemiology"},{"key":"542_CR24","doi-asserted-by":"publisher","first-page":"1227","DOI":"10.2105\/AJPH.2016.303199","volume":"106","author":"TL Lash","year":"2016","unstructured":"Lash, T. L., Fox, M. P., Cooney, D., Lu, Y. & Forshee, R. A. Quantitative bias analysis in regulatory settings. Am. J. Public Health 106, 1227\u20131230 (2016).","journal-title":"Am. J. Public Health"},{"key":"542_CR25","doi-asserted-by":"publisher","first-page":"509","DOI":"10.1097\/EDE.0000000000001209","volume":"31","author":"LJ Collin","year":"2020","unstructured":"Collin, L. J. et al. Adaptive validation design: a Bayesian approach to validation substudy design with prospective data collection. Epidemiol. 31, 509 (2020).","journal-title":"Epidemiol."},{"key":"542_CR26","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1007\/s40264-018-0766-8","volume":"42","author":"F Liu","year":"2019","unstructured":"Liu, F., Jagannatha, A. & Yu, H. Towards drug safety surveillance and pharmacovigilance: current progress in detecting medication and adverse drug events from electronic health records. Drug Saf. 42, 95\u201397 (2019).","journal-title":"Drug Saf."},{"key":"542_CR27","doi-asserted-by":"crossref","first-page":"m4856","DOI":"10.1136\/bmj.m4856","volume":"372","author":"SV Wang","year":"2021","unstructured":"Wang, S. V. et al. STaRT-RWE: structured template for planning and reporting on the implementation of real world evidence studies. BMJ 372, m4856 (2021).","journal-title":"BMJ"},{"key":"542_CR28","doi-asserted-by":"publisher","first-page":"k3532","DOI":"10.1136\/bmj.k3532","volume":"363","author":"SM Langan","year":"2018","unstructured":"Langan, S. M. et al. The reporting of studies conducted using observational routinely collected health data statement for pharmacoepidemiology (RECORD-PE). BMJ 363, k3532 (2018).","journal-title":"BMJ"},{"key":"542_CR29","unstructured":"Innovation Center (IC) Master Plan. Available at https:\/\/www.sentinelinitiative.org\/news-events\/publications-presentations\/innovation-center-ic-master-plan. Accessed February 11, 2021."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00542-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00542-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00542-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,3]],"date-time":"2022-12-03T19:07:05Z","timestamp":1670094425000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00542-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,20]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["542"],"URL":"https:\/\/doi.org\/10.1038\/s41746-021-00542-0","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,20]]},"assertion":[{"value":"23 September 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 November 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 December 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 February 2022","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Update","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"In the original version of this article, the given and family names of Gerald Dal Pan were incorrectly structured. The name was displayed correctly in all versions at the time of publication. The original article has been corrected.","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Dr. Desai reports serving as Principal Investigator on research grants from Bayer and Novartis to the Brigham and Women\u2019s Hospital on studies outside of the submitted work. Dr. Schneeweiss is a consultant to Aetion, a software manufacturer of which he owns equity. His interests were declared, reviewed, and approved by the Brigham and Women\u2019s Hospital and Partners HealthCare System in accordance with their institutional compliance policies. Dr. Ball is an author on US Patent 9,075,796, \u201cText mining for large medical text datasets and corresponding medical text classification using informative feature selection.\u201d At present this patent is not licensed and does not generate royalties. Dr. Gagne is currently an employee of Johnson & Johnson. Dr. Nelson reports research funding from Moderna for service on their safety monitoring committee. The remaining authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"170"}}