{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T04:40:42Z","timestamp":1772858442978,"version":"3.50.1"},"reference-count":36,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2025,1,31]],"date-time":"2025-01-31T00:00:00Z","timestamp":1738281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,12,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>One of the key limitations of electronic health record (EHR) data is that not all health care encounters are observed. The degree to which patient information is captured is referred to as observability. Poor observability, particularly differential observability, can lead to biased estimates and inference. As such, understanding the degree of observability is important in EHR-based studies. In this study, we propose using external data with known observability to assess the degree of overall observability in EHRs. We also construct a test for differential observability in the target EHR data set. Using principles from the transportability literature, we show that we can use a balancing score-based weight to estimate the observability of our target outcome. We conduct a series of simulation experiments to understand the conditions under which data set features must be required to obtain proper inference. To illustrate this, we consider hospital readmissions among patients with end-stage renal disease as our outcome of interest. We use administrative claims data, where the outcome is fully observed, as our external data.<\/jats:p>","DOI":"10.1093\/aje\/kwaf013","type":"journal-article","created":{"date-parts":[[2025,1,30]],"date-time":"2025-01-30T07:25:34Z","timestamp":1738221934000},"page":"3224-3432","source":"Crossref","is-referenced-by-count":1,"title":["Estimating the observability of an outcome from an electronic health record data set using external data"],"prefix":"10.1093","volume":"194","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-5365-4778","authenticated-orcid":false,"given":"Mengying","family":"Yan","sequence":"first","affiliation":[{"name":"Department of Biostatistics & Bioinformatics, Duke University School of Medicine , Durham, NC 27705,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hwanhee","family":"Hong","sequence":"additional","affiliation":[{"name":"Department of Biostatistics & Bioinformatics, Duke University School of Medicine , Durham, NC 27705,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Wilson","sequence":"additional","affiliation":[{"name":"Department of Biostatistics & Bioinformatics, Duke University School of Medicine , Durham, NC 27705,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin A","family":"Goldstein","sequence":"additional","affiliation":[{"name":"Department of Biostatistics & Bioinformatics, Duke University School of Medicine , Durham, NC 27705,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2025,1,31]]},"reference":[{"issue":"5","key":"2025120218122588500_ref1","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1016\/j.spl.2018.02.044","article-title":"The role of statistics in the era of big data: electronic health records for healthcare research","volume":"136","author":"Sharples","year":"2018","journal-title":"Stat Probability Lett."},{"issue":"1","key":"2025120218122588500_ref2","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1093\/jamia\/ocw042","article-title":"Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review","volume":"24","author":"Goldstein","year":"2017","journal-title":"J Am Med Inform Assoc."},{"issue":"8)(suppl 3","key":"2025120218122588500_ref3","doi-asserted-by":"publisher","first-page":"S30","DOI":"10.1097\/MLR.0b013e31829b1dbd","article-title":"Caveats for the use of operational electronic health record data in comparative effectiveness research","volume":"51","author":"Hersh","year":"2013","journal-title":"Med Care."},{"issue":"1","key":"2025120218122588500_ref4","doi-asserted-by":"publisher","first-page":"14","DOI":"10.5334\/egems.218","article-title":"A data quality assessment guideline for electronic health record data reuse","volume":"5","author":"Weiskopf","year":"2017","journal-title":"EGEMS (Wash DC)."},{"issue":"12","key":"2025120218122588500_ref5","doi-asserted-by":"publisher","first-page":"e893","DOI":"10.1016\/S2589-7500(22)00154-6","article-title":"Leveraging electronic health records for data science: common pitfalls and how to avoid them","volume":"4","author":"Sauer","year":"2022","journal-title":"Lancet Digital Health."},{"issue":"1","key":"2025120218122588500_ref6","doi-asserted-by":"publisher","first-page":"22","DOI":"10.5334\/egems.243","article-title":"Illustrating informed presence bias in electronic health records data: how patient interactions with a health system can impact inference","volume":"5","author":"Phelan","year":"2017","journal-title":"EGEMS (Wash DC)"},{"issue":"3","key":"2025120218122588500_ref7","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1093\/biomet\/63.3.581","article-title":"Inference and missing data","volume":"63","author":"Rubin","year":"1976","journal-title":"Biometrika."},{"key":"2025120218122588500_ref8","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1145\/1401890.1401920","volume-title":"Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD 08","author":"Elkan","year":"2008"},{"issue":"4","key":"2025120218122588500_ref9","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1007\/s10994-020-05877-5","article-title":"Learning from positive and unlabeled data: a survey","volume":"109","author":"Bekker","year":"2020","journal-title":"Mach Learn."},{"key":"2025120218122588500_ref10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3465416.3483305","volume-title":"Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization","author":"Suresh","year":"2023"},{"issue":"11","key":"2025120218122588500_ref11","doi-asserted-by":"publisher","first-page":"847","DOI":"10.1093\/aje\/kww112","article-title":"Controlling for informed presence bias due to the number of health encounters in an electronic health record","volume":"184","author":"Goldstein","year":"2016","journal-title":"Am J Epidemiol."},{"issue":"1","key":"2025120218122588500_ref12","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1097\/EDE.0000000000001432","article-title":"On the nature of informative presence bias in analyses of electronic health records","volume":"33","author":"McGee","year":"2022","journal-title":"Epidemiology."},{"issue":"5","key":"2025120218122588500_ref13","doi-asserted-by":"publisher","first-page":"937","DOI":"10.1093\/jamia\/ocac019","article-title":"Observability and its impact on differential bias for clinical prediction models","volume":"29","author":"Yan","year":"2022","journal-title":"J Am Med Inform Assoc."},{"issue":"3","key":"2025120218122588500_ref14","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1097\/EDE.0000000000000794","article-title":"Out-of-system care and recording of patient characteristics critical for comparative effectiveness research","volume":"29","author":"Lin","year":"2018","journal-title":"Epidemiology."},{"issue":"2022","key":"2025120218122588500_ref15","doi-asserted-by":"publisher","first-page":"1339","DOI":"10.2147\/CLEP.S370031","article-title":"Advancing an algorithm for the identification of patients with high data-continuity in electronic health records","volume":"14","author":"Merola","year":"2022","journal-title":"Clin Epidemiol."},{"issue":"2","key":"2025120218122588500_ref16","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1111\/j.1467-985X.2010.00673.x","article-title":"The use of propensity scores to assess the generalizability of results from randomized trials: use of propensity scores to assess generalizability","volume":"174","author":"Stuart","year":"2011","journal-title":"J R Stat Soc A Stat Soc."},{"issue":"1","key":"2025120218122588500_ref17","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1093\/aje\/kwq084","article-title":"Generalizing evidence from randomized clinical trials to target populations: the ACTG 320 trial","volume":"172","author":"Cole","year":"2010","journal-title":"Am J Epidemiol."},{"issue":"1","key":"2025120218122588500_ref18","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1146\/annurev-statistics-042522-103837","article-title":"A review of generalizability and transportability","volume":"10","author":"Degtiar","year":"2023","journal-title":"Annu Rev Stat Appl."},{"issue":"5","key":"2025120218122588500_ref19","doi-asserted-by":"publisher","first-page":"688","DOI":"10.1037\/h0037350","article-title":"Estimating causal effects of treatments in randomized and nonrandomized studies","volume":"66","author":"Rubin","year":"1974","journal-title":"J Educ Psychol."},{"issue":"521","key":"2025120218122588500_ref20","doi-asserted-by":"publisher","first-page":"390","DOI":"10.1080\/01621459.2016.1260466","article-title":"Balancing covariates via propensity score weighting","volume":"113","author":"Li","year":"2018","journal-title":"J Am Stat Assoc."},{"issue":"12","key":"2025120218122588500_ref21","doi-asserted-by":"publisher","first-page":"3721","DOI":"10.1177\/0962280220940334","article-title":"Propensity score weighting under limited overlap and model misspecification","volume":"29","author":"Zhou","year":"2020","journal-title":"Stat Methods Med Res."},{"issue":"4","key":"2025120218122588500_ref22","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1016\/j.csda.2013.10.018","article-title":"Plasmode simulation for the evaluation of pharmacoepidemiologic methods in complex healthcare databases","volume":"72","author":"Franklin","year":"2014","journal-title":"Comput Stat Data Anal."},{"issue":"3","key":"2025120218122588500_ref23","doi-asserted-by":"publisher","first-page":"A8","DOI":"10.1053\/j.ajkd.2022.12.001","article-title":"US Renal Data System 2022 annual data report: epidemiology of kidney disease in the United States","volume":"81","author":"Johansen","year":"2023","journal-title":"Am J Kidney Dis."},{"issue":"11","key":"2025120218122588500_ref24","doi-asserted-by":"publisher","first-page":"2528","DOI":"10.1093\/ndt\/gfad099","article-title":"Patient-focused outcomes after initiation of dialysis for ESRD: mortality, hospitalization and functional impairment","volume":"38","author":"Kolbrink","year":"2023","journal-title":"Nephrol Dial Transplant."},{"issue":"1","key":"2025120218122588500_ref25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1017\/cts.2020.499","article-title":"Development of an electronic health records datamart to support clinical and population health research","volume":"5","author":"Hurst","year":"2020","journal-title":"J Clin Transl Sci."},{"issue":"3","key":"2025120218122588500_ref26","doi-asserted-by":"publisher","DOI":"10.1016\/j.socscimed.2022.114759","article-title":"Using electronic health records to understand the population of local children captured in a large health system in Durham County, NC, USA, and implications for population health research","volume":"296","author":"Stolte","year":"2022","journal-title":"Soc Sci Med."},{"key":"2025120218122588500_ref27","volume-title":"Clinical Classifications for Health Policy Research, Version 2: Hospital Inpatient Statistics","author":"Elixhauser","year":"1996"},{"key":"2025120218122588500_ref28","author":"Agency for Healthcare Research and Quality","year":"2017"},{"key":"2025120218122588500_ref29"},{"issue":"1","key":"2025120218122588500_ref30","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the Lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J R Stat Soc B Methodol."},{"issue":"3","key":"2025120218122588500_ref31","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1080\/00273171.2011.568786","article-title":"An introduction to propensity score methods for reducing the effects of confounding in observational studies","volume":"46","author":"Austin","year":"2011","journal-title":"Multivariate Behav Res."},{"issue":"28","key":"2025120218122588500_ref32","doi-asserted-by":"publisher","first-page":"3661","DOI":"10.1002\/sim.6607","article-title":"Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies","volume":"34","author":"Austin","year":"2015","journal-title":"Stat Med."},{"issue":"1","key":"2025120218122588500_ref33","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1080\/19345747.2015.1060282","article-title":"Assessing methods for generalizing experimental impact estimates to target populations","volume":"9","author":"Kern","year":"2016","journal-title":"J Res Educ Effect."},{"issue":"1","key":"2025120218122588500_ref34","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1093\/biomet\/70.1.41","article-title":"The central role of the propensity score in observational studies for causal effects","volume":"70","author":"Rosenbaum","year":"1983","journal-title":"Biometrika."},{"issue":"8","key":"2025120218122588500_ref35","doi-asserted-by":"publisher","first-page":"1010","DOI":"10.1093\/aje\/kwx164","article-title":"Transportability of trial results using inverse odds of sampling weights","volume":"186","author":"Westreich","year":"2017","journal-title":"Am J Epidemiol."},{"issue":"5","key":"2025120218122588500_ref36","doi-asserted-by":"publisher","first-page":"830","DOI":"10.1016\/j.jbi.2013.06.010","article-title":"Defining and measuring completeness of electronic health records for secondary use","volume":"46","author":"Weiskopf","year":"2013","journal-title":"J Biomed Inform."}],"container-title":["American Journal of Epidemiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/aje\/advance-article-pdf\/doi\/10.1093\/aje\/kwaf013\/61722613\/kwaf013.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/aje\/article-pdf\/194\/12\/3224\/61722613\/kwaf013.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/aje\/article-pdf\/194\/12\/3224\/61722613\/kwaf013.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T23:12:35Z","timestamp":1764717155000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/aje\/article\/194\/12\/3224\/7994438"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,31]]},"references-count":36,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2025,1,31]]},"published-print":{"date-parts":[[2025,12,2]]}},"URL":"https:\/\/doi.org\/10.1093\/aje\/kwaf013","relation":{},"ISSN":["0002-9262","1476-6256"],"issn-type":[{"value":"0002-9262","type":"print"},{"value":"1476-6256","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,12]]},"published":{"date-parts":[[2025,1,31]]}}}