{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T03:55:19Z","timestamp":1788839719735,"version":"build-2803163510"},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"11","license":[{"start":{"date-parts":[[2022,8,4]],"date-time":"2022-08-04T00:00:00Z","timestamp":1659571200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["U2COD023196"],"award-info":[{"award-number":["U2COD023196"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["UL1TR002243"],"award-info":[{"award-number":["UL1TR002243"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,10,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Synthetic data are increasingly relied upon to share electronic health record (EHR) data while maintaining patient privacy. Current simulation methods can generate longitudinal data, but the results are unreliable for several reasons. First, the synthetic data drifts from the real data distribution over time. Second, the typical approach to quality assessment, which is based on the extent to which real records can be distinguished from synthetic records using a critic model, often fails to recognize poor simulation results. In this article, we introduce a longitudinal simulation framework, called LS-EHR, which addresses these issues.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and methods<\/jats:title>\n                  <jats:p>LS-EHR enhances simulation through conditional fuzzing and regularization, rejection sampling, and prior knowledge embedding. We compare LS-EHR to the state-of-the-art using data from 60\u200a000 EHRs from Vanderbilt University Medical Center (VUMC) and the All of Us Research Program. We assess discrimination between real and synthetic data over time. We evaluate the generation process and critic model using the area under the receiver operating characteristic curve (AUROC). For the critic, a higher value indicates a more robust model for quality assessment. For the generation process, a lower value indicates better synthetic data quality.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The LS-EHR critic improves discrimination AUROC from 0.655 to 0.909 and 0.692 to 0.918 for VUMC and All of Us data, respectively. By using the new critic, the LS-EHR generation model reduces the AUROC from 0.909 to 0.758 and 0.918 to 0.806.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>LS-EHR can substantially improve the usability of simulated longitudinal EHR data.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocac131","type":"journal-article","created":{"date-parts":[[2022,8,5]],"date-time":"2022-08-05T05:49:41Z","timestamp":1659678581000},"page":"1890-1898","source":"Crossref","is-referenced-by-count":15,"title":["Keeping synthetic patients on track: feedback mechanisms to mitigate performance drift in longitudinal health data simulation"],"prefix":"10.1093","volume":"29","author":[{"given":"Ziqi","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Computer Science, Vanderbilt University , Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6719-1388","authenticated-orcid":false,"given":"Chao","family":"Yan","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bradley A","family":"Malin","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Vanderbilt University , Nashville, Tennessee, USA"},{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"},{"name":"Department of Biostatistics, Vanderbilt University Medical Center , Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,8,4]]},"reference":[{"key":"2022101107582032000_ocac131-B1","first-page":"277","author":"Machanavajjhala","year":"2008"},{"issue":"11","key":"2022101107582032000_ocac131-B2","doi-asserted-by":"crossref","first-page":"e23139","DOI":"10.2196\/23139","article-title":"Evaluating identity disclosure risk in fully synthetic health data: model development and validation","volume":"22","author":"El Emam","year":"2020","journal-title":"J Med Internet Res"},{"key":"2022101107582032000_ocac131-B3","doi-asserted-by":"crossref","first-page":"103977","DOI":"10.1016\/j.jbi.2021.103977","article-title":"Membership inference attacks against synthetic health data","volume":"125","author":"Zhang","year":"2022","journal-title":"J Biomed Inform"},{"key":"2022101107582032000_ocac131-B4","first-page":"286","author":"Choi","year":"2017"},{"issue":"3","key":"2022101107582032000_ocac131-B5","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1093\/jamia\/ocy142","article-title":"Synthesizing electronic health records using improved generative adversarial networks","volume":"26","author":"Baowaly","year":"2019","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2022101107582032000_ocac131-B6","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1093\/jamia\/ocz161","article-title":"Ensuring electronic medical record simulation through better training, modeling, and evaluation","volume":"27","author":"Zhang","year":"2020","journal-title":"J Am Med Inform Assoc"},{"key":"2022101107582032000_ocac131-B7","first-page":"1335","author":"Yan","year":"2020"},{"issue":"8","key":"2022101107582032000_ocac131-B8","doi-asserted-by":"crossref","first-page":"1350","DOI":"10.1093\/jamia\/ocac045","article-title":"Demonstrating an approach for evaluating synthetic geospatial and temporal epidemiologic data utility: results from analyzing &gt;1.8 million SARS-CoV-2 tests in the United States National COVID Cohort Collaborative (N3C)","volume":"29","author":"Thomas","year":"2022","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2022101107582032000_ocac131-B9","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1093\/jamia\/ocaa196","article-title":"Haendel MA, Chute CG, Bennett TD, et al. The National COVID Cohort Collaborative (N3C): rationale, design, infrastructure, and deployment","volume":"28","year":"2021","journal-title":"J Am Med Inform Assoc"},{"key":"2022101107582032000_ocac131-B10","year":"2022"},{"issue":"9","key":"2022101107582032000_ocac131-B11","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.1093\/jamia\/ocaa119","article-title":"Generating sequential electronic health records using dual adversarial autoencoder","volume":"27","author":"Lee","year":"2020","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2022101107582032000_ocac131-B12","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1093\/jamia\/ocaa262","article-title":"SynTEG: a framework for temporal structured electronic health data simulation","volume":"28","author":"Zhang","year":"2021","journal-title":"J Am Med Inform Assoc"},{"key":"2022101107582032000_ocac131-B13","first-page":"260","author":"Biswal","year":"2021"},{"key":"2022101107582032000_ocac131-B14","author":"Li"},{"key":"2022101107582032000_ocac131-B15","author":"Wang"},{"issue":"7","key":"2022101107582032000_ocac131-B16","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1056\/NEJMsr1809937","article-title":"The \u201cAll of Us\u201d research program","volume":"381","author":"All of Us Research Program Investigators","year":"2019","journal-title":"N Engl J Med"},{"issue":"3","key":"2022101107582032000_ocac131-B17","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1038\/clpt.2008.89","article-title":"Development of a large-scale de-identified DNA biobank to enable personalized medicine","volume":"84","author":"Roden","year":"2008","journal-title":"Clin Pharmacol Ther"},{"key":"2022101107582032000_ocac131-B18","first-page":"2672","author":"Goodfellow","year":"2014"},{"key":"2022101107582032000_ocac131-B19","first-page":"28","author":"Bengio","year":"2015"},{"key":"2022101107582032000_ocac131-B20","author":"Ranzato","year":"2016"},{"key":"2022101107582032000_ocac131-B21","author":"Van den Oord"},{"key":"2022101107582032000_ocac131-B22","first-page":"9929","author":"Wang","year":"2020"},{"key":"2022101107582032000_ocac131-B23","first-page":"1597","author":"Chen","year":"2020"},{"key":"2022101107582032000_ocac131-B24","author":"Ranjan"},{"key":"2022101107582032000_ocac131-B25","first-page":"1041","author":"Wang","year":"2017"},{"key":"2022101107582032000_ocac131-B26","first-page":"342","author":"Casella","year":"2004"},{"key":"2022101107582032000_ocac131-B27","author":"Azadi","year":"2019"},{"issue":"1","key":"2022101107582032000_ocac131-B28","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1093\/jamia\/ocaa249","article-title":"Optimizing the synthesis of clinical trial data using sequential trees","volume":"28","author":"Emam","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"2022101107582032000_ocac131-B29","doi-asserted-by":"crossref","first-page":"e35734","DOI":"10.2196\/35734","article-title":"Utility metrics for evaluating synthetic health data generation methods: validation study","volume":"10","author":"El Emam","year":"2022","journal-title":"JMIR Med Inform"},{"key":"2022101107582032000_ocac131-B30","first-page":"32","author":"Zellers","year":"2019"},{"issue":"12","key":"2022101107582032000_ocac131-B31","doi-asserted-by":"crossref","first-page":"1102","DOI":"10.1038\/nbt.2749","article-title":"Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association study data","volume":"31","author":"Denny","year":"2013","journal-title":"Nat Biotechnol"},{"issue":"9","key":"2022101107582032000_ocac131-B32","doi-asserted-by":"crossref","first-page":"1205","DOI":"10.1093\/bioinformatics\/btq126","article-title":"PheWAS: demonstrating the feasibility of a phenome-wide scan to discover gene\u2013disease associations","volume":"26","author":"Denny","year":"2010","journal-title":"Bioinformatics"},{"key":"2022101107582032000_ocac131-B33"},{"issue":"6","key":"2022101107582032000_ocac131-B34","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1038\/s41551-021-00751-8","article-title":"Synthetic data in machine learning for medicine and healthcare","volume":"5","author":"Chen","year":"2021","journal-title":"Nat Biomed Eng"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/11\/1890\/46379128\/ocac131.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/11\/1890\/46379128\/ocac131.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,11]],"date-time":"2022-10-11T08:00:06Z","timestamp":1665475206000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/29\/11\/1890\/6655786"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,4]]},"references-count":34,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2022,8,4]]},"published-print":{"date-parts":[[2022,10,7]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocac131","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,11,1]]},"published":{"date-parts":[[2022,8,4]]}}}