{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T07:58:51Z","timestamp":1781596731125,"version":"3.54.5"},"reference-count":55,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2016,12,11]],"date-time":"2016-12-11T00:00:00Z","timestamp":1481414400000},"content-version":"vor","delay-in-days":120,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"National Science Foundation","award":["IIS-1418511 and CCF-1533768"],"award-info":[{"award-number":["IIS-1418511 and CCF-1533768"]}]},{"DOI":"10.13039\/100000050","name":"National Heart, Lung, and Blood Institute","doi-asserted-by":"crossref","award":["1R01H116832-01"],"award-info":[{"award-number":["1R01H116832-01"]}],"id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,3,1]]},"abstract":"<jats:p>Objective: We explored whether use of deep learning to model temporal relations among events in electronic health records (EHRs) would improve model performance in predicting initial diagnosis of heart failure (HF) compared to conventional methods that ignore temporality.<\/jats:p><jats:p>Materials and Methods: Data were from a health system\u2019s EHR on 3884 incident HF cases and 28\u2009903 controls, identified as primary care patients, between May 16, 2000, and May 23, 2013. Recurrent neural network (RNN) models using gated recurrent units (GRUs) were adapted to detect relations among time-stamped events (eg, disease diagnosis, medication orders, procedure orders, etc.) with a 12- to 18-month observation window of cases and controls. Model performance metrics were compared to regularized logistic regression, neural network, support vector machine, and K-nearest neighbor classifier approaches.<\/jats:p><jats:p>Results: Using a 12-month observation window, the area under the curve (AUC) for the RNN model was 0.777, compared to AUCs for logistic regression (0.747), multilayer perceptron (MLP) with 1 hidden layer (0.765), support vector machine (SVM) (0.743), and K-nearest neighbor (KNN) (0.730). When using an 18-month observation window, the AUC for the RNN model increased to 0.883 and was significantly higher than the 0.834 AUC for the best of the baseline methods (MLP).<\/jats:p><jats:p>Conclusion: Deep learning models adapted to leverage temporal relations appear to improve performance of models for detection of incident heart failure with a short observation window of 12\u201318 months.<\/jats:p>","DOI":"10.1093\/jamia\/ocw112","type":"journal-article","created":{"date-parts":[[2016,8,14]],"date-time":"2016-08-14T00:48:24Z","timestamp":1471135704000},"page":"361-370","source":"Crossref","is-referenced-by-count":733,"title":["Using recurrent neural network models for early detection of heart failure onset"],"prefix":"10.1093","volume":"24","author":[{"given":"Edward","family":"Choi","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology, Atlanta"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andy","family":"Schuetz","sequence":"additional","affiliation":[{"name":"Sutter Health, Walnut Creek, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Walter F","family":"Stewart","sequence":"additional","affiliation":[{"name":"Sutter Health, Walnut Creek, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jimeng","family":"Sun","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2016,8,13]]},"reference":[{"issue":"3","key":"2020110612385445200_ocw112-B1","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1001\/jama.292.3.344","article-title":"Trends in heart failure incidence and survival in a community-based population","volume":"292","author":"Roger","year":"2004","journal-title":"JAMA"},{"issue":"4","key":"2020110612385445200_ocw112-B2","first-page":"1","article-title":"Deaths: final data for 2010","volume":"61","author":"Murphy","year":"2010","journal-title":"Natl Vital Stat Rep"},{"key":"2020110612385445200_ocw112-B3","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1056\/NEJM199209033271003","article-title":"Effect of enalapril on mortality and the development of heart failure in asymptomatic patients with reduced left ventricular ejection fractions","volume":"327","author":"Investigators","year":"1992","journal-title":"N Engl J Med"},{"issue":"9","key":"2020110612385445200_ocw112-B4","doi-asserted-by":"crossref","first-page":"1284","DOI":"10.1161\/01.CIR.0000054165.93055.42","article-title":"Prevention of heart failure in patients in the Heart Outcomes Prevention Evaluation (HOPE) study","volume":"107","author":"Arnold","year":"2003","journal-title":"Circulation"},{"issue":"5","key":"2020110612385445200_ocw112-B5","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1001\/archinternmed.2010.427","article-title":"Antihypertensive treatment and development of heart failure in hypertension: a Bayesian network meta-analysis of studies in patients with hypertension and high cardiovascular risk","volume":"171","author":"Sciarretta","year":"2011","journal-title":"Arch Int Med"},{"issue":"10","key":"2020110612385445200_ocw112-B6","doi-asserted-by":"crossref","first-page":"1350","DOI":"10.1161\/01.CIR.0000054675.30348.9A","article-title":"Glitazones and heart failure critical appraisal for the clinician","volume":"107","author":"Wang","year":"2003","journal-title":"Circulation"},{"key":"2020110612385445200_ocw112-B7","article-title":"Early detection of heart failure with varying prediction windows by structured and unstructured data in electronic health records","volume-title":"IEEE Engineering in Medicine and Biology Society","author":"Wang","year":"2015"},{"key":"2020110612385445200_ocw112-B8","article-title":"Combining knowledge and data driven insights for identifying risk factors using electronic health records","volume-title":"American Medical Informatics Association","author":"Sun","year":"2012"},{"issue":"6","key":"2020110612385445200_ocw112-B9","doi-asserted-by":"crossref","first-page":"S106","DOI":"10.1097\/MLR.0b013e3181de9e17","article-title":"Prediction modeling using EHR data: challenges, strategies, and a comparison of machine learning approaches","volume":"48","author":"Wu","year":"2010","journal-title":"Med Care"},{"key":"2020110612385445200_ocw112-B10","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2015.7298932","article-title":"Deep visual-semantic alignments for generating image descriptions","volume-title":"Computer Vision and Pattern Recognition (CVPR)","author":"Karpathy","year":"2015"},{"key":"2020110612385445200_ocw112-B11","article-title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Empirical Methods in Natural Language Processing","volume-title":"(EMNLP)","author":"Cho"},{"issue":"7","key":"2020110612385445200_ocw112-B12","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.1162\/neco.2006.18.7.1527","article-title":"A fast learning algorithm for deep belief nets","volume":"18","author":"Hinton","year":"2006","journal-title":"Neural Comput"},{"issue":"1","key":"2020110612385445200_ocw112-B13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000006","article-title":"Learning deep architectures for AI","volume":"2","author":"Bengio","year":"2009","journal-title":"Foundations Trends Machine Learning."},{"key":"2020110612385445200_ocw112-B14","article-title":"Imagenet classification with deep convolutional neural networks","volume-title":"Advances in Neural Information Processing Systems (NIPS)","author":"Krizhevsky","year":"2012"},{"key":"2020110612385445200_ocw112-B15","doi-asserted-by":"crossref","DOI":"10.1145\/1390156.1390294","article-title":"Extracting and composing robust features with denoising autoencoders","volume-title":"International Conference on Machine learning (ICML)","author":"Vincent","year":"2008"},{"key":"2020110612385445200_ocw112-B16","doi-asserted-by":"crossref","DOI":"10.1109\/ICASSP.2013.6639343","article-title":"Building high-level features using large scale unsupervised learning","volume-title":"International Conference on Machine Learning (ICML)","author":"Le"},{"key":"2020110612385445200_ocw112-B17","article-title":"Unsupervised feature learning for audio classification using convolutional deep belief networks","volume-title":"Advances in Neural Information Processing Systems (NIPS)","author":"Lee","year":"2009"},{"issue":"6","key":"2020110612385445200_ocw112-B18","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","article-title":"Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups","volume":"29","author":"Hinton","year":"2012","journal-title":"Signal Process Mag"},{"key":"2020110612385445200_ocw112-B19","article-title":"Efficient estimation of word representations in vector space","volume-title":"arXiv preprint arXiv:1301.3781","author":"Mikolov","year":"2013"},{"key":"2020110612385445200_ocw112-B20","article-title":"Distributed representations of words and phrases and their compositionality","volume-title":"Advances in Neural Information Processing Systems (NIPS)","author":"Mikolov","year":"2013"},{"key":"2020110612385445200_ocw112-B21","article-title":"Semi-supervised recursive autoencoders for predicting sentiment distributions","volume-title":"Empirical Methods in Natural Language Processing","author":"Socher","year":"2011"},{"issue":"8","key":"2020110612385445200_ocw112-B22","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"2020110612385445200_ocw112-B23","doi-asserted-by":"crossref","DOI":"10.1109\/ICDAR.2009.184","article-title":"ICDAR 2009 handwriting recognition competition","volume-title":"International Conference on Document Analysis and Recognition","author":"Grosicki","year":"2009"},{"key":"2020110612385445200_ocw112-B24","doi-asserted-by":"crossref","DOI":"10.21437\/Interspeech.2014-80","article-title":"Long short-term memory recurrent neural network architectures for large scale acoustic modeling","volume-title":"International Speech Communication Association","author":"Sak","year":"2014"},{"key":"2020110612385445200_ocw112-B25","article-title":"Recurrent neural network regularization","volume-title":"arXiv preprint arXiv:1409.2329","author":"Zaremba","year":"2014"},{"key":"2020110612385445200_ocw112-B26","doi-asserted-by":"crossref","DOI":"10.3115\/v1\/P15-1002","article-title":"Addressing the rare word problem in neural machine translation","volume-title":"Association for Computational Linguistics (ACL)","author":"Luong","year":"2015"},{"key":"2020110612385445200_ocw112-B27","article-title":"An empirical exploration of recurrent network architectures","volume-title":"International Conference on Machine Learning","author":"Jozefowicz","year":"2015"},{"issue":"6","key":"2020110612385445200_ocw112-B28","doi-asserted-by":"crossref","first-page":"e66341","DOI":"10.1371\/journal.pone.0066341","article-title":"Computational phenotype discovery using unsupervised feature learning over noisy, sparse, and irregular clinical data","volume":"8","author":"Lasko","year":"2013","journal-title":"PloS One"},{"key":"2020110612385445200_ocw112-B29","article-title":"Deep computational phenotyping","volume-title":"Knowledge Discovery and Data Mining","author":"Che","year":"2015"},{"key":"2020110612385445200_ocw112-B30","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v29i1.9484","article-title":"PD disease state assessment in naturalistic environments using deep learning","volume-title":"AAAI","author":"Hammerla","year":"2015"},{"key":"2020110612385445200_ocw112-B31","article-title":"Learning to diagnose with LSTM recurrent neural networks","volume-title":"arXiv preprint arXiv: 1511.03677","author":"Lipton","year":"2016"},{"key":"2020110612385445200_ocw112-B32","first-page":"584","article-title":"Exploring the application of deep learning techniques on medical text corpora","volume":"205","author":"Minarro-Gimenez","year":"2013","journal-title":"Stud Health Technol Inform"},{"key":"2020110612385445200_ocw112-B33","doi-asserted-by":"crossref","DOI":"10.1145\/2661829.2661974","article-title":"Medical semantic similarity with a neural language model","volume-title":"International Conference on Information and Knowledge Management","author":"De Vine","year":"2014"},{"key":"2020110612385445200_ocw112-B34","article-title":"Learning low-dimensional representations of medical concepts","volume-title":"American Medical Informatics Association on Clinical Research Informatics","author":"Choi","year":"2016"},{"key":"2020110612385445200_ocw112-B35","article-title":"Medical concept representation learning from electronic health records and its application on heart failure prediction","volume-title":"arXiv preprint arXiv:1602.03686","author":"Choi","year":"2016"},{"issue":"15","key":"2020110612385445200_ocw112-B36","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1001\/jama.2011.451","article-title":"A predictive model for progression of chronic kidney disease to kidney failure","volume":"305","author":"Tangri","year":"2011","journal-title":"JAMA"},{"key":"2020110612385445200_ocw112-B37","doi-asserted-by":"crossref","DOI":"10.1109\/EMBC.2012.6346556","article-title":"Disease progression modeling using hidden Markov models","volume-title":"Engineering in Medicine and Biology Society","author":"Sukkar","year":"2012"},{"key":"2020110612385445200_ocw112-B38","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1016\/j.neuroimage.2013.03.073","article-title":"Modeling disease progression via multi-task learning","volume":"78","author":"Zhou","year":"2013","journal-title":"NeuroImage"},{"key":"2020110612385445200_ocw112-B39","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-40763-5_55","article-title":"Longitudinal modeling of glaucoma progression using 2-dimensional continuous-time hidden Markov model","volume-title":"Medical Image Computing and Computer-Assisted Intervention (MICCAI)","author":"Liu","year":"2013"},{"key":"2020110612385445200_ocw112-B40","article-title":"A framework for individualizing predictions of disease trajectories by exploiting multi-resolution structure","volume-title":"Advances in Neural Information Processing Systems (NIPS)","author":"Schulam","year":"2015"},{"key":"2020110612385445200_ocw112-B41","doi-asserted-by":"crossref","DOI":"10.1145\/2623330.2623754","article-title":"Unsupervised learning of disease progression models","volume-title":"Knowledge Discovery and Data Mining (KDD)","author":"Wang","year":"2014"},{"key":"2020110612385445200_ocw112-B42","doi-asserted-by":"crossref","DOI":"10.1109\/ICDM.2015.144","article-title":"Constructing disease network and temporal progression model via context-sensitive Hawkes process","volume-title":"International Conference on Data Mining (ICDM)","author":"Choi","year":"2015"},{"key":"2020110612385445200_ocw112-B43","article-title":"A primer on neural network models for natural language processing","volume-title":"arXiv preprint arXiv:1510.00726","author":"Goldberg","year":"2015"},{"key":"2020110612385445200_ocw112-B44","volume-title":"Pattern Recognition and Machine Learning (Information Science and Statistics)","author":"Bishop","year":"2006"},{"key":"2020110612385445200_ocw112-B45","article-title":"Theano: a CPU and GPU Math Expression Compiler","volume-title":"Python for Scientific Computing Conference","author":"Bergstra","year":"2010"},{"issue":"3","key":"2020110612385445200_ocw112-B46","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1093\/oxfordjournals.aje.a113439","article-title":"On the need for the rare disease assumption in case-control studies","volume":"116","author":"Greenland","year":"1982","journal-title":"Am J Epidemiol."},{"issue":"7","key":"2020110612385445200_ocw112-B47","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1016\/j.cardfail.2014.03.008","article-title":"Prevalence of heart failure signs and symptoms in a large primary care population identified through the use of text and data mining of the electronic health record","volume":"20","author":"Vijayakrishnan","year":"2014","journal-title":"J Cardiac Failure"},{"issue":"5","key":"2020110612385445200_ocw112-B48","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1016\/j.amjmed.2012.10.022","article-title":"Contemporary prevalence and correlates of incident heart failure with preserved ejection fraction","volume":"126","author":"Gurwitz","year":"2013","journal-title":"Am J Med"},{"key":"2020110612385445200_ocw112-B49","author":"Clinical Classifications Software (CCS) for ICD-9-CM"},{"key":"2020110612385445200_ocw112-B50","author":"Medi-Span Electronic Drug File (MED-File) v2","year":"2016"},{"key":"2020110612385445200_ocw112-B51","author":"Clinical Classifications Software for Services and Procedures","year":"2016"},{"key":"2020110612385445200_ocw112-B52","article-title":"ADADELTA: An adaptive learning rate method","volume-title":"arXiv preprint arXiv:1212.5701","author":"Zeiler","year":"2012"},{"key":"2020110612385445200_ocw112-B53","first-page":"1","article-title":"Hypertension among adults in the United States: National Health and Nutrition Examination Survey, 2011-2012","volume":"113","author":"Nwankwo","year":"2013","journal-title":"NCHS Data Brief"},{"key":"2020110612385445200_ocw112-B54","volume-title":"Coronary Artery Disease (CAD)","author":"CDC"},{"key":"2020110612385445200_ocw112-B55","article-title":"Visualizing and understanding recurrent networks","volume-title":"arXiv preprint arXiv:1506.02078","author":"Karpathy","year":"2015"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/2\/361\/34148331\/ocw112.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/2\/361\/34148331\/ocw112.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,5]],"date-time":"2022-07-05T23:29:47Z","timestamp":1657063787000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/24\/2\/361\/2631499"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,8,13]]},"references-count":55,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2016,8,13]]},"published-print":{"date-parts":[[2017,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocw112","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2017,3]]},"published":{"date-parts":[[2016,8,13]]}}}