{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,12]],"date-time":"2026-07-12T04:58:59Z","timestamp":1783832339133,"version":"3.55.0"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"S3","license":[{"start":{"date-parts":[[2020,7,1]],"date-time":"2020-07-01T00:00:00Z","timestamp":1593561600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,7,9]],"date-time":"2020-07-09T00:00:00Z","timestamp":1594252800000},"content-version":"vor","delay-in-days":8,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2020,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n<jats:title>Background<\/jats:title>\n<jats:p>The interpretability of results predicted by the machine learning models is vital, especially in the critical fields like healthcare. With the increasingly adoption of electronic healthcare records (EHR) by the medical organizations in the last decade, which accumulated abundant electronic patient data, neural networks or deep learning techniques are gradually being applied to clinical tasks by utilizing the huge potential of EHR data. However, typical deep learning models are black-boxes, which are not transparent and the prediction outcomes of which are difficult to interpret.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Methods<\/jats:title>\n<jats:p>To remedy this limitation, we propose an attention neural network model for interpretable clinical prediction. In detail, the proposed model employs an attention mechanism to capture critical\/essential features with their attention signals on the prediction results, such that the predictions generated by the neural network model can be interpretable.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Results<\/jats:title>\n<jats:p>We evaluate our proposed model on a real-world clinical dataset consisting of 736 samples to predict readmissions for heart failure patients. The performance of the proposed model achieved 66.7 and 69.1% in terms of accuracy and AUC, respectively, and outperformed the baseline models. Besides, we displayed patient-specific attention weights, which can not only help clinicians understand the prediction outcomes, but also assist them to select individualized treatment strategies or intervention plans.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Conclusions<\/jats:title>\n<jats:p>The experimental results demonstrate that the proposed model can improve both the prediction performance and interpretability by equipping the model with an attention mechanism.<\/jats:p>\n<\/jats:sec>","DOI":"10.1186\/s12911-020-1110-7","type":"journal-article","created":{"date-parts":[[2020,7,9]],"date-time":"2020-07-09T08:08:50Z","timestamp":1594282130000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":64,"title":["Interpretable clinical prediction via attention-based neural network"],"prefix":"10.1186","volume":"20","author":[{"given":"Peipei","family":"Chen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Dong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinliang","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xudong","family":"Lu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Uzay","family":"Kaymak","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhengxing","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,7,9]]},"reference":[{"issue":"5","key":"1110_CR1","doi-asserted-by":"publisher","first-page":"1589","DOI":"10.1109\/JBHI.2017.2767063","volume":"22","author":"B Shickel","year":"2018","unstructured":"Shickel B, Tighe P, Bihorac A, Rashidi P. Deep EHR: A survey of recent advances in deep learning techniques for electronic health record (EHR) analysis. IEEE J Biomed Health. 2018;22(5):1589\u2013604.","journal-title":"IEEE J Biomed Health"},{"issue":"10","key":"1110_CR2","doi-asserted-by":"publisher","first-page":"1419","DOI":"10.1093\/jamia\/ocy068","volume":"25","author":"C Xiao","year":"2018","unstructured":"Xiao C, Choi E, Sun J. Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review. J Am Med Inform Assoc. 2018;25(10):1419\u201328.","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"1110_CR3","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1038\/s41591-018-0316-z","volume":"25","author":"A Esteva","year":"2019","unstructured":"Esteva A, Robicquet A, Ramsundar B, et al. A guide to deep learning in healthcare. Nat Med. 2019;25(1):24.","journal-title":"Nat Med"},{"issue":"5","key":"1110_CR4","doi-asserted-by":"publisher","first-page":"956","DOI":"10.1109\/TBME.2017.2731158","volume":"65","author":"Z Huang","year":"2017","unstructured":"Huang Z, Dong W, Duan H, et al. A regularized deep learning approach for clinical risk prediction of acute coronary syndrome using electronic health records. IEEE Trans Biomed Eng. 2017;65(5):956\u201368.","journal-title":"IEEE Trans Biomed Eng"},{"key":"1110_CR5","doi-asserted-by":"crossref","unstructured":"Yang C, Delcher C, Shenkman E, et al. Predicting 30-day all-cause readmissions from hospital inpatient discharge data. In: 2016 IEEE 18th International conference on e-Health networking, applications and services (Healthcom); Munich: IEEE; 2016. p. 1\u20136.","DOI":"10.1109\/HealthCom.2016.7749452"},{"key":"1110_CR6","first-page":"3020","volume-title":"International Conference on Machine Learning","author":"F Johansson","year":"2016","unstructured":"Johansson F, Shalit U, Sontag D. Learning representations for counterfactual inference. In: International Conference on Machine Learning; 2016. p. 3020\u20139."},{"key":"1110_CR7","doi-asserted-by":"crossref","unstructured":"Zhang Y, Chen R, Tang J, et al. LEAP: learning to prescribe effective and safe treatment combinations for multimorbidity. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; New York: ACM; 2017. p. 1315\u201324.","DOI":"10.1145\/3097983.3098109"},{"key":"1110_CR8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.dsp.2017.10.011","volume":"73","author":"G Montavon","year":"2018","unstructured":"Montavon G, Samek W, M\u00fcller KR. Methods for interpreting and understanding deep neural networks. Digit Signal Process. 2018;73:1\u201315.","journal-title":"Digit Signal Process"},{"key":"1110_CR9","doi-asserted-by":"crossref","unstructured":"Wang F, Kaushal R, Khullar D. Should health care demand interpretable artificial intelligence or accept \u201cblack box\u201d medicine? Ann Intern Med.\u00a02020;172:59\u201360.\u00a0[Epub ahead of print 17 December 2019].","DOI":"10.7326\/M19-2548"},{"issue":"10","key":"1110_CR10","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1145\/3233231","volume":"61","author":"ZC Lipton","year":"2018","unstructured":"Lipton ZC. The mythos of model interpretability. Commun ACM. 2018;61(10):36\u201343.","journal-title":"Commun ACM"},{"key":"1110_CR11","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C. Why should I trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; New York: ACM; 2016. p. 1135\u201344.","DOI":"10.1145\/2939672.2939778"},{"key":"1110_CR12","series-title":"PhD thesis, Massachusetts Institute of Technology","volume-title":"Interactive and interpretable machine learning models for human machine collaboration","author":"B Kim","year":"2015","unstructured":"Kim, Been. Interactive and interpretable machine learning models for human machine collaboration. PhD thesis, Massachusetts Institute of Technology, 2015."},{"issue":"1","key":"1110_CR13","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1016\/j.dss.2010.12.003","volume":"51","author":"J Huysmans","year":"2011","unstructured":"Huysmans J, Dejaeger K, Mues C, et al. An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models. Decis Support Syst. 2011;51(1):141\u201354.","journal-title":"Decis Support Syst"},{"key":"1110_CR14","doi-asserted-by":"crossref","unstructured":"Ahmad MA, Eckert C, Teredesai A. Interpretable machine learning in healthcare. In: Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics; New York: ACM; 2018. p. 559\u201360.","DOI":"10.1145\/3233547.3233667"},{"key":"1110_CR15","first-page":"189","volume":"34","author":"ME Kaminski","year":"2019","unstructured":"Kaminski ME. The right to explanation, explained. Berkeley Tech LJ. 2019;34:189.","journal-title":"Berkeley Tech LJ"},{"key":"1110_CR16","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"J Zhao","year":"2017","unstructured":"Zhao J, Wang T, Yatskar M, et al. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing; 2017."},{"key":"1110_CR17","doi-asserted-by":"crossref","unstructured":"Caruana R, Lou Y, Gehrke J, et al. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; New York: ACM; 2015. p. 1721\u201330.","DOI":"10.1145\/2783258.2788613"},{"key":"1110_CR18","first-page":"24","volume-title":"International Conference on Theory and Applications of Fuzzy Systems and Soft Computing","author":"U Kaymak","year":"2018","unstructured":"Kaymak U. On using fuzzy sets in healthcare process analysis. In: International Conference on Theory and Applications of Fuzzy Systems and Soft Computing. Cham: Springer; 2018. p. 24."},{"key":"1110_CR19","doi-asserted-by":"crossref","unstructured":"McAuley J, Leskovec J. Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM conference on Recommender systems; New York: ACM; 2013. p. 165\u201372.","DOI":"10.1145\/2507157.2507163"},{"issue":"Nov","key":"1110_CR20","first-page":"2579","volume":"9","author":"L Maaten","year":"2008","unstructured":"Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008;9(Nov):2579\u2013605.","journal-title":"J Mach Learn Res"},{"key":"1110_CR21","unstructured":"Caruana R, Kangarloo H, Dionisio JD, et al. Case-based explanation of non-case-based learning methods. In: Proceedings of the AMIA Symposium;\u00a0Washington, DC: American Medical Informatics Association; 1999. p. 212."},{"key":"1110_CR22","doi-asserted-by":"publisher","first-page":"103303","DOI":"10.1016\/j.jbi.2019.103303","volume":"100","author":"P Chen","year":"2019","unstructured":"Chen P, Dong W, Lu X, et al. Deep representation learning for individualized treatment effect estimation using electronic health records. J Biomed Inform. 2019;100:103303.","journal-title":"J Biomed Inform"},{"key":"1110_CR23","volume-title":"Neural machine translation by jointly learning to align and translate","author":"D Bahdanau","year":"2014","unstructured":"Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014."},{"key":"1110_CR24","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems","author":"A Vaswani","year":"2017","unstructured":"Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. In: Advances in Neural Information Processing Systems; 2017. p. 5998\u20136008."},{"key":"1110_CR25","first-page":"577","volume-title":"Advances in Neural Information Processing Systems","author":"JK Chorowski","year":"2015","unstructured":"Chorowski JK, Bahdanau D, Serdyuk D, et al. Attention-based models for speech recognition. In: Advances in Neural Information Processing Systems; 2015. p. 577\u201385."},{"key":"1110_CR26","doi-asserted-by":"crossref","unstructured":"Choi E, Bahadori MT, Song L, et al. GRAM: graph-based attention model for healthcare representation learning. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining;\u00a0New York: ACM; 2017. p. 787\u201395.","DOI":"10.1145\/3097983.3098126"},{"key":"1110_CR27","first-page":"3504","volume-title":"Advances in Neural Information Processing Systems","author":"E Choi","year":"2016","unstructured":"Choi E, Bahadori MT, Sun J, et al. Retain: an interpretable predictive model for healthcare using reverse time attention mechanism. In: Advances in Neural Information Processing Systems; 2016. p. 3504\u201312."},{"key":"1110_CR28","doi-asserted-by":"crossref","unstructured":"Zhang X, Qian B, Li X, et al. An interpretable fast model for predicting the risk of heart failure. In: Proceedings of the 2019 SIAM International Conference on Data Mining; Calgary, Alberta, Canada: Society for Industrial and Applied Mathematics; 2019. p. 576\u201384.","DOI":"10.1137\/1.9781611975673.65"},{"key":"1110_CR29","volume-title":"An introduction to medical statistics","author":"M Bland","year":"2015","unstructured":"Bland M. An introduction to medical statistics. UK: Oxford University Press; 2015."},{"issue":"16","key":"1110_CR30","doi-asserted-by":"publisher","first-page":"e147","DOI":"10.1016\/j.jacc.2013.05.019","volume":"62","author":"CW Yancy","year":"2013","unstructured":"Yancy CW, Jessup M, Bozkurt B, et al. 2013 ACCF\/AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation\/American Heart Association task force on practice guidelines. J Am Coll Cardiol. 2013;62(16):e147\u2013239.","journal-title":"J Am Coll Cardiol"},{"issue":"6","key":"1110_CR31","doi-asserted-by":"publisher","first-page":"568","DOI":"10.1016\/j.hlc.2015.11.010","volume":"25","author":"MK Mojadidi","year":"2016","unstructured":"Mojadidi MK, Galeas JN, Goodman-Meza D, et al. Thrombocytopaenia as a prognostic indicator in heart failure with reduced ejection fraction. Heart Lung Circ. 2016;25(6):568\u201375.","journal-title":"Heart Lung Circ"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-020-1110-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-020-1110-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-020-1110-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T23:18:29Z","timestamp":1625786309000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-020-1110-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7]]},"references-count":31,"journal-issue":{"issue":"S3","published-print":{"date-parts":[[2020,7]]}},"alternative-id":["1110"],"URL":"https:\/\/doi.org\/10.1186\/s12911-020-1110-7","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7]]},"assertion":[{"value":"9 July 2020","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"131"}}