{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T03:36:10Z","timestamp":1775014570497,"version":"3.50.1"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,4,20]],"date-time":"2020-04-20T00:00:00Z","timestamp":1587340800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,4,20]],"date-time":"2020-04-20T00:00:00Z","timestamp":1587340800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>During the perioperative period patients often suffer complications, including acute kidney injury (AKI), reintubation, and mortality. In order to effectively prevent these complications, high-risk patients must be readily identified. However, most current risk scores are designed to predict a single postoperative complication and often lack specificity on the patient level. In other fields, machine learning (ML) has been shown to successfully create models to predict multiple end points using a single input feature set. We hypothesized that ML can be used to create models to predict postoperative mortality, AKI, reintubation, and a combined outcome using a single set of features available at the end of surgery. A set of 46 features available at the end of surgery, including drug dosing, blood loss, vital signs, and others were extracted. Additionally, six additional features accounting for total intraoperative hypotension were extracted and trialed for different models. A total of 59,981 surgical procedures met inclusion criteria and the deep neural networks (DNN) were trained on 80% of the data, with 20% reserved for testing. The network performances were then compared to ASA Physical Status. In addition to creating separate models for each outcome, a multitask learning model was trialed that used information on all outcomes to predict the likelihood of each outcome individually. The overall rate of the examined complications in this data set was 0.79% for mortality, 22.3% (of 21,676 patients with creatinine values) for AKI, and 1.1% for reintubation. Overall, there was significant overlap between the various model types for each outcome, with no one modeling technique consistently performing the best. However, the best DNN models did beat the ASA score for all outcomes other than mortality. The highest area under the receiver operating characteristic curve (AUC) models were 0.792 (0.775\u20130.808) for AKI, 0.879 (0.851\u20130.905) for reintubation, 0.907 (0.872\u20130.938) for mortality, and 0.874 (0.864\u20130.866) for any outcome. The ASA score alone achieved AUCs of 0.652 (0.636\u20130.669) for AKI, 0.787 (0.757\u20130.818) for reintubation, 0.839 (0.804\u20130.875) for mortality, and 0.76 (0.748\u20130.773) for any outcome. Overall, the DNN architecture was able to create models that outperformed the ASA physical status to predict all outcomes based on a single feature set, consisting of objective data available at the end of surgery. No one model architecture consistently performed the best.<\/jats:p>","DOI":"10.1038\/s41746-020-0248-0","type":"journal-article","created":{"date-parts":[[2020,4,20]],"date-time":"2020-04-20T10:03:19Z","timestamp":1587376999000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":67,"title":["Development and validation of a deep neural network model to predict postoperative mortality, acute kidney injury, and reintubation using a single feature set"],"prefix":"10.1038","volume":"3","author":[{"given":"Ira S.","family":"Hofer","sequence":"first","affiliation":[]},{"given":"Christine","family":"Lee","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8148-4590","authenticated-orcid":false,"given":"Eilon","family":"Gabel","sequence":"additional","affiliation":[]},{"given":"Pierre","family":"Baldi","sequence":"additional","affiliation":[]},{"given":"Maxime","family":"Cannesson","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,4,20]]},"reference":[{"key":"248_CR1","doi-asserted-by":"publisher","first-page":"49","DOI":"10.2147\/IBPC.S45292","volume":"7","author":"L Lonjaret","year":"2014","unstructured":"Lonjaret, L., Lairez, O., Minville, V. & Geeraerts, T. Optimal perioperative management of arterial blood pressure. Integr. Blood Press. Control 7, 49\u201359 (2014).","journal-title":"Integr. Blood Press. Control"},{"key":"248_CR2","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1213\/01.ANE.0000147519.82841.5E","volume":"100","author":"TG Monk","year":"2005","unstructured":"Monk, T. G., Saini, V., Weldon, B. C. & Sigl, J. C. Anesthetic management and one-year mortality after noncardiac surgery. Anesth. Analg. 100, 4\u201310 (2005).","journal-title":"Anesth. Analg."},{"key":"248_CR3","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1001\/archsurg.137.5.611","volume":"137","author":"MA Healey","year":"2002","unstructured":"Healey, M. A., Shackford, S. R., Osler, T. M., Rogers, F. B. & Burns, E. Complications in surgical patients. Arch. Surg. 137, 611\u2013617 (2002).","journal-title":"Arch. Surg."},{"key":"248_CR4","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1016\/j.jss.2013.01.032","volume":"181","author":"SE Tevis","year":"2013","unstructured":"Tevis, S. E. & Kennedy, G. D. Postoperative complications and implications on patient-centered outcomes. J. Surg. Res. 181, 106\u2013113 (2013).","journal-title":"J. Surg. Res."},{"key":"248_CR5","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1097\/ALN.0000000000000922","volume":"124","author":"J van Waes","year":"2016","unstructured":"van Waes, J. et al. Association between intraoperative hypotension and myocardial injury after vascular surgery. Anesthesiology 124, 35\u201344 (2016).","journal-title":"Anesthesiology"},{"key":"248_CR6","doi-asserted-by":"publisher","first-page":"1642","DOI":"10.1001\/jama.2017.4360","volume":"317","author":"Writing Committee for the, V.S.I.","year":"2017","unstructured":"Writing Committee for the, V.S.I. et al. Association of Postoperative high-sensitivity troponin levels with myocardial injury and 30-day mortality among patients undergoing noncardiac surgery. JAMA 317, 1642\u20131651 (2017).","journal-title":"JAMA"},{"key":"248_CR7","doi-asserted-by":"publisher","first-page":"564","DOI":"10.1097\/ALN.0000000000000113","volume":"120","author":"F Botto","year":"2014","unstructured":"Botto, F. et al. Myocardial injury after noncardiac surgery: a large, international, prospective cohort study establishing diagnostic criteria, characteristics, predictors, and 30-day outcomes. Anesthesiology 120, 564\u2013578 (2014).","journal-title":"Anesthesiology"},{"key":"248_CR8","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1097\/ALN.0000000000001432","volume":"126","author":"V Salmasi","year":"2017","unstructured":"Salmasi, V. et al. Relationship between intraoperative hypotension, defined by either reduction from baseline or absolute thresholds, and acute kidney and myocardial injury after noncardiac surgery: a retrospective cohort analysis. Anesthesiology 126, 47\u201365 (2017).","journal-title":"Anesthesiology"},{"key":"248_CR9","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1097\/ALN.0b013e3182a10e26","volume":"119","author":"M Walsh","year":"2013","unstructured":"Walsh, M. et al. Relationship between intraoperative mean arterial pressure and clinical outcomes after noncardiac surgery: toward an empirical definition of hypotension. Anesthesiology 119, 507\u2013515 (2013).","journal-title":"Anesthesiology"},{"key":"248_CR10","doi-asserted-by":"publisher","first-page":"1480","DOI":"10.1213\/ANE.0000000000001557","volume":"123","author":"AD Oprea","year":"2016","unstructured":"Oprea, A. D. et al. Baseline pulse pressure, acute kidney injury, and mortality after noncardiac surgery. Anesth. Analg. 123, 1480\u20131489 (2016).","journal-title":"Anesth. Analg."},{"key":"248_CR11","doi-asserted-by":"publisher","first-page":"924","DOI":"10.1213\/ANE.0b013e3182a1ec84","volume":"117","author":"M Walsh","year":"2013","unstructured":"Walsh, M. et al. The association between perioperative hemoglobin and acute kidney injury in patients having noncardiac surgery. Anesth. Analg. 117, 924\u2013931 (2013).","journal-title":"Anesth. Analg."},{"key":"248_CR12","doi-asserted-by":"publisher","first-page":"418","DOI":"10.1093\/bja\/aeu458","volume":"115","author":"J Hirsch","year":"2015","unstructured":"Hirsch, J., DePalma, G., Tsai, T. T., Sands, L. P. & Leung, J. M. Impact of intraoperative hypotension and blood pressure fluctuations on early postoperative delirium after non-cardiac surgery. Br. J. Anaesth. 115, 418\u2013426 (2015).","journal-title":"Br. J. Anaesth."},{"key":"248_CR13","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1016\/S0002-9343(98)00292-7","volume":"105","author":"ER Marcantonio","year":"1998","unstructured":"Marcantonio, E. R., Goldman, L., Orav, E. J., Cook, E. F. & Lee, T. H. The association of intraoperative factors with the development of postoperative delirium. Am. J. Med 105, 380\u2013384 (1998).","journal-title":"Am. J. Med"},{"key":"248_CR14","doi-asserted-by":"publisher","first-page":"1135","DOI":"10.1213\/ANE.0000000000001797","volume":"124","author":"WH Stapelfeldt","year":"2017","unstructured":"Stapelfeldt, W. H. et al. The SLUScore: a novel method for detecting hazardous hypotension in adult patients undergoing noncardiac surgical procedures. Anesth. Analg. 124, 1135\u20131152 (2017).","journal-title":"Anesth. Analg."},{"key":"248_CR15","doi-asserted-by":"publisher","first-page":"1217","DOI":"10.1097\/ALN.0b013e3181c14930","volume":"111","author":"JB Bijker","year":"2009","unstructured":"Bijker, J. B. et al. Intraoperative hypotension and 1-year mortality after noncardiac surgery. Anesthesiology 111, 1217\u20131226 (2009).","journal-title":"Anesthesiology"},{"key":"248_CR16","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1097\/ALN.0000000000000756","volume":"123","author":"TG Monk","year":"2015","unstructured":"Monk, T. G. et al. Association between intraoperative hypotension and hypertension and 30-day postoperative mortality in noncardiac surgery. Anesthesiology 123, 307\u2013319 (2015).","journal-title":"Anesthesiology"},{"key":"248_CR17","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1016\/S0140-6736(18)33139-8","volume":"393","author":"D Nepogodiev","year":"2019","unstructured":"Nepogodiev, D. et al. Global burden of postoperative death. Lancet 393, 401 (2019).","journal-title":"Lancet"},{"key":"248_CR18","doi-asserted-by":"publisher","first-page":"682","DOI":"10.1213\/ANE.0000000000002606","volume":"126","author":"S Aronson","year":"2018","unstructured":"Aronson, S. et al. A perioperative medicine model for population health: an integrated approach for an evolving clinical science. Anesth. Analg. 126, 682\u2013690 (2018).","journal-title":"Anesth. Analg."},{"key":"248_CR19","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.bja.2018.10.021","volume":"122","author":"F Carli","year":"2019","unstructured":"Carli, F. & Feldman, L. S. From preoperative risk assessment and prediction to risk attenuation: a case for prehabilitation. Br. J. Anaesth. 122, 11\u201313 (2019).","journal-title":"Br. J. Anaesth."},{"key":"248_CR20","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1097\/00000542-197810000-00001","volume":"49","author":"AS Keats","year":"1978","unstructured":"Keats, A. S. The ASA classification of physical status-a recapitulation. Anesthesiology 49, 233\u2013236 (1978).","journal-title":"Anesthesiology"},{"key":"248_CR21","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1097\/ALN.0000000000000972","volume":"124","author":"Y Le Manach","year":"2016","unstructured":"Le Manach, Y. et al. Preoperative score to predict postoperative mortality (POSPOM): derivation and validation. Anesthesiology 124, 570\u2013579 (2016).","journal-title":"Anesthesiology"},{"key":"248_CR22","doi-asserted-by":"publisher","first-page":"707","DOI":"10.1007\/BF01709751","volume":"22","author":"JL Vincent","year":"1996","unstructured":"Vincent, J. L. et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction\/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 22, 707\u2013710 (1996).","journal-title":"Intensive Care Med."},{"key":"248_CR23","doi-asserted-by":"publisher","first-page":"768","DOI":"10.1093\/bja\/aes022","volume":"108","author":"F Chung","year":"2012","unstructured":"Chung, F. et al. High STOP-Bang score indicates a high probability of obstructive sleep apnoea. Br. J. Anaesth. 108, 768\u2013775 (2012).","journal-title":"Br. J. Anaesth."},{"key":"248_CR24","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1097\/ALN.0000000000002186","volume":"129","author":"CK Lee","year":"2018","unstructured":"Lee, C. K., Hofer, I., Gabel, E., Baldi, P. & Cannesson, M. Development and validation of a deep neural network model for prediction of postoperative in-hospital mortality. Anesthesiology 129, 649\u2013662 (2018).","journal-title":"Anesthesiology"},{"key":"248_CR25","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1097\/ALN.0000000000002300","volume":"129","author":"F Hatib","year":"2018","unstructured":"Hatib, F. et al. Machine-learning algorithm to predict hypotension based on high-fidelity arterial pressure waveform analysis. Anesthesiology 129, 663\u2013674 (2018).","journal-title":"Anesthesiology"},{"key":"248_CR26","doi-asserted-by":"publisher","first-page":"675","DOI":"10.1097\/ALN.0000000000002374","volume":"129","author":"S Kendale","year":"2018","unstructured":"Kendale, S., Kulkarni, P., Rosenberg, A. D. & Wang, J. Supervised machine-learning predictive analytics for prediction of postinduction hypotension. Anesthesiology 129, 675\u2013688 (2018).","journal-title":"Anesthesiology"},{"key":"248_CR27","doi-asserted-by":"publisher","first-page":"525","DOI":"10.1097\/ALN.0b013e31829ce6e6","volume":"119","author":"MJ Sigakis","year":"2013","unstructured":"Sigakis, M. J., Bittner, E. A. & Wanderer, J. P. Validation of a risk stratification index and risk quantification index for predicting patient outcomes: in-hospital mortality, 30-day mortality, 1-year mortality, and length-of-stay. Anesthesiology 119, 525\u2013540 (2013).","journal-title":"Anesthesiology"},{"key":"248_CR28","unstructured":"Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. R. Improving neural networks by preventing co-adaptation of feature detectors. Preprint at https:\/\/arxiv.org\/abs\/1207.0580 (2012)."},{"key":"248_CR29","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.5870","volume":"18","author":"W Luo","year":"2016","unstructured":"Luo, W. et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J. Med. Internet Res. 18, e323 (2016).","journal-title":"J. Med. Internet Res."},{"key":"248_CR30","doi-asserted-by":"publisher","first-page":"1880","DOI":"10.1213\/ANE.0000000000001201","volume":"122","author":"IS Hofer","year":"2016","unstructured":"Hofer, I. S., Gabel, E., Pfeffer, M., Mahbouba, M. & Mahajan, A. A systematic approach to creation of a perioperative data warehouse. Anesth. Analg. 122, 1880\u20131884 (2016).","journal-title":"Anesth. Analg."},{"key":"248_CR31","doi-asserted-by":"publisher","DOI":"10.1186\/cc5713","volume":"11","author":"RL Mehta","year":"2007","unstructured":"Mehta, R. L. et al. Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury. Crit. Care 11, R31 (2007).","journal-title":"Crit. Care"},{"key":"248_CR32","doi-asserted-by":"publisher","first-page":"1423","DOI":"10.1213\/ANE.0000000000001997","volume":"124","author":"E Gabel","year":"2017","unstructured":"Gabel, E. et al. Creation and validation of an automated algorithm to determine postoperative ventilator requirements after cardiac surgery. Anesth. Analg. 124, 1423\u20131430 (2017).","journal-title":"Anesth. Analg."},{"key":"248_CR33","doi-asserted-by":"publisher","first-page":"933","DOI":"10.1213\/ANE.0000000000001526","volume":"123","author":"JK Hsieh","year":"2016","unstructured":"Hsieh, J. K. et al. The association between mild intraoperative hypotension and stroke in general surgery patients. Anesth. Analg. 123, 933\u2013939 (2016).","journal-title":"Anesth. Analg."},{"key":"248_CR34","first-page":"1929","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929\u20131958 (2014).","journal-title":"J. Mach. Learn. Res."},{"key":"248_CR35","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1016\/j.artint.2014.02.004","volume":"210","author":"P Baldi","year":"2014","unstructured":"Baldi, P. & Sadowski, P. The dropout learning algorithm. Artif. Intell. 210, 78\u2013122 (2014).","journal-title":"Artif. Intell."},{"key":"248_CR36","doi-asserted-by":"publisher","first-page":"1336","DOI":"10.1097\/ALN.0b013e318219d5f9","volume":"114","author":"JE Dalton","year":"2011","unstructured":"Dalton, J. E. et al. Development and validation of a risk quantification index for 30-day postoperative mortality and morbidity in noncardiac surgical patients. Anesthesiology 114, 1336\u20131344 (2011).","journal-title":"Anesthesiology"},{"key":"248_CR37","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825\u20132830 (2011).","journal-title":"J. Mach. Learn. Res."},{"key":"248_CR38","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1007\/BF02295996","volume":"12","author":"Q McNemar","year":"1947","unstructured":"McNemar, Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12, 153\u2013157 (1947).","journal-title":"Psychometrika"},{"key":"248_CR39","doi-asserted-by":"publisher","first-page":"638","DOI":"10.21105\/joss.00638","volume":"3","author":"S Raschka","year":"2018","unstructured":"Raschka, S. MLxtend: providing machine learning and data science utilities and extensions to Python\u2019s scientific computing stack. J. Open Source Softw. 3, 638 (2018).","journal-title":"J. Open Source Softw."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0248-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0248-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0248-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T02:10:10Z","timestamp":1670379010000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0248-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,20]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["248"],"URL":"https:\/\/doi.org\/10.1038\/s41746-020-0248-0","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,20]]},"assertion":[{"value":"14 June 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 February 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 April 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"M.C. is a consultant for Edwards Lifesciences and Masimo Corp, and has funded research from Edwards Lifesciences and Masimo Corp. He is also the founder of Sironis and owns patents for closed loop hemodynamic management that have been licensed to Edwards Lifesciences. I.S.H. is the founder and President of Clarity Healthcare Analytics Inc., a company that assists hospitals with extracting and using data from their EMRs. I.S.H. also receives research funding from Merck Pharmaceuticals. E.G. is founder and Secretary of Clarity Healthcare Analytics Inc., a company that assists hospitals with extracting and using data from their EMRs. C.L. is an employee of Edwards Lifesciences.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"58"}}