{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T01:12:06Z","timestamp":1774401126409,"version":"3.50.1"},"reference-count":67,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,10,26]],"date-time":"2020-10-26T00:00:00Z","timestamp":1603670400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,10,26]],"date-time":"2020-10-26T00:00:00Z","timestamp":1603670400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Acute kidney injury (AKI) is a major complication after cardiothoracic surgery. Early prediction of AKI could prompt preventive measures, but is challenging in the clinical routine. One important reason is that the amount of postoperative data is too massive and too high-dimensional to be effectively processed by the human operator. We therefore sought to develop a deep-learning-based algorithm that is able to predict postoperative AKI prior to the onset of symptoms and complications. Based on 96 routinely collected parameters we built a recurrent neural network (RNN) for real-time prediction of AKI after cardiothoracic surgery. From the data of 15,564 admissions we constructed a balanced training set (2224 admissions) for the development of the RNN. The model was then evaluated on an independent test set (350 admissions) and yielded an area under curve (AUC) (95% confidence interval) of 0.893 (0.862\u20130.924). We compared the performance of our model against that of experienced clinicians. The RNN significantly outperformed clinicians (AUC\u2009=\u20090.901 vs. 0.745, <jats:italic>p<\/jats:italic>\u2009&lt;\u20090.001) and was overall well calibrated. This was not the case for the physicians, who systematically underestimated the risk (<jats:italic>p<\/jats:italic>\u2009&lt;\u20090.001). In conclusion, the RNN was superior to physicians in the prediction of AKI after cardiothoracic surgery. It could potentially be integrated into hospitals\u2019 electronic health records for real-time patient monitoring and may help to detect early AKI and hence modify the treatment in perioperative care.<\/jats:p>","DOI":"10.1038\/s41746-020-00346-8","type":"journal-article","created":{"date-parts":[[2020,10,26]],"date-time":"2020-10-26T11:03:40Z","timestamp":1603710220000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":127,"title":["Deep-learning-based real-time prediction of acute kidney injury outperforms human predictive performance"],"prefix":"10.1038","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5984-4836","authenticated-orcid":false,"given":"Nina","family":"Rank","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7921-8103","authenticated-orcid":false,"given":"Boris","family":"Pfahringer","sequence":"additional","affiliation":[]},{"given":"J\u00f6rg","family":"Kempfert","sequence":"additional","affiliation":[]},{"given":"Christof","family":"Stamm","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1631-4824","authenticated-orcid":false,"given":"Titus","family":"K\u00fchne","sequence":"additional","affiliation":[]},{"given":"Felix","family":"Schoenrath","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7911-8620","authenticated-orcid":false,"given":"Volkmar","family":"Falk","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9895-4061","authenticated-orcid":false,"given":"Carsten","family":"Eickhoff","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6944-2478","authenticated-orcid":false,"given":"Alexander","family":"Meyer","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,10,26]]},"reference":[{"key":"346_CR1","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1016\/S0002-9343(98)00058-8","volume":"104","author":"GM Chertow","year":"1998","unstructured":"Chertow, G. M., Levy, E. M., Hammermeister, K. E., Grover, F. & Daley, J. Independent association between acute renal failure and mortality following cardiac surgery 12. Am. J. Med. 104, 343\u2013348 (1998).","journal-title":"Am. J. Med."},{"key":"346_CR2","doi-asserted-by":"publisher","first-page":"2444","DOI":"10.1161\/CIRCULATIONAHA.108.800011","volume":"119","author":"CE Hobson","year":"2009","unstructured":"Hobson, C. E. et al. Acute kidney injury is associated with increased long-term mortality after cardiothoracic surgery. Circulation 119, 2444\u20132453 (2009).","journal-title":"Circulation"},{"key":"346_CR3","doi-asserted-by":"publisher","first-page":"2659","DOI":"10.1097\/CCM.0b013e3182281f1b","volume":"39","author":"T Mandelbaum","year":"2011","unstructured":"Mandelbaum, T. et al. Outcome of critically ill patients with acute kidney injury using the Acute Kidney Injury Network criteria. Crit. Care Med. 39, 2659\u20132664 (2011).","journal-title":"Crit. Care Med."},{"key":"346_CR4","doi-asserted-by":"publisher","first-page":"827","DOI":"10.1016\/j.amjmed.2005.01.069","volume":"118","author":"YP Ympa","year":"2005","unstructured":"Ympa, Y. P., Sakr, Y., Reinhart, K. & Vincent, J.-L. Has mortality from acute renal failure decreased? A systematic review of the literature. Am. J. Med. 118, 827\u2013832 (2005).","journal-title":"Am. J. Med."},{"key":"346_CR5","doi-asserted-by":"publisher","first-page":"1207","DOI":"10.1097\/SLA.0000000000000732","volume":"261","author":"C Hobson","year":"2015","unstructured":"Hobson, C. et al. Cost and mortality associated with postoperative acute kidney injury. Ann. Surg. 261, 1207\u20131214 (2015).","journal-title":"Ann. Surg."},{"key":"346_CR6","doi-asserted-by":"publisher","first-page":"70","DOI":"10.12788\/jhm.2683","volume":"12","author":"SA Silver","year":"2017","unstructured":"Silver, S. A., Long, J., Zheng, Y. & Chertow, G. M. Cost of acute kidney injury in hospitalized patients. J. Hosp. Med. 12, 70\u201376 (2017).","journal-title":"J. Hosp. Med."},{"key":"346_CR7","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1159\/000475607","volume":"137","author":"SA Silver","year":"2017","unstructured":"Silver, S. A. & Chertow, G. M. The economic consequences of acute kidney injury. Nephron 137, 297\u2013301 (2017).","journal-title":"Nephron"},{"key":"346_CR8","doi-asserted-by":"crossref","first-page":"c179","DOI":"10.1159\/000339789","volume":"120","author":"A Khwaja","year":"2012","unstructured":"Khwaja, A. KDIGO clinical practice guidelines for acute kidney injury. Nephron Clin. Pract. 120, c179\u2013c184 (2012).","journal-title":"Nephron Clin. Pract."},{"key":"346_CR9","doi-asserted-by":"publisher","first-page":"A211","DOI":"10.1016\/S0735-1097(15)60211-X","volume":"65","author":"I Spanuchart","year":"2015","unstructured":"Spanuchart, I., Cheungpasitporn, W., Thongprayoon, C., Ratanapo, S. & Srivali, N. Off-pump versus on-pump coronary artery bypass surgery: an updated meta-analysis of randomized controlled trials on acute kidney injury and mortality outcomes. J. Am. Coll. Cardiol. 65, A211 (2015).","journal-title":"J. Am. Coll. Cardiol."},{"key":"346_CR10","doi-asserted-by":"publisher","first-page":"1734","DOI":"10.2215\/CJN.02800310","volume":"5","author":"VF Seabra","year":"2010","unstructured":"Seabra, V. F., Alobaidi, S., Balk, E. M., Poon, A. H. & Jaber, B. L. Off-pump coronary artery bypass surgery and acute kidney injury: a meta-analysis of randomized controlled trials. Clin. J. Am. Soc. Nephrol. 5, 1734\u20131744 (2010).","journal-title":"Clin. J. Am. Soc. Nephrol."},{"issue":"Suppl 2","key":"346_CR11","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1159\/000361062","volume":"37","author":"H Mao","year":"2014","unstructured":"Mao, H. et al. Cardiac surgery-associated acute kidney injury. Blood Purif. 37(Suppl 2), 34\u201350 (2014).","journal-title":"Blood Purif."},{"key":"346_CR12","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1038\/nrneph.2017.119","volume":"13","author":"Y Wang","year":"2017","unstructured":"Wang, Y. & Bellomo, R. Cardiac surgery-associated acute kidney injury: risk factors, pathophysiology and treatment. Nat. Rev. Nephrol. 13, 697\u2013711 (2017).","journal-title":"Nat. Rev. Nephrol."},{"key":"346_CR13","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1053\/j.ackd.2016.02.007","volume":"23","author":"S Faubel","year":"2016","unstructured":"Faubel, S. & Shah, P. B. Immediate consequences of acute kidney injury: the impact of traditional and nontraditional complications on mortality in acute kidney injury. Adv. Chronic Kidney Dis. 23, 179\u2013185 (2016).","journal-title":"Adv. Chronic Kidney Dis."},{"key":"346_CR14","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.bone.2017.03.047","volume":"100","author":"CCW Hsia","year":"2017","unstructured":"Hsia, C. C. W., Ravikumar, P. & Ye, J. Acute lung injury complicating acute kidney injury: a model of endogenous \u03b1Klotho deficiency and distant organ dysfunction. Bone 100, 100\u2013109 (2017).","journal-title":"Bone"},{"key":"346_CR15","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1007\/s00134-010-2089-9","volume":"37","author":"RL Mehta","year":"2011","unstructured":"Mehta, R. L. et al. Sepsis as a cause and consequence of acute kidney injury: Program to Improve Care in Acute Renal Disease. Intensive Care Med. 37, 241\u2013248 (2011).","journal-title":"Intensive Care Med."},{"key":"346_CR16","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1038\/ki.2011.379","volume":"81","author":"SG Coca","year":"2012","unstructured":"Coca, S. G., Singanamala, S. & Parikh, C. R. Chronic kidney disease after acute kidney injury: a systematic review and meta-analysis. Kidney Int. 81, 442\u2013448 (2012).","journal-title":"Kidney Int."},{"key":"346_CR17","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1053\/j.ajkd.2010.08.026","volume":"57","author":"G Balasubramanian","year":"2011","unstructured":"Balasubramanian, G. et al. Early nephrologist involvement in hospital-acquired acute kidney injury: a pilot study. Am. J. Kidney Dis. 57, 228\u2013234 (2011).","journal-title":"Am. J. Kidney Dis."},{"key":"346_CR18","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0070482","volume":"8","author":"VT Costa e Silva","year":"2013","unstructured":"Costa e Silva, V. T. et al. Nephrology referral and outcomes in critically ill acute kidney injury patients. PLoS ONE 8, e70482 (2013).","journal-title":"PLoS ONE"},{"key":"346_CR19","doi-asserted-by":"publisher","first-page":"1551","DOI":"10.1007\/s00134-016-4670-3","volume":"43","author":"M Meersch","year":"2017","unstructured":"Meersch, M. et al. Prevention of cardiac surgery-associated AKI by implementing the KDIGO guidelines in high risk patients identified by biomarkers: the PrevAKI randomized controlled trial. Intensive Care Med. 43, 1551\u20131561 (2017).","journal-title":"Intensive Care Med."},{"key":"346_CR20","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1016\/j.athoracsur.2011.09.010","volume":"93","author":"SC Huen","year":"2012","unstructured":"Huen, S. C. & Parikh, C. R. Predicting acute kidney injury after cardiac surgery: a systematic review. Ann. Thorac. Surg. 93, 337\u2013347 (2012).","journal-title":"Ann. Thorac. Surg."},{"key":"346_CR21","doi-asserted-by":"publisher","first-page":"878","DOI":"10.1161\/01.CIR.95.4.878","volume":"95","author":"GM Chertow","year":"1997","unstructured":"Chertow, G. M. et al. Preoperative renal risk stratification. Circulation 95, 878\u2013884 (1997).","journal-title":"Circulation"},{"key":"346_CR22","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1681\/ASN.2004040331","volume":"16","author":"CV Thakar","year":"2005","unstructured":"Thakar, C. V., Arrigain, S., Worley, S., Yared, J.-P. & Paganini, E. P. A clinical score to predict acute renal failure after cardiac surgery. J. Am. Soc. Nephrol. 16, 162\u2013168 (2005).","journal-title":"J. Am. Soc. Nephrol."},{"key":"346_CR23","doi-asserted-by":"publisher","first-page":"2208","DOI":"10.1161\/CIRCULATIONAHA.106.635573","volume":"114","author":"RH Mehta","year":"2006","unstructured":"Mehta, R. H. et al. Bedside tool for predicting the risk of postoperative dialysis in patients undergoing cardiac surgery. Circulation 114, 2208\u20132216 (2006). quiz 2208.","journal-title":"Circulation"},{"key":"346_CR24","doi-asserted-by":"publisher","first-page":"624","DOI":"10.1038\/sj.ki.5002419","volume":"72","author":"H Palomba","year":"2007","unstructured":"Palomba, H., de Castro, I., Neto, A. L. C., Lage, S. & Yu, L. Acute kidney injury prediction following elective cardiac surgery: AKICS Score. Kidney Int. 72, 624\u2013631 (2007).","journal-title":"Kidney Int."},{"key":"346_CR25","doi-asserted-by":"publisher","first-page":"733","DOI":"10.1161\/CIRCULATIONAHA.106.623538","volume":"115","author":"S Aronson","year":"2007","unstructured":"Aronson, S. et al. Risk index for perioperative renal dysfunction\/failure: critical dependence on pulse pressure hypertension. Circulation 115, 733\u2013742 (2007).","journal-title":"Circulation"},{"key":"346_CR26","doi-asserted-by":"publisher","first-page":"1801","DOI":"10.1001\/jama.297.16.1801","volume":"297","author":"DN Wijeysundera","year":"2007","unstructured":"Wijeysundera, D. N. et al. Derivation and validation of a simplified predictive index for renal replacement therapy after cardiac surgery. JAMA 297, 1801\u20131809 (2007).","journal-title":"JAMA"},{"key":"346_CR27","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1111\/j.0956-7976.2005.00782.x","volume":"16","author":"GS Halford","year":"2005","unstructured":"Halford, G. S., Baker, R., McCredden, J. E. & Bain, J. D. How many variables can humans process? Psychol. Sci. 16, 70\u201376 (2005).","journal-title":"Psychol. Sci."},{"key":"346_CR28","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0155705","volume":"11","author":"P Thottakkara","year":"2016","unstructured":"Thottakkara, P. et al. Application of machine learning techniques to high-dimensional clinical data to forecast postoperative complications. PLoS ONE 11, e0155705 (2016).","journal-title":"PLoS ONE"},{"key":"346_CR29","doi-asserted-by":"publisher","first-page":"652","DOI":"10.1097\/SLA.0000000000002706","volume":"269","author":"A Bihorac","year":"2019","unstructured":"Bihorac, A. et al. MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery. Ann. Surg. 269, 652\u2013662 (2019).","journal-title":"Ann. Surg."},{"key":"346_CR30","doi-asserted-by":"publisher","first-page":"1935","DOI":"10.2215\/CJN.00280116","volume":"11","author":"JL Koyner","year":"2016","unstructured":"Koyner, J. L., Adhikari, R. & Edelson, D. P. Development of a multicenter ward\u2013based AKI prediction model. Clin. J. Am. Soc. Nephrol. 11, 1935\u20131943 (2016).","journal-title":"Clin. J. Am. Soc. Nephrol."},{"key":"346_CR31","doi-asserted-by":"publisher","first-page":"1070","DOI":"10.1097\/CCM.0000000000003123","volume":"46","author":"JL Koyner","year":"2018","unstructured":"Koyner, J. L., Carey, K. A., Edelson, D. P. & Churpek, M. M. The development of a machine learning inpatient acute kidney injury prediction model. Crit. Care Med. 46, 1070\u20131077 (2018).","journal-title":"Crit. Care Med."},{"key":"346_CR32","first-page":"565","volume":"2017","author":"P Cheng","year":"2017","unstructured":"Cheng, P., Waitman, L. R., Hu, Y. & Liu, M. Predicting inpatient acute kidney injury over different time horizons: how early and accurate? AMIA Annu. Symp. Proc. 2017, 565\u2013574 (2017).","journal-title":"AMIA Annu. Symp. Proc."},{"key":"346_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/2054358118776326","volume":"5","author":"H Mohamadlou","year":"2018","unstructured":"Mohamadlou, H. et al. Prediction of acute kidney injury with a machine learning algorithm using electronic health record data. Can. J. Kidney Health Dis. 5, 1\u20139 (2018).","journal-title":"Can. J. Kidney Health Dis."},{"key":"346_CR34","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1038\/s41586-019-1390-1","volume":"572","author":"N Toma\u0161ev","year":"2019","unstructured":"Toma\u0161ev, N. et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature 572, 116\u2013119 (2019).","journal-title":"Nature"},{"key":"346_CR35","doi-asserted-by":"publisher","first-page":"905","DOI":"10.1016\/S2213-2600(18)30300-X","volume":"6","author":"A Meyer","year":"2018","unstructured":"Meyer, A. et al. Machine learning for real-time prediction of complications in critical care: a retrospective study. Lancet Respir. Med. 6, 905\u2013914 (2018).","journal-title":"Lancet Respir. Med."},{"key":"346_CR36","doi-asserted-by":"crossref","unstructured":"Hosmer, D. W., Jr., Lemeshow, S. & Sturdivant, R. X. Applied Logistic Regression (John Wiley & Sons, 2013).","DOI":"10.1002\/9781118548387"},{"key":"346_CR37","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1097\/01.mnh.0000165894.90748.72","volume":"14","author":"ML Praught","year":"2005","unstructured":"Praught, M. L. & Shlipak, M. G. Are small changes in serum creatinine an important risk factor? Curr. Opin. Nephrol. Hypertens. 14, 265\u2013270 (2005).","journal-title":"Curr. Opin. Nephrol. Hypertens."},{"key":"346_CR38","doi-asserted-by":"publisher","first-page":"I139","DOI":"10.1161\/CIRCULATIONAHA.107.723213","volume":"116","author":"JR Brown","year":"2007","unstructured":"Brown, J. R. et al. Multivariable prediction of renal insufficiency developing after cardiac surgery. Circulation 116, I139\u2013I143 (2007).","journal-title":"Circulation"},{"key":"346_CR39","doi-asserted-by":"publisher","first-page":"1935","DOI":"10.2215\/CJN.00280116","volume":"11","author":"JL Koyner","year":"2016","unstructured":"Koyner, J. L., Adhikari, R., Edelson, D. P. & Churpek, M. M. Development of a multicenter ward-based AKI prediction model. Clin. J. Am. Soc. Nephrol. 11, 1935\u20131943 (2016).","journal-title":"Clin. J. Am. Soc. Nephrol."},{"key":"346_CR40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/2054358118776326","volume":"5","author":"H Mohamadlou","year":"2018","unstructured":"Mohamadlou, H. et al. Prediction of acute kidney injury with a machine learning algorithm using electronic health record data. Can. J. Kidney Health Dis. 5, 1\u20139 (2018).","journal-title":"Can. J. Kidney Health Dis."},{"key":"346_CR41","doi-asserted-by":"publisher","first-page":"55","DOI":"10.7326\/M14-0697","volume":"162","author":"GS Collins","year":"2015","unstructured":"Collins, G. S., Reitsma, J. B., Altman, D. G. & Moons, K. G. M. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann. Intern. Med. 162, 55\u201363 (2015).","journal-title":"Ann. Intern. Med."},{"key":"346_CR42","first-page":"743","volume":"78","author":"CA Naughton","year":"2008","unstructured":"Naughton, C. A. Drug-induced nephrotoxicity. Am. Fam. Physician 78, 743\u2013750 (2008).","journal-title":"Am. Fam. Physician"},{"key":"346_CR43","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1007\/BF03160941","volume":"4","author":"M Mazer","year":"2008","unstructured":"Mazer, M. & Perrone, J. Acetaminophen-induced nephrotoxicity: pathophysiology, clinical manifestations, and management. J. Med. Toxicol. 4, 2\u20136 (2008).","journal-title":"J. Med. Toxicol."},{"key":"346_CR44","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1177\/2168479014536897","volume":"49","author":"A Kitano","year":"2014","unstructured":"Kitano, A., Motohashi, H., Takayama, A., Inui, K.-I. & Yano, Y. Valacyclovir-Induced Acute Kidney Injury in Japanese Patients Based on the PMDA Adverse Drug Reactions Reporting Database. Drug Inf. J. 49, 81\u201385 (2014).","journal-title":"Drug Inf. J."},{"key":"346_CR45","doi-asserted-by":"publisher","first-page":"121","DOI":"10.4103\/1319-2442.124524","volume":"25","author":"MD Redondo-Pachon","year":"2014","unstructured":"Redondo-Pachon, M. D. et al. Acute renal failure and severe thrombocytopenia associated with metamizole. Saudi J. Kidney Dis. Transpl. 25, 121\u2013125 (2014).","journal-title":"Saudi J. Kidney Dis. Transpl."},{"key":"346_CR46","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.1056\/NEJMoa070403","volume":"358","author":"CG Koch","year":"2008","unstructured":"Koch, C. G. et al. Duration of red-cell storage and complications after cardiac surgery. N. Engl. J. Med. 358, 1229\u20131239 (2008).","journal-title":"N. Engl. J. Med."},{"key":"346_CR47","doi-asserted-by":"publisher","first-page":"680","DOI":"10.1161\/CIRCINTERVENTIONS.112.971291","volume":"5","author":"R-J Nuis","year":"2012","unstructured":"Nuis, R.-J. et al. Blood transfusion and the risk of acute kidney injury after transcatheter aortic valve implantation. Circ. Cardiovasc. Interv. 5, 680\u2013688 (2012).","journal-title":"Circ. Cardiovasc. Interv."},{"key":"346_CR48","doi-asserted-by":"publisher","first-page":"444","DOI":"10.1109\/JPROC.2015.2501978","volume":"104","author":"AEW Johnson","year":"2016","unstructured":"Johnson, A. E. W. et al. Machine Learning and Decision Support in Critical Care. Proc. IEEE Inst. Electr. Electron. Eng. 104, 444\u2013466 (2016).","journal-title":"Proc. IEEE Inst. Electr. Electron. Eng."},{"key":"346_CR49","doi-asserted-by":"crossref","unstructured":"LeCun, Y. A., Bottou, L., Orr, G. B. & M\u00fcller, K.-R. in Neural Networks: Tricks of the Trade: Second Edition (eds. Montavon, G., Orr, G. B. & M\u00fcller, K.-R.) 9\u201348 (Springer Berlin Heidelberg, 2012).","DOI":"10.1007\/978-3-642-35289-8_3"},{"key":"346_CR50","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1109\/MCSE.2007.53","volume":"9","author":"F Perez","year":"2007","unstructured":"Perez, F. & Granger, B. E. IPython: a system for interactive scientific computing. Comput. Sci. Eng. 9, 21\u201329 (2007).","journal-title":"Comput. Sci. Eng."},{"key":"346_CR51","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1109\/MCSE.2007.55","volume":"9","author":"JD Hunter","year":"2007","unstructured":"Hunter, J. D. Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9, 90\u201395 (2007).","journal-title":"Comput. Sci. Eng."},{"key":"346_CR52","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825\u20132830 (2011).","journal-title":"J. Mach. Learn. Res."},{"key":"346_CR53","first-page":"51","volume":"445","author":"W McKinney","year":"2010","unstructured":"McKinney, W., Others. Data structures for statistical computing in python. Proc. 9th Python Sci. Conf. 445, 51\u201356 (2010). Austin, TX.","journal-title":"Proc. 9th Python Sci. Conf."},{"key":"346_CR54","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1109\/MCSE.2011.37","volume":"13","author":"S van der Walt","year":"2011","unstructured":"van der Walt, S., Colbert, S. C. & Varoquaux, G. The NumPy Array: A Structure for Efficient Numerical Computation. Comput. Sci. Eng. 13, 22\u201330 (2011).","journal-title":"Comput. Sci. Eng."},{"key":"346_CR55","unstructured":"Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Preprint at http:\/\/arxiv.org\/abs\/1412.6980 (2014)"},{"key":"346_CR56","unstructured":"Abadi, M. et al. TensorFlow: large-scale machine learning on heterogeneous distributed systems, 2015. Software available from https:\/\/www.tensorflow.org\/about\/bib."},{"key":"346_CR57","doi-asserted-by":"crossref","unstructured":"BRIER & W, G. Verification of Forecasts Expressed in terms of probability. Monthey Weather Rev. 78, 1\u20133 (1950).","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2"},{"key":"346_CR58","unstructured":"Kalton, G., Michael Brick, J. & L\u00ea, T. Chapter VI Estimating components of design effects for use in sample design. http:\/\/citeseerx.ist.psu.edu\/viewdoc\/summary?doi:10.1.1.522.3221."},{"key":"346_CR59","unstructured":"Gonzalez, E. J. & Foy, P. Third International Mathematics and Science Study, Technical Report: Estimation of sampling variability, design effects, and effective sample sizes. p. 87 (II, Boston College Chestnut Hill, Massachusetts, USA, 1997)."},{"key":"346_CR60","doi-asserted-by":"publisher","first-page":"377","DOI":"10.1002\/1097-0258(20010215)20:3<377::AID-SIM799>3.0.CO;2-N","volume":"20","author":"SM Kerry","year":"2001","unstructured":"Kerry, S. M. & Bland, J. M. Unequal cluster sizes for trials in English and Welsh general practice: implications for sample size calculations. Stat. Med. 20, 377\u2013390 (2001).","journal-title":"Stat. Med."},{"key":"346_CR61","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1111\/j.2041-210X.2011.00125.x","volume":"3","author":"ME Wolak","year":"2012","unstructured":"Wolak, M. E., Fairbairn, D. J. & Paulsen, Y. R. Guidelines for estimating repeatability. Methods Ecol. Evol. 3, 129\u2013137 (2012).","journal-title":"Methods Ecol. Evol."},{"key":"346_CR62","unstructured":"Press, W. H., Teukolsky, S. A., Vetterling, W. T. & Flannery, B. P. In Numerical Recipes in FORTRAN: The Art of Scientific Computing 2, 299\u2013306 (Cambridge University Press, 1992)."},{"key":"346_CR63","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1007\/BF01475864","volume":"77","author":"H Weyl","year":"1916","unstructured":"Weyl, H. \u00dcber die Gleichverteilung von Zahlen mod. Eins. Math. Ann. 77, 313\u2013352 (1916).","journal-title":"Math. Ann."},{"key":"346_CR64","doi-asserted-by":"publisher","first-page":"837","DOI":"10.2307\/2531595","volume":"44","author":"ER DeLong","year":"1988","unstructured":"DeLong, E. R., DeLong, D. M. & Clarke-Pearson, D. L. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44, 837\u2013845 (1988).","journal-title":"Biometrics"},{"key":"346_CR65","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-12-77","volume":"12","author":"X Robin","year":"2011","unstructured":"Robin, X. et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinforma. 12, 77 (2011).","journal-title":"BMC Bioinforma."},{"key":"346_CR66","unstructured":"Lele, S. R., Keim, J. L. & Solymos, P. ResourceSelection: resource selection (probability) functions for use-availability data. R package version 3\u20132 (2017). Software available at https:\/\/cran.r-project.org\/src\/contrib\/Archive\/ResourceSelection\/."},{"key":"346_CR67","volume":"5","author":"W Jiang","year":"2016","unstructured":"Jiang, W. et al. Dynamic predictive scores for cardiac surgery\u2013associated acute kidney injury. J. Am. Heart Assoc. 5, e003754 (2016).","journal-title":"J. Am. Heart Assoc."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00346-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00346-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00346-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T02:41:17Z","timestamp":1670380877000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00346-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,26]]},"references-count":67,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["346"],"URL":"https:\/\/doi.org\/10.1038\/s41746-020-00346-8","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,26]]},"assertion":[{"value":"10 February 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 September 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 October 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A.M. declares the receipt of consulting and lecturing fees from Medtronic GmbH and Edwards Lifesciences Services GmbH, and consulting fees from Pfizer. C.E. declares ownership of shares in codiag AG. F.S. declares the receipt of honoraria, consultancy fees or travel support from Medtronic GmbH, Biotronik SE & Co., Abbott GmbH & Co. KG, Sanofi S.A., Cardiorentis AG, Novartis Pharma GmbH. J.K. declares the receipt of lecturing fees from, Boston Scientific, LSI Solutions, Edwards, Medtronic, Abbott, Ascyrus Medical GmbH. V.F. declares (institutional) financial activities with Medtronic, Biotronik, Abbott, Boston, Edwards, Berlin Heart, Novartis, Jotec, Zurich Heart in relation to Educational Grants, honoraria, consultancy, research & study funds, fees for travel support. All other authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"139"}}