{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T08:31:02Z","timestamp":1780389062929,"version":"3.54.1"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"S10","license":[{"start":{"date-parts":[[2020,8,1]],"date-time":"2020-08-01T00:00:00Z","timestamp":1596240000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,8,25]],"date-time":"2020-08-25T00:00:00Z","timestamp":1598313600000},"content-version":"vor","delay-in-days":24,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2020,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n<jats:title>Background<\/jats:title>\n<jats:p>The scope of this work is to build a Machine Learning model able to predict patients risk to contract a multidrug resistant urinary tract infection (MDR UTI) after hospitalization. To achieve this goal, we used different popular Machine Learning tools. Moreover, we integrated an easy-to-use cloud platform, called <jats:italic>DSaaS<\/jats:italic> (<jats:italic>Data Science as a Service<\/jats:italic>), well suited for hospital structures, where healthcare operators might not have specific competences in using programming languages but still, they do need to analyze data as a continuous process. Moreover, <jats:italic>DSaaS<\/jats:italic> allows the validation of data analysis models based on supervised Machine Learning regression and classification algorithms.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Results<\/jats:title>\n<jats:p>We used <jats:italic>DSaaS<\/jats:italic> on a real antibiotic stewardship dataset to make predictions about antibiotic resistance in the Clinical Pathology Operative Unit of the Principe di Piemonte Hospital in Senigallia, Marche, Italy. Data related to a total of 1486 hospitalized patients with nosocomial urinary tract infection (UTI). Sex, age, age class, ward and time period, were used to predict the onset of a MDR UTI. Machine Learning methods such as Catboost, Support Vector Machine and Neural Networks were utilized to build predictive models. Among the performance evaluators, already implemented in <jats:italic>DSaaS,<\/jats:italic> we used accuracy (ACC), area under receiver operating characteristic curve (AUC-ROC), area under Precision-Recall curve (AUC-PRC), F1 score, sensitivity (SEN), specificity and Matthews correlation coefficient (MCC). Catboost exhibited the best predictive results (MCC 0.909; SEN 0.904; F1 score 0.809; AUC-PRC 0.853, AUC-ROC 0.739; ACC 0.717) with the highest value in every metric.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Conclusions<\/jats:title>\n<jats:p>the predictive model built with <jats:italic>DSaaS<\/jats:italic> may serve as a useful support tool for physicians treating hospitalized patients with a high risk to acquire MDR UTIs. We obtained these results using only five easy and fast predictors accessible for each patient hospitalization. In future, <jats:italic>DSaaS<\/jats:italic> will be enriched with more features like unsupervised Machine Learning techniques, streaming data analysis, distributed calculation and big data storage and management to allow researchers to perform a complete data analysis pipeline. The <jats:italic>DSaaS<\/jats:italic> prototype is available as a demo at the following address: <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/dsaas-demo.shinyapps.io\/Server\/\">https:\/\/dsaas-demo.shinyapps.io\/Server\/<\/jats:ext-link><\/jats:p>\n<\/jats:sec>","DOI":"10.1186\/s12859-020-03566-7","type":"journal-article","created":{"date-parts":[[2020,8,25]],"date-time":"2020-08-25T02:02:24Z","timestamp":1598320944000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":46,"title":["Machine learning models predicting multidrug resistant urinary tract infections using \u201cDsaaS\u201d"],"prefix":"10.1186","volume":"21","author":[{"given":"Alessio","family":"Mancini","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leonardo","family":"Vito","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elisa","family":"Marcelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marco","family":"Piangerelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Renato","family":"De Leone","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sandra","family":"Pucciarelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emanuela","family":"Merelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,8,25]]},"reference":[{"key":"3566_CR1","first-page":"43","volume":"9","author":"A Mancini","year":"2019","unstructured":"Mancini A, Pucciarelli S, Lombardi FE, Barocci S, Pauri P, Lodolini S. Differences between community- and hospital-acquired urinary tract infections in a tertiary care hospital. New Microbiol. 2019;9:43 [1]:[Epub ahead of print]. PMID: 31814033.","journal-title":"New Microbiol"},{"key":"3566_CR2","first-page":"978","volume":"5","author":"ML Tlachac","year":"2018","unstructured":"Tlachac ML, Rundensteiner E, Barton K, Troppy S, Beaulac K, Doron S. Predicting future antibiotic susceptibility using regression-based methods on longitudinal Massachusetts Antibiogram data. Biostec. 2018;5:978\u201389.","journal-title":"Biostec"},{"issue":"10","key":"3566_CR3","doi-asserted-by":"publisher","first-page":"e51","DOI":"10.1093\/cid\/ciw118","volume":"62","author":"TF Barlam","year":"2016","unstructured":"Barlam TF, Cosgrove SE, Abbo LM, Macdougall C, Schuetz AN, Septimus EJ, et al. Implementing an antibiotic stewardship program: guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis. 2016;62(10):e51\u201377.","journal-title":"Clin Infect Dis"},{"issue":"5","key":"3566_CR4","doi-asserted-by":"publisher","first-page":"576","DOI":"10.1159\/000049840","volume":"40","author":"KG Naber","year":"2015","unstructured":"Naber KG, Bergman B, Bishop MC, Bjerklund-Johansen TE, Botto H, Lobel B, et al. EAU guidelines for the management of urinary and male genital tract infections. Urinary tract infection [UTI] working Group of the Health Care Office [HCO] of the European Association of Urology [EAU]. Eur Urol. 2015;40(5):576\u201388.","journal-title":"Eur Urol"},{"issue":"2","key":"3566_CR5","doi-asserted-by":"publisher","first-page":"342","DOI":"10.3201\/eid0702.010240","volume":"7","author":"DG Maki","year":"2001","unstructured":"Maki DG, Tambyah PA. Engineering out the risk for infection with urinary catheters. Emerg Infect Dis. 2001;7(2):342\u20137.","journal-title":"Emerg Infect Dis"},{"issue":"12","key":"3566_CR6","doi-asserted-by":"publisher","first-page":"653","DOI":"10.1038\/nrurol.2010.190","volume":"7","author":"B Foxman","year":"2010","unstructured":"Foxman B. The epidemiology of urinary tract infection. Nat Rev Urol. 2010;7(12):653\u201360.","journal-title":"Nat Rev Urol"},{"issue":"1","key":"3566_CR7","first-page":"80","volume":"11","author":"HJ Woodford","year":"2011","unstructured":"Woodford HJ, George J. Diagnosis and management of urinary infections in older people. Clin Med J R Coll Phys London. 2011;11(1):80\u20133.","journal-title":"Clin Med J R Coll Phys London"},{"key":"3566_CR8","doi-asserted-by":"publisher","first-page":"175","DOI":"10.4103\/0974-2700.55329","volume":"2","author":"F Lateef","year":"2009","unstructured":"Lateef F. Hospital design for better infection control. J Emerg Trauma Shock. 2009;2:175\u20139.","journal-title":"J Emerg Trauma Shock"},{"issue":"4","key":"3566_CR9","first-page":"277","volume":"40","author":"CL Ventola","year":"2015","unstructured":"Ventola CL. The antibiotic resistance crisis: part 1: causes and threats. P T A Peer-Rev J Formul Manag. 2015;40(4):277\u201383.","journal-title":"P T A Peer-Rev J Formul Manag"},{"key":"3566_CR10","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","volume":"16","author":"NV Chawla","year":"2002","unstructured":"Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res. 2002;16:321\u201357.","journal-title":"J Artif Intell Res"},{"key":"3566_CR11","doi-asserted-by":"publisher","first-page":"988","DOI":"10.1109\/72.788640","volume":"10","author":"VN Vapnik","year":"1999","unstructured":"Vapnik VN. An overview of statistical learning theory. IEEE Trans Neural Netw. 1999;10:988\u201399.","journal-title":"IEEE Trans Neural Netw"},{"key":"3566_CR12","first-page":"1","volume-title":"CatBoost: gradient boosting with categorical features support","author":"AV Dorogush","year":"2018","unstructured":"Dorogush AV, Ershov V, Gulin A. CatBoost: gradient boosting with categorical features support; 2018. p. 1\u201311. [Section 4]. Available from: http:\/\/arxiv.org\/abs\/1810.11363."},{"key":"3566_CR13","first-page":"409","volume":"13","author":"S Haykin","year":"1994","unstructured":"Haykin S. Neural networks: a comprehensive foundation. Knowl Eng Rev. 1994;13:409\u201312.","journal-title":"Knowl Eng Rev"},{"key":"3566_CR14","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1109\/72.668881","volume":"9","author":"R De Leone","year":"1998","unstructured":"De Leone R, Capparuccia R, Merelli E. A successive overrelaxation backpropagation algorithm for neural-network training. IEEE Trans Neural Netw. 1998;9:381\u20138.","journal-title":"IEEE Trans Neural Netw"},{"key":"3566_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v028.i05","volume":"28","author":"M Kuhn","year":"2008","unstructured":"Kuhn M. Building predictive models in R using the caret package. J Stat Softw. 2008;28:1\u201326.","journal-title":"J Stat Softw"},{"key":"3566_CR16","volume-title":"Restful web services: the basics. Online artic IBM dev tech Libr","author":"A Rodriguez","year":"2008","unstructured":"Rodriguez A. Restful web services: the basics. Online artic IBM dev tech Libr; 2008."},{"key":"3566_CR17","doi-asserted-by":"crossref","unstructured":"Peng G, Ritchey NA, Casey KS, Kearns EJ, Privette JL, Saunders D, et al. Scientific stewardship in the open data and big data era - roles and responsibilities of stewards and other major product stakeholders. D-Lib Mag. 2016;22.","DOI":"10.1045\/may2016-peng"},{"issue":"5","key":"3566_CR18","first-page":"309","volume":"36","author":"CDC, NHSN","year":"2014","unstructured":"CDC, NHSN. CDC \/ NHSN surveillance definitions for specific types of infections. Surveill Defin. 2014;36(5):309\u201332.","journal-title":"Surveill Defin"},{"issue":"10 Suppl 2","key":"3566_CR19","doi-asserted-by":"publisher","first-page":"S165","DOI":"10.1016\/j.ajic.2007.10.006","volume":"35","author":"JD Siegel","year":"2007","unstructured":"Siegel JD, Rhinehart E, Jackson M, Chiarello L. Management of multidrug-resistant organisms in health care settings, 2006. Am J Infect Control. 2007;35(10 Suppl 2):S165\u201393.","journal-title":"Am J Infect Control"},{"issue":"5","key":"3566_CR20","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1038\/nrmicro3432","volume":"13","author":"AL Flores-Mireles","year":"2015","unstructured":"Flores-Mireles AL, Walker JN, Caparon M, Hultgren SJ. Urinary tract infections: epidemiology, mechanisms of infection and treatment options. Nat Rev Microbiol. 2015;13(5):269\u201384.","journal-title":"Nat Rev Microbiol"},{"key":"3566_CR21","first-page":"1","volume":"53","author":"L Scrucca","year":"2015","unstructured":"Scrucca L. GA : a package for genetic algorithms in R. J Stat Softw. 2015;53:1.","journal-title":"J Stat Softw"},{"issue":"5","key":"3566_CR22","first-page":"1","volume":"6","author":"MA Little","year":"2017","unstructured":"Little MA, Varoquaux G, Saeb S, Lonini L, Jayaraman A, Mohr DC, et al. Using and understanding cross-validation strategies. Perspectives on Saeb et al Gigascience. 2017;6(5):1\u20136.","journal-title":"Perspectives on Saeb et al Gigascience"},{"key":"3566_CR23","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-6849-3","volume-title":"Applied predictive modeling. Applied predictive modeling","author":"M Kuhn","year":"2013","unstructured":"Kuhn M, Johnson K. Applied predictive modeling. Applied predictive modeling; 2013. Springer ed."},{"key":"3566_CR24","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1109\/TNB.2007.897492","volume":"6","author":"E Bartocci","year":"2007","unstructured":"Bartocci E, Cacciagrano D, Cannata N, Corradini F, Merelli E, Milanesi L, et al. An agent-based multilayer architecture for bioinformatics grids. IEEE Transact Nanobiosci. 2007;6:142\u20138.","journal-title":"IEEE Transact Nanobiosci"},{"key":"3566_CR25","doi-asserted-by":"publisher","first-page":"392","DOI":"10.1186\/s13104-018-3482-7","volume":"11","author":"M Piangerelli","year":"2018","unstructured":"Piangerelli M, Rucco M, Tesei L, Merelli E. Topological classifier for detecting the emergence of epileptic seizures. BMC Res Notes. 2018;11:392.","journal-title":"BMC Res Notes"},{"key":"3566_CR26","volume-title":"Visualizing 2-simplex formation of metabolic reactions. Submitted to JMGM","author":"M Piangerelli","year":"2020","unstructured":"Piangerelli M, Maestri S, Merelli E. Visualizing 2-simplex formation of metabolic reactions. Submitted to JMGM; 2020."},{"issue":"Suppl 15","key":"3566_CR27","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1186\/s12859-018-2422-9","volume":"19","author":"A Mancini","year":"2018","unstructured":"Mancini A, Eyassu F, Conway M, Occhipinti A, Li\u00f2 P, Angione C, et al. CiliateGEM: an open-project and a tool for predictions of ciliate metabolic variations and experimental condition design. BMC Bioinformatics. 2018;19(Suppl 15):442.","journal-title":"BMC Bioinformatics"},{"issue":"4","key":"3566_CR28","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/s10916-017-0715-6","volume":"41","author":"HO Alanazi","year":"2017","unstructured":"Alanazi HO, Abdullah AH, Qureshi KN. A critical review for developing accurate and dynamic predictive models using machine learning methods in medicine and health care. J Med Syst. 2017;41(4):69.","journal-title":"J Med Syst"},{"issue":"4","key":"3566_CR29","doi-asserted-by":"publisher","first-page":"302","DOI":"10.1016\/j.injury.2006.01.034","volume":"37","author":"M Bhandari","year":"2006","unstructured":"Bhandari M, Giannoudis PV. Evidence-based medicine: what it is and what it is not. Injury. 2006;37(4):302\u20136.","journal-title":"Injury"},{"key":"3566_CR30","first-page":"1","volume":"1","author":"IA Scott","year":"2018","unstructured":"Scott IA. Machine learning and evidence-based medicine. Ann Intern Med. 2018;1:1.","journal-title":"Ann Intern Med"},{"key":"3566_CR31","first-page":"e0118432","volume":"1","author":"S Takaya","year":"2015","unstructured":"Takaya S, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One. 2015;1:e0118432.","journal-title":"PLoS One"},{"key":"3566_CR32","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1186\/s12864-019-6413-7","volume":"21","author":"D Chicco","year":"2020","unstructured":"Chicco D, Jurman G. The advantages of the Matthews correlation coefficient [MCC] over F1 score and accuracy in binary classification evaluation. BMC Genomics. 2020;21:6.","journal-title":"BMC Genomics"},{"key":"3566_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v049.i04","volume":"49","author":"M Austenfeld","year":"2012","unstructured":"Austenfeld M. A graphical user Interface for R in a rich client platform for ecological modeling. J Stat Softw. 2012;49:1.","journal-title":"J Stat Softw"},{"key":"3566_CR34","first-page":"625","volume":"50","author":"H Zou","year":"2010","unstructured":"Zou H, Li G. Diagnosis, prevention, and treatment of catheter-associated urinary tract infection in adults: 2009 international clinical practice guidelines from the Infectious Diseases Society of America. Chin J Infect Chemother. 2010;50:625.","journal-title":"Chin J Infect Chemother"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-03566-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-020-03566-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-03566-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,24]],"date-time":"2021-08-24T23:11:48Z","timestamp":1629846708000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-020-03566-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8]]},"references-count":34,"journal-issue":{"issue":"S10","published-print":{"date-parts":[[2020,8]]}},"alternative-id":["3566"],"URL":"https:\/\/doi.org\/10.1186\/s12859-020-03566-7","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8]]},"assertion":[{"value":"25 August 2020","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and\/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"All patients provided written informed consent to participate once hospitalized. For the sake of privacy, data were de-identified before analysis. The manuscript does not contain any individual\u2019s data in any form.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"All authors have read and approved the final manuscript, and none of them have financial or competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"347"}}