{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T05:37:53Z","timestamp":1781242673194,"version":"3.54.1"},"reference-count":55,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2021,4,19]],"date-time":"2021-04-19T00:00:00Z","timestamp":1618790400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Commonwealth Industrial and Scientific Research Organisation"},{"DOI":"10.13039\/100015539","name":"Australian government","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100015539","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Australian Government Research Training Program scholarship"},{"DOI":"10.13039\/501100000943","name":"CSIRO","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000943","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,7,30]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>Glycemic control is an important component of critical care. We present a data-driven method for predicting intensive care unit (ICU) patient response to glycemic control protocols while accounting for patient heterogeneity and variations in care.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>Using electronic medical records (EMRs) of 18\u00a0961 ICU admissions from the MIMIC-III dataset, including 318\u00a0574 blood glucose measurements, we train and validate a gradient boosted tree machine learning (ML) algorithm to forecast patient blood glucose and a 95% prediction interval at 2-hour intervals. The model uses as inputs irregular multivariate time series data relating to recent in-patient medical history and glycemic control, including previous blood glucose, nutrition, and insulin dosing.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Our forecasting model using routinely collected EMRs achieves performance comparable to previous models developed in planned research studies using continuous blood glucose monitoring. Model error, expressed as mean absolute percentage error is 16.5%\u201316.8%, with Clarke error grid analysis demonstrating that 97% of predictions would be clinically acceptable. The 95% prediction intervals achieve near intended coverage at 93%\u201394%.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>ML algorithms built on observational data sources, such as EMRs, present a promising approach for personalization and automation of glycemic control in critical care. Future research may benefit from applying a combination of methodologies and data sources to develop robust methodologies that account for the variations seen in ICU patients and difficultly in detecting the extremes of observed blood glucose values.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>We demonstrate that EMRs can be used to train ML algorithms that may be suitable for incorporation into ICU decision support systems.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocab060","type":"journal-article","created":{"date-parts":[[2021,3,22]],"date-time":"2021-03-22T12:17:12Z","timestamp":1616415432000},"page":"1642-1650","source":"Crossref","is-referenced-by-count":25,"title":["Incorporating real-world evidence into the development of patient blood glucose prediction algorithms for the ICU"],"prefix":"10.1093","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9554-8869","authenticated-orcid":false,"given":"Oisin","family":"Fitzgerald","sequence":"first","affiliation":[{"name":"Centre for Big Data Research in Health, UNSW Sydney, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8823-7090","authenticated-orcid":false,"given":"Oscar","family":"Perez-Concha","sequence":"additional","affiliation":[{"name":"Centre for Big Data Research in Health, UNSW Sydney, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3704-7975","authenticated-orcid":false,"given":"Blanca","family":"Gallego","sequence":"additional","affiliation":[{"name":"Centre for Big Data Research in Health, UNSW Sydney, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manoj K","family":"Saxena","sequence":"additional","affiliation":[{"name":"The George Institute for Global Health, UNSW Sydney, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lachlan","family":"Rudd","sequence":"additional","affiliation":[{"name":"Data and Analytics, eHealth NSW, Chatswood, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alejandro","family":"Metke-Jimenez","sequence":"additional","affiliation":[{"name":"Australian e-Health Research Centre, CSIRO, Brisbane, QLD, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0390-661X","authenticated-orcid":false,"given":"Louisa","family":"Jorm","sequence":"additional","affiliation":[{"name":"Centre for Big Data Research in Health, UNSW Sydney, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,4,19]]},"reference":[{"issue":"Suppl 3","key":"2021073020282158300_ocab060-B1","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1186\/s13054-017-1910-z","article-title":"The coming era of precision medicine for intensive care","volume":"21","author":"Vincent","year":"2017","journal-title":"Crit Care"},{"issue":"1","key":"2021073020282158300_ocab060-B2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13054-017-1653-x","article-title":"A path to precision in the ICU","volume":"21","author":"Maslove","year":"2017","journal-title":"Crit Care"},{"issue":"1","key":"2021073020282158300_ocab060-B3","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1186\/s13054-017-1836-5","article-title":"Precision medicine for all? Challenges and opportunities for a precision medicine approach to critical illness","volume":"21","author":"Seymour","year":"2017","journal-title":"Crit Care"},{"issue":"10141","key":"2021073020282158300_ocab060-B4","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/S0140-6736(18)30696-2","article-title":"Sepsis and septic shock","volume":"392","author":"Cecconi","year":"2018","journal-title":"Lancet"},{"issue":"6","key":"2021073020282158300_ocab060-B5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3127881","article-title":"Mining electronic health records (EHRs): a survey","volume":"50","author":"Yadav","year":"2018","journal-title":"ACM Comput Surv"},{"issue":"2","key":"2021073020282158300_ocab060-B6","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1109\/JPROC.2015.2501978","article-title":"Machine learning and decision support in critical care","volume":"104","author":"Johnson","year":"2016","journal-title":"Proc IEEE Inst Electr Electron Eng"},{"issue":"11","key":"2021073020282158300_ocab060-B7","doi-asserted-by":"crossref","first-page":"1716","DOI":"10.1038\/s41591-018-0213-5","article-title":"The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care","volume":"24","author":"Komorowski","year":"2018","journal-title":"Nat Med"},{"issue":"3","key":"2021073020282158300_ocab060-B8","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1007\/s00134-017-4683-6","article-title":"Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016","volume":"43","author":"Rhodes","year":"2017","journal-title":"Intensive Care Med"},{"issue":"1","key":"2021073020282158300_ocab060-B9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-020-67864-z","article-title":"Real-world characterization of blood glucose control and insulin use in the intensive care unit","volume":"10","author":"Baker","year":"2020","journal-title":"Sci Rep"},{"issue":"4","key":"2021073020282158300_ocab060-B10","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1016\/j.bpa.2009.08.005","article-title":"Hyperglycaemia as part of the stress response: the underlying mechanisms","volume":"23","author":"Van Cromphaut","year":"2009","journal-title":"Best Pract Res Clin Anaesthesiol"},{"issue":"2","key":"2021073020282158300_ocab060-B11","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1097\/CCM.0b013e318194b097","article-title":"Early blood glucose control and mortality in critically ill patients in Australia","volume":"37","author":"Bagshaw","year":"2009","journal-title":"Crit Care Med"},{"issue":"9677","key":"2021073020282158300_ocab060-B12","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1016\/S0140-6736(09)60553-5","article-title":"Stress hyperglycaemia","volume":"373","author":"Dungan","year":"2009","journal-title":"Lancet"},{"issue":"6","key":"2021073020282158300_ocab060-B13","doi-asserted-by":"crossref","first-page":"1225","DOI":"10.1097\/01.CCM.0000165962.16682.46","article-title":"Give your patient a fast hug (at least) once a day","volume":"33","author":"Vincent","year":"2005","journal-title":"Crit Care Med"},{"issue":"2","key":"2021073020282158300_ocab060-B14","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1177\/1062860613489339","article-title":"Inpatient glycemic control: best practice advice from the Clinical Guidelines Committee of the American College of Physicians","volume":"29","author":"Qaseem","year":"2014","journal-title":"Am J Med Qual"},{"issue":"3","key":"2021073020282158300_ocab060-B15","doi-asserted-by":"crossref","first-page":"217","DOI":"10.4065\/mcp.2009.0394","article-title":"Egi M, Bellomo R, Stachowski E, et al. Hypoglycemia and outcome in critically ill patients","volume":"85","year":"2010","journal-title":"Mayo Clin Proc"},{"issue":"9","key":"2021073020282158300_ocab060-B16","doi-asserted-by":"crossref","first-page":"E7","DOI":"10.1002\/jhm.533","article-title":"Inpatient glucose control: a glycemic survey of 126 US hospitals","volume":"4","author":"Cook","year":"2009","journal-title":"J Hosp Med"},{"issue":"13","key":"2021073020282158300_ocab060-B17","doi-asserted-by":"crossref","first-page":"1283","DOI":"10.1056\/NEJMoa0810625","article-title":"Intensive versus conventional glucose control in critically ill patients","volume":"360","year":"2009","journal-title":"N Engl J Med"},{"issue":"19","key":"2021073020282158300_ocab060-B18","doi-asserted-by":"crossref","first-page":"1359","DOI":"10.1056\/NEJMoa011300","article-title":"Intensive insulin therapy in critically ill patients","volume":"345","author":"Van Den Berghe","year":"2001","journal-title":"N Engl J Med"},{"issue":"5","key":"2021073020282158300_ocab060-B19","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1056\/NEJMoa052521","article-title":"Intensive insulin therapy in the medical ICU","volume":"354","author":"Van den Berghe","year":"2006","journal-title":"N Engl J Med"},{"issue":"12","key":"2021073020282158300_ocab060-B20","doi-asserted-by":"crossref","first-page":"3190","DOI":"10.1097\/CCM.0b013e31818f21aa","article-title":"Intensive versus conventional insulin therapy: a randomized controlled trial in medical and surgical critically ill patients","volume":"36","author":"Arabi","year":"2008","journal-title":"Crit Care Med"},{"issue":"2","key":"2021073020282158300_ocab060-B21","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1056\/NEJMoa070716","article-title":"Intensive insulin therapy and pentastarch resuscitation in severe sepsis","volume":"358","author":"Brunkhorst","year":"2008","journal-title":"N Engl J Med"},{"issue":"5","key":"2021073020282158300_ocab060-B22","doi-asserted-by":"crossref","first-page":"R120","DOI":"10.1186\/cc7017","article-title":"Strict glycaemic control in patients hospitalised in a mixed medical and surgical intensive care unit: a randomised clinical trial","volume":"12","author":"De La Rosa","year":"2008","journal-title":"Crit Care"},{"issue":"1","key":"2021073020282158300_ocab060-B23","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1378\/chest.10-1478","article-title":"Glycemic control in the ICU","volume":"140","author":"Egi","year":"2011","journal-title":"Chest"},{"key":"2021073020282158300_ocab060-B24","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1016\/j.arcontrol.2019.03.007","article-title":"Glycemic control in the intensive care unit: a control systems perspective","volume":"48","author":"Chase","year":"2019","journal-title":"Annu Rev Control"},{"issue":"4","key":"2021073020282158300_ocab060-B25","doi-asserted-by":"crossref","first-page":"1005","DOI":"10.2337\/dc06-1964","article-title":"Intensive insulin therapy in critical care: a review of 12 protocols","volume":"30","author":"Wilson","year":"2007","journal-title":"Diabetes Care"},{"issue":"2","key":"2021073020282158300_ocab060-B26","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1177\/0148607115621863","article-title":"Guidelines for the provision and assessment of nutrition support therapy in the adult critically ill patient: Society of Critical Care Medicine (SCCM) and American Society for Parenteral and Enteral Nutrition (ASPEN)","volume":"40","author":"McClave","year":"2016","journal-title":"JPEN J Parenter Enteral Nutr"},{"issue":"5","key":"2021073020282158300_ocab060-B27","doi-asserted-by":"crossref","first-page":"R163","DOI":"10.1186\/cc8129","article-title":"Computerized intensive insulin dosing can mitigate hypoglycemia and achieve tight glycemic control when glucose measurement is performed frequently and on time","volume":"13","author":"Juneja","year":"2009","journal-title":"Crit Care"},{"issue":"4","key":"2021073020282158300_ocab060-B28","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1097\/MCO.0b013e3282fcea2a","article-title":"A. Tight glucose control in intensive care units: an update with an emphasis on nutritional issues","volume":"11","author":"Elia","year":"2008","journal-title":"Curr Opin Clin Nutr Metab Care"},{"issue":"3","key":"2021073020282158300_ocab060-B29","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1080\/21548331.2015.1066227","article-title":"Glycemic control in the critically ill: what have we learned since NICE-SUGAR?","volume":"43","author":"Krinsley","year":"2015","journal-title":"Hosp Pract (1995)"},{"issue":"1","key":"2021073020282158300_ocab060-B30","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1186\/s13054-017-1725-y","article-title":"Untangling glycaemia and mortality in critical care","volume":"21","author":"Uyttendaele","year":"2017","journal-title":"Crit Care"},{"issue":"8","key":"2021073020282158300_ocab060-B31","doi-asserted-by":"crossref","first-page":"959","DOI":"10.1088\/0967-3334\/29\/8\/008","article-title":"A simulation model of glucose regulation in the critically ill","volume":"29","author":"Hovorka","year":"2008","journal-title":"Physiol Meas"},{"issue":"2","key":"2021073020282158300_ocab060-B32","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/j.cmpb.2007.04.006","article-title":"Stochastic modelling of insulin sensitivity and adaptive glycemic control for critical care","volume":"89","author":"Lin","year":"2008","journal-title":"Comput Methods Programs Biomed"},{"issue":"1","key":"2021073020282158300_ocab060-B33","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.bspc.2006.03.002","article-title":"Model-based glycaemic control in critical care\u2014A review of the state of the possible","volume":"1","author":"Chase","year":"2006","journal-title":"Biomed Signal Process Control"},{"issue":"7","key":"2021073020282158300_ocab060-B34","doi-asserted-by":"crossref","first-page":"1543","DOI":"10.1109\/TBME.2017.2686432","article-title":"Generalisability of a virtual trials method for glycaemic control in intensive care","volume":"65","author":"Dickson","year":"2018","journal-title":"IEEE Trans Biomed Eng"},{"issue":"5","key":"2021073020282158300_ocab060-B35","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1089\/dia.2019.0252","article-title":"An artificial neural network-based predictive model to support optimization of inpatient glycemic control","volume":"22","author":"Pappada","year":"2020","journal-title":"Diabetes Technol Ther"},{"issue":"7","key":"2021073020282158300_ocab060-B36","doi-asserted-by":"crossref","first-page":"e69475","DOI":"10.1371\/journal.pone.0069475","article-title":"Evaluation of a model for glycemic prediction in critically ill surgical patients","volume":"8","author":"Pappada","year":"2013","journal-title":"PLoS One"},{"issue":"1","key":"2021073020282158300_ocab060-B37","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/1754-9493-4-15","article-title":"Development of a neural network model for predicting glucose levels in a surgical critical care setting","volume":"4","author":"Pappada","year":"2010","journal-title":"Patient Saf Surg"},{"issue":"11","key":"2021073020282158300_ocab060-B38","doi-asserted-by":"crossref","first-page":"1057","DOI":"10.1088\/0967-3334\/27\/11\/001","article-title":"An adaptive input-output modeling approach for predicting the glycemia of critically ill patients","volume":"27","author":"Van Herpe","year":"2006","journal-title":"Physiol Meas"},{"key":"2021073020282158300_ocab060-B39","first-page":"6639","article-title":"Prokhorenkova L, Gusev G, Vorobev A, Dorogush AV, Gulin A. CatBoost: unbiased boosting with categorical features","year":"2018","journal-title":"Adv Neural Inform Process Syst"},{"key":"2021073020282158300_ocab060-B40","article-title":"Learnings from Kaggle\u2019s Forecasting Competitions","author":"Bojer","year":"2020","journal-title":"Work Paper"},{"issue":"1","key":"2021073020282158300_ocab060-B41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"issue":"1","key":"2021073020282158300_ocab060-B42","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1093\/jamia\/ocw042","article-title":"Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review","volume":"24","author":"Goldstein","year":"2017","journal-title":"J Am Med Inform Assoc"},{"key":"2021073020282158300_ocab060-B43","first-page":"Xiv","article-title":"Uncertainty in gradient boosting via Ensembles","author":"Ustimenko","year":"2020","journal-title":"arXiv Preprint"},{"issue":"2","key":"2021073020282158300_ocab060-B44","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1111\/j.2517-6161.1964.tb00553.x","article-title":"An analysis of transformations","volume":"26","author":"Box","year":"1964","journal-title":"Royal Statistic\/\/\/al Society: Series B (Methodological)"},{"key":"2021073020282158300_ocab060-B45","volume-title":"R: A Language and Environment for Statistical Computing","year":"2013"},{"key":"2021073020282158300_ocab060-B46","first-page":"2","article-title":"Data. Table: Extension of data","author":"Dowle","year":"2019","journal-title":"Frame. R Package Version 1.12"},{"issue":"2","key":"2021073020282158300_ocab060-B47","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1002\/wics.147","article-title":"ggplot2","volume":"3","author":"Wickham","year":"2011","journal-title":"Wires Comp Stat"},{"issue":"5","key":"2021073020282158300_ocab060-B48","doi-asserted-by":"crossref","first-page":"622","DOI":"10.2337\/diacare.10.5.622","article-title":"Evaluating clinical accuracy of systems for self-monitoring of blood glucose","volume":"10","author":"Clarke","year":"1987","journal-title":"Diabetes Care"},{"key":"2021073020282158300_ocab060-B49","doi-asserted-by":"crossref","DOI":"10.1002\/9780470316801","volume-title":"Finding Groups in Data: An Introduction to Cluster Analysis","author":"Kaufman","year":"1990"},{"key":"2021073020282158300_ocab060-B50","first-page":"Xiv","article-title":"Delay-induced uncertainty in physiological systems","author":"Karamched","year":"2020","journal-title":"arXiv Preprint ar"},{"issue":"4","key":"2021073020282158300_ocab060-B51","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.1378\/chest.126.4.1292","article-title":"The hospital mortality of patients admitted to the ICU on weekends","volume":"126","author":"Ensminger","year":"2004","journal-title":"Chest"},{"key":"2021073020282158300_ocab060-B52","first-page":"1","article-title":"The use of continuous glucose monitoring combined with computer-based eMPC algorithm for tight glucose control in cardiosurgical ICU","volume":"2013","author":"Kopeck\u00fd","year":"2013","journal-title":"Bio Med Res Int"},{"issue":"12","key":"2021073020282158300_ocab060-B53","doi-asserted-by":"crossref","first-page":"e48058","DOI":"10.1371\/journal.pone.0048058","article-title":"Population physiology: leveraging electronic health record data to understand human endocrine dynamics","volume":"7","author":"Albers","year":"2012","journal-title":"PLoS One"},{"issue":"2","key":"2021073020282158300_ocab060-B54","doi-asserted-by":"crossref","first-page":"e97","DOI":"10.1097\/CCM.0000000000001389","article-title":"Glucose meters: here today, gone tomorrow?","volume":"44","author":"Rice","year":"2016","journal-title":"Crit Care Med"},{"issue":"1","key":"2021073020282158300_ocab060-B55","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1007\/s00134-004-2500-5","article-title":"Analysis of blood glucose measurements using capillary and arterial blood samples in intensive care patients","volume":"31","author":"Kulkarni","year":"2005","journal-title":"Intensive Care Med"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/8\/1642\/39502378\/ocab060.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/8\/1642\/39502378\/ocab060.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,26]],"date-time":"2024-08-26T14:49:25Z","timestamp":1724683765000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/8\/1642\/6237196"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,19]]},"references-count":55,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,4,19]]},"published-print":{"date-parts":[[2021,7,30]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab060","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,8,1]]},"published":{"date-parts":[[2021,4,19]]}}}