{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T05:10:35Z","timestamp":1790053835959,"version":"4.0.1"},"reference-count":40,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2021,10,26]],"date-time":"2021-10-26T00:00:00Z","timestamp":1635206400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008055","name":"Fundaci\u00f3n Eugenio Rodr\u00edguez Pascual","doi-asserted-by":"publisher","award":["Development of Adaptive and Bioinspired Systems for Glycaemic Control with Continuous Subcutaneous Insulin Infusions and Continuous Glucose Monitors"],"award-info":[{"award-number":["Development of Adaptive and Bioinspired Systems for Glycaemic Control with Continuous Subcutaneous Insulin Infusions and Continuous Glucose Monitors"]}],"id":[{"id":"10.13039\/100008055","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Spanish Ministerio de Innovaci\u00f3n, Ciencia y Universidad","award":["RTI2018-095180-B-I00"],"award-info":[{"award-number":["RTI2018-095180-B-I00"]}]},{"name":"Madrid Regional Government \u2013FEDER","award":["B2017\/BMD3773 (GenObIA-CM) and Y2018\/NMT- 4668 (Micro-Stress- MAP-CM)"],"award-info":[{"award-number":["B2017\/BMD3773 (GenObIA-CM) and Y2018\/NMT- 4668 (Micro-Stress- MAP-CM)"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This article proposes two ensemble neural network-based models for blood glucose prediction at three different prediction horizons\u201430, 60, and 120 min\u2014and compares their performance with ten recently proposed neural networks. The twelve models\u2019 performances are evaluated under the same OhioT1DM Dataset, preprocessing workflow, and tools at the three prediction horizons using the most common metrics in blood glucose prediction, and we rank the best-performing ones using three methods devised for the statistical comparison of the performance of multiple algorithms: scmamp, model confidence set, and superior predictive ability. Our analysis provides a comparison of the state-of-the-art neural networks for blood glucose prediction, estimating the model\u2019s error, highlighting those with the highest probability of being the best predictors, and providing a guide for their use in clinical practice.<\/jats:p>","DOI":"10.3390\/s21217090","type":"journal-article","created":{"date-parts":[[2021,10,26]],"date-time":"2021-10-26T23:54:33Z","timestamp":1635292473000},"page":"7090","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["Ensemble Models of Cutting-Edge Deep Neural Networks for Blood Glucose Prediction in Patients with Diabetes"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7616-4963","authenticated-orcid":false,"given":"F\u00e9lix","family":"Tena","sequence":"first","affiliation":[{"name":"Department of Computer Architecture, Facultad de Inform\u00e1tica, Universidad Complutense de Madrid, 28040 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5064-2587","authenticated-orcid":false,"given":"Oscar","family":"Garnica","sequence":"additional","affiliation":[{"name":"Department of Computer Architecture, Facultad de Inform\u00e1tica, Universidad Complutense de Madrid, 28040 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4192-1300","authenticated-orcid":false,"given":"Juan","family":"Lanchares","sequence":"additional","affiliation":[{"name":"Department of Computer Architecture, Facultad de Inform\u00e1tica, Universidad Complutense de Madrid, 28040 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3046-6368","authenticated-orcid":false,"given":"Jose Ignacio","family":"Hidalgo","sequence":"additional","affiliation":[{"name":"Department of Computer Architecture, Facultad de Inform\u00e1tica, Universidad Complutense de Madrid, 28040 Madrid, Spain"},{"name":"Instituto de Tecnolog\u00eda del Conocimiento, Universidad Complutense de Madrid, 28040 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ghosh, S., and Collier, A. (2012). Section 1\u2014Diagnosis, Classification, Epidemiology and Biochemistry, Churchill Livingstone.","DOI":"10.1016\/B978-0-443-10081-9.00008-7"},{"key":"ref_2","unstructured":"International Diabetes Federation (2019). IDF Diabetes Atlas, IDF. [9th ed.]."},{"key":"ref_3","unstructured":"Mayo Clinic (2020). Hyperglycemia in Diabetes, Mayo Clinic."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1016\/S0950-351X(05)80207-1","article-title":"Control of glycaemia","volume":"7","author":"Gerich","year":"1993","journal-title":"Baillieres Clin. Endocrinol. Metab."},{"key":"ref_5","unstructured":"Mayo Clinic (2020). Diabetic Hypoglycemia, Mayo Clinic."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hidalgo, I., Colmenar, J., Velasco, J., Kronberger, G., Winkler, S., Garnica, O., and Lanchares, J. (2018). Identification of models for glucose blood values in diabetics by grammatical evolution. Handbook of Grammatical Evolution, Springer.","DOI":"10.1007\/978-3-319-78717-6_15"},{"key":"ref_7","unstructured":"Meijner, C., and Persson, S. (2017). Blood Glucose Prediction for Type 1 Diabetes Using Machine Learning Long Short-Term Memory Based Models for Blood Glucose Prediction. [Master\u2019s Thesis, Chalmers Tekniska H\u00f6gskola]."},{"key":"ref_8","unstructured":"Wiley, M.T. (2011). Machine Learning for Diabetes Decision Support. [Ph.D. Thesis, Ohio University]."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bunescu, R., Struble, N., Marling, C., Shubrook, J., and Schwartz, F. (2013, January 4\u20137). Blood glucose level prediction using physiological models and support vector regression. Proceedings of the 2013 12th International Conference on Machine Learning and Applications, Miami, FL, USA.","DOI":"10.1109\/ICMLA.2013.30"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10916-017-0788-2","article-title":"Data based prediction of blood glucose concentrations using evolutionary methods","volume":"41","author":"Hidalgo","year":"2017","journal-title":"J. Med. Syst."},{"key":"ref_11","unstructured":"Gurney, K. (2004). An Introduction to Neural Networks, UCL Press."},{"key":"ref_12","unstructured":"Nielsen, M.A. (2020, September 01). Neural Networks and Deep Learning. Available online: http:\/\/neuralnetworksanddeeplearning.com."},{"key":"ref_13","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_14","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2020, July 01). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available online: tensorflow.org."},{"key":"ref_15","first-page":"1","article-title":"A cnn adapted to time series for the classification of supernovae","volume":"90","author":"Anthony","year":"2019","journal-title":"Electron. Imaging"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2451","DOI":"10.1162\/089976600300015015","article-title":"Learning to forget: Continual Prediction with Lstm","volume":"12","author":"Gers","year":"2000","journal-title":"Neural Comput."},{"key":"ref_18","unstructured":"Olah, C. (2020, March 01). Understanding Lstm Networks; 2015. Available online: https:\/\/colah.github.io\/posts\/2015-08-Understanding-LSTMs\/."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Mirshekarian, S., Bunescu, R., Marling, C., and Schwartz, F. (2017, January 11\u201315). Using lstms to learn physiological models of blood glucose behavior. Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju, Korea.","DOI":"10.1109\/EMBC.2017.8037460"},{"key":"ref_20","unstructured":"G\u00fclesir, M.V., der Einreichung, T., Kauschke, S., and G\u00fclesir, A.G. (2018). Predicting Blood Glucose Levels of Diabetes Patients. [Master\u2019s Thesis, TU Darmstadt]."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Sun, Q., Jankovic, M., Bally, L., and Mougiakakou, S. (2018, January 20\u201321). Predicting blood glucose with an lstm and bi-lstm based deep neural network. Proceedings of the 2018 14th Symposium on Neural Networks and Applications (NEUREL), Belgrade, Serbia.","DOI":"10.1109\/NEUREL.2018.8586990"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Idriss, T.E., Idri, A., Abnane, I., and Bakkoury, Z. (2019, January 1\u20134). Predicting blood glucose using an lstm neural network. Proceedings of the 2019 Federated Conference on Computer Science and Information Systems (FedCSIS), Leipzig, Germany.","DOI":"10.15439\/2019F159"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"103255","DOI":"10.1016\/j.engappai.2019.103255","article-title":"Engineering applications of artificial intelligence therapy-driven deep glucose forecasting","volume":"87","author":"Aiello","year":"2019","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1007\/s41666-020-00068-2","article-title":"Dilated recurrent neural networks for glucose forecasting in type 1 diabetes","volume":"4","author":"Zhu","year":"2020","journal-title":"J. Healthc. Inform. Res."},{"key":"ref_25","unstructured":"Mayo, M., and Kounty, T. (2005, January 21). Neural multi-class classification approach to blood glucose level forecasting with prediction uncertainty visualisation. Proceedings of the 5th International Workshop on Knowledge Discovery in Healthcare Data, Chicago, IL, USA."},{"key":"ref_26","first-page":"208936","article-title":"Risk analysis of blood glucose data: A quantitative approach to optimizing the control of insulin dependent diabetes","volume":"3","author":"Kovatchev","year":"2000","journal-title":"J. Theor. Med."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mu\u00f1oz-Organero, M. (2020). Deep physiological model for blood glucose prediction in t1dm patients. Sensors, 20.","DOI":"10.3390\/s20143896"},{"key":"ref_28","unstructured":"Khadem, H., Nemat, H., Elliott, J., and Benaissa, M. (2020, January 29\u201330). Multi-Lag Stacking for Blood Glucose Level Prediction; Conference on Blood Glucose Level Prediction Challenge 2020. Proceedings of the 5th International Workshop on Knowledge Discovery in Healthcare Data Co-Located with 24th European Conference on Artificial Intelligence (ECAI 2020), Santiago de Compostela, Spain."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Abdi, H., and Williams, L.J. (2013). Partial Least Squares Methods: Partial Least Squares Correlation and Partial Least Square Regression, Humana Press.","DOI":"10.1007\/978-1-62703-059-5_23"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Dietterich, T.G. (2000). Ensemble methods in machine learning. Proceedings of the First International Workshop on Multiple Classifier Systems, Cagliari, Italy, 21\u201323 June 2000, Springer.","DOI":"10.1007\/3-540-45014-9_1"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kotu, V., and Deshpande, B. (2015). Chapter 2\u2014data mining process. Predictive Analytics and Data Mining, Morgan Kaufmann.","DOI":"10.1016\/B978-0-12-801460-8.00002-1"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Calvo, B., Ceberio, J., and Lozano, J.A. (2018). Bayesian inference for algorithm ranking analysis. Proceedings of the Genetic and Evolutionary Computation Conference Companion, Association for Computing Machinery.","DOI":"10.1145\/3205651.3205658"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"248","DOI":"10.32614\/RJ-2016-017","article-title":"scmamp: Statistical Comparison of Multiple Algorithms in Multiple Problems","volume":"8","author":"Calvo","year":"2016","journal-title":"R J."},{"key":"ref_34","first-page":"71","article-title":"The Ohiot1dm Dataset for Blood Glucose Level Prediction: Update 2020","volume":"2675","author":"Marling","year":"2020","journal-title":"CEUR Work. Proc."},{"key":"ref_35","unstructured":"Kingma, D., and Ba, J. (2014, January 14\u201316). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.2337\/dc17-1600","article-title":"International consensus on use of continuous glucose monitoring","volume":"40","author":"Danne","year":"2017","journal-title":"Diabetes Care"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1143","DOI":"10.2337\/diacare.23.8.1143","article-title":"A new consensus error grid to evaluate the clinical significance of inaccuracies in the measurement of blood glucose","volume":"23","author":"Parkes","year":"2000","journal-title":"Diabetes Care"},{"key":"ref_38","unstructured":"Hansen, P.R., Lunde, A., and Nason, J.M. (2005). Model Confidence Sets for Forecasting Models, Federal Reserve Bank of Atlanta. Working Paper 2005-07."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1198\/073500105000000063","article-title":"A test for superior predictive ability","volume":"23","author":"Hansen","year":"2005","journal-title":"J. Bus. Econ. Stat."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1097","DOI":"10.1111\/1468-0262.00152","article-title":"A reality check for data snooping","volume":"68","author":"White","year":"2000","journal-title":"Econometrica"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/21\/7090\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:23:52Z","timestamp":1760167432000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/21\/7090"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,26]]},"references-count":40,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["s21217090"],"URL":"https:\/\/doi.org\/10.3390\/s21217090","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,26]]}}}