{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T20:12:25Z","timestamp":1790107945357,"version":"4.0.1"},"reference-count":20,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2025,7,11]],"date-time":"2025-07-11T00:00:00Z","timestamp":1752192000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,7,11]],"date-time":"2025-07-11T00:00:00Z","timestamp":1752192000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007778","name":"Aegean University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007778","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Data Sci Anal"],"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Patients with diabetes struggle to control blood glucose with appropriate insulin doses. Classic prediction methods such as multi-linear regression often lead to ineffective blood\u2019s glucose control, because they are not capable to identify the correct neighborhood (other similar patients) of the target patient, since they do not take under consideration demographic (age, gender, etc.) or other clinical similarities of patients\u2019 therapeutics (blood glucose levels, etc.). In this paper, we use deep reinforcement learning (DRL), and define a transition probability function to intelligently select the right neighbors of the target patient. We also define a reward and policy function for our DRL method, which can predict the optimal insulin dose for a patient and achieves optimal glucose control, while decreasing hypoglycemia risk. Our experimental results show that our Deep Q Network (DQN) algorithm can assist doctors to administer optimal insulin dose for diabetes treatment and keep the glucose levels in blood for longer time periods in accepted values range.<\/jats:p>","DOI":"10.1007\/s41060-025-00857-1","type":"journal-article","created":{"date-parts":[[2025,7,11]],"date-time":"2025-07-11T14:27:10Z","timestamp":1752244030000},"page":"6841-6854","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Deep reinforcement learning for personalized insulin dosing and glucose control of hospitalized in ICU patients"],"prefix":"10.1007","volume":"20","author":[{"given":"Panagiotis","family":"Symeonidis","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Evangelos","family":"Rizos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christos","family":"Andras","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stergios","family":"Hairistanidis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yannis","family":"Manolopoulos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"Zanker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,7,11]]},"reference":[{"key":"857_CR1","unstructured":"Diabetes teaching center: Calculating insulin dose, https:\/\/dtc.ucsf.edu\/. Technical report, University of California, San Francisco (2023)"},{"key":"857_CR2","doi-asserted-by":"publisher","first-page":"15245","DOI":"10.1038\/s41598-024-62912-4","volume":"14","author":"S Ahmad","year":"2024","unstructured":"Ahmad, S., Beneyto, A., Zhu, T., Contreras, I., Georgiou, P., Vehi, J.: An automatic deep reinforcement learning bolus calculator for automated insulin delivery systems. Sci. Rep. 14, 15245 (2024)","journal-title":"Sci. Rep."},{"key":"857_CR3","unstructured":"Andersen, K. E., H\u00f8jbjerre, M.: A Bayesian approach to Bergman\u2019s minimal model. In Proceedings of the ... Aalborg University (2000)"},{"issue":"12","key":"857_CR4","doi-asserted-by":"publisher","first-page":"1512","DOI":"10.2337\/diab.38.12.1512","volume":"38","author":"RN Bergman","year":"1989","unstructured":"Bergman, R.N.: Lilly lecture 1989: toward physiological understanding of glucose tolerance: minimal-model approach. Diabetes 38(12), 1512\u20131527 (1989)","journal-title":"Diabetes"},{"key":"857_CR5","doi-asserted-by":"publisher","DOI":"10.3389\/fendo.2020.583016","volume":"11","author":"RN Bergman","year":"2021","unstructured":"Bergman, R.N.: Origins and history of the minimal model of glucose regulation. Front. Endocrinol. 11, 583016 (2021)","journal-title":"Front. Endocrinol."},{"issue":"6","key":"857_CR6","doi-asserted-by":"publisher","first-page":"E667","DOI":"10.1152\/ajpendo.1979.236.6.E667","volume":"236","author":"RN Bergman","year":"1979","unstructured":"Bergman, R.N., Ider, Y.Z., Bowden, C.R., Cobelli, C.: Quantitative estimation of insulin sensitivity. Am. J. Physiol.-Endocrinol. Metab. 236(6), E667 (1979)","journal-title":"Am. J. Physiol.-Endocrinol. Metab."},{"issue":"11","key":"857_CR7","first-page":"1271","volume":"47","author":"J Boh\u00e9","year":"2021","unstructured":"Boh\u00e9, J., Abidi, H., Brunot, V., Klich, A., Klouche, K., Sedillot, N., Tchenio, X., et al.: Individualised versus conventional glucose control in critically-ill patients: the controling study\u2014a randomized clinical trial. Intensive Care Med. 47(11), 1271\u20131283 (2021)","journal-title":"Intensive Care Med."},{"key":"857_CR8","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2022.104492","volume":"81","author":"S Del Giorno","year":"2023","unstructured":"Del Giorno, S., D\u2019Antoni, F., Piemonte, V., Merone, M.: A new glycemic closed-loop control based on dyna-q for type-1-diabetes. Biomed. Signal Process. Control 81, 104492 (2023)","journal-title":"Biomed. Signal Process. Control"},{"issue":"12","key":"857_CR9","doi-asserted-by":"publisher","first-page":"3001","DOI":"10.1097\/CCM.0b013e3181b083f7","volume":"37","author":"M Falciglia","year":"2009","unstructured":"Falciglia, M., Freyberg, R.W., Almenoff, P.L., Dalessio, D.A., Render, M.L.: Hyperglycemia-related mortality in critically ill patients varies with admission diagnosis. Crit. Care Med. 37(12), 3001\u20133009 (2009)","journal-title":"Crit. Care Med."},{"key":"857_CR10","doi-asserted-by":"crossref","unstructured":"Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., et\u00a0al.: Deep Q-learning from demonstrations. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume\u00a032 (2018)","DOI":"10.1609\/aaai.v32i1.11757"},{"key":"857_CR11","doi-asserted-by":"crossref","unstructured":"Lin, L.-J., Mitchell, T. M.: Reinforcement learning with hidden states. In: Proceedings of the 2nd International Conference on From Animals to Animats 2: Simulation of Adaptive Behavior: Simulation of Adaptive Behavior, pp. 271\u2013280 (1993)","DOI":"10.7551\/mitpress\/3116.003.0038"},{"key":"857_CR12","doi-asserted-by":"crossref","unstructured":"Liu, X., Jankovic, I., Chen, J. H.: Predicting inpatient glucose levels and insulin dosing by machine learning on electronic health records. medRxiv (2020)","DOI":"10.1101\/2020.03.02.20029017"},{"key":"857_CR13","unstructured":"Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: Proceedings of the 33rd International Conference on Machine Learning (ICML), pp. 1928\u20131937 (2016)"},{"key":"857_CR14","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv:1312.5602 (2013)"},{"issue":"10","key":"857_CR15","doi-asserted-by":"publisher","first-page":"2212","DOI":"10.1093\/jamia\/ocab099","volume":"28","author":"M Nguyen","year":"2021","unstructured":"Nguyen, M., Jankovic, I., Kalesinskas, L., Baiocchi, M., Chen, J.H.: Machine learning for initial insulin estimation in hospitalized patients. J. Am. Med. Inform. Assoc. 28(10), 2212\u20132219 (2021)","journal-title":"J. Am. Med. Inform. Assoc."},{"issue":"5","key":"857_CR16","doi-asserted-by":"publisher","first-page":"2536","DOI":"10.1109\/JBHI.2023.3249571","volume":"27","author":"G Noaro","year":"2023","unstructured":"Noaro, G., Zhu, T., Cappon, G., Facchinetti, A., Georgiou, P.: A personalized and adaptive insulin bolus calculator based on double deep q-learning to improve type 1 diabetes management. IEEE J. Biomed. Health Inform. 27(5), 2536\u20132543 (2023)","journal-title":"IEEE J. Biomed. Health Inform."},{"issue":"7","key":"857_CR17","doi-asserted-by":"publisher","first-page":"874","DOI":"10.1164\/rccm.202202-0329OC","volume":"206","author":"A Poole","year":"2022","unstructured":"Poole, A., Finnis, M., Anstey, J., Bellomo, R., Bihari, S., Biradar, V., Doherty, S., Eastwood, G., et al.: The effect of a liberal approach to glucose control in critically ill patients with type 2 diabetes: a multicenter, parallel-group, open-label randomized clinical trial. Am. J. Respir. Crit. Care Med. 206(7), 874\u2013882 (2022)","journal-title":"Am. J. Respir. Crit. Care Med."},{"key":"857_CR18","doi-asserted-by":"crossref","unstructured":"Symeonidis, P., Chairistanidis, S., Zanker, M.: Deep reinforcement learning for medicine recommendation. In: Proceedings of the 22nd IEEE International Conference on Bioinformatics and Bioengineering (BIBE), pp. 85\u201390 (2022)","DOI":"10.1109\/BIBE55377.2022.00026"},{"key":"857_CR19","first-page":"2613","volume":"23","author":"H van Hasselt","year":"2010","unstructured":"van Hasselt, H.: Double Q-learning. Adv. Neural. Inf. Process. Syst. 23, 2613\u20132621 (2010)","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"857_CR20","unstructured":"Wang, M., Liu, M., Liu, J., Wang, S., Long, G., Qian, B.: Safe medicine recommendation via medical knowledge graph embedding. arXiv preprint arXiv:1710.05980 (2017)"}],"container-title":["International Journal of Data Science and Analytics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-025-00857-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41060-025-00857-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-025-00857-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,27]],"date-time":"2025-09-27T11:58:46Z","timestamp":1758974326000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41060-025-00857-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,11]]},"references-count":20,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,11]]}},"alternative-id":["857"],"URL":"https:\/\/doi.org\/10.1007\/s41060-025-00857-1","relation":{},"ISSN":["2364-415X","2364-4168"],"issn-type":[{"value":"2364-415X","type":"print"},{"value":"2364-4168","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,11]]},"assertion":[{"value":"24 July 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 June 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 July 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The data used in this study is from the MIMIC III database, which is publicly available. The use of this database has been approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center (Boston, MA) and the Massachusetts Institute of Technology (Cambridge, MA). All patient data in MIMIC III is de-identified in compliance with the Health Insurance Portability and Accountability Act (HIPAA).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}