{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T20:53:21Z","timestamp":1793393601180,"version":"4.5.0"},"reference-count":37,"publisher":"Public Library of Science (PLoS)","issue":"1","license":[{"start":{"date-parts":[[2025,1,27]],"date-time":"2025-01-27T00:00:00Z","timestamp":1737936000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004515","name":"Universiti Kebangsaan Malaysia","doi-asserted-by":"publisher","award":["GGPM-2022-063"],"award-info":[{"award-number":["GGPM-2022-063"]}],"id":[{"id":"10.13039\/501100004515","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Patients with type 1 diabetes and their physicians have long desired a fully closed-loop artificial pancreas (AP) system that can alleviate the burden of blood glucose regulation. Although deep reinforcement learning (DRL) methods theoretically enable adaptive insulin dosing control, they face numerous challenges, including safety and training efficiency, which have hindered their clinical application. This paper proposes a safe and efficient adaptive insulin delivery controller based on DRL. It employed ten tricks to enhance the proximal policy optimization (PPO) algorithm, improving training efficiency. Additionally, a dual safety mechanism of \u2019proactive guidance + reactive correction\u2019 was introduced to reduce the risks of hyperglycemia and hypoglycemia and to prevent emergencies. Performance evaluations in the Simglucose simulator demonstrate that the proposed controller achieved an 87.45% time in range (TIR) median, superior to baseline methods, with a lower incidence of hypoglycemia, notably eliminating severe hypoglycemia and treatment failures. These encouraging results indicate that the DRL-based fully closed-loop AP controller has taken an essential step toward clinical implementation.<\/jats:p>","DOI":"10.1371\/journal.pone.0317662","type":"journal-article","created":{"date-parts":[[2025,1,27]],"date-time":"2025-01-27T18:22:47Z","timestamp":1738002167000},"page":"e0317662","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":19,"title":["A safe-enhanced fully closed-loop artificial pancreas controller based on deep reinforcement learning"],"prefix":"10.1371","volume":"20","author":[{"given":"Yan Feng","family":"Zhao","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6839-0784","authenticated-orcid":true,"given":"Jun Kit","family":"Chaw","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5316-3936","authenticated-orcid":true,"given":"Mei Choo","family":"Ang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiqi","family":"Tew","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiao Yang","family":"Shi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiang","family":"Cheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,1,27]]},"reference":[{"issue":"4","key":"pone.0317662.ref001","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.1109\/JBHI.2020.3014556","article-title":"Basal glucose control in type 1 diabetes using deep reinforcement learning: An in silico validation","volume":"25","author":"T. Zhu","year":"2021","journal-title":"IEEE J. Biomed. Heal. Informatics"},{"issue":"1","key":"pone.0317662.ref002","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1109\/MCS.2017.2767119","article-title":"Overnight hypoglycemia and hyperglycemia mitigation for individuals with type 1 diabetes: How risks can be reduced","volume":"38","author":"B. W. Bequette","year":"2018","journal-title":"IEEE Control Syst."},{"key":"pone.0317662.ref003","doi-asserted-by":"crossref","first-page":"101603","DOI":"10.1016\/j.bspc.2019.101603","article-title":"Automatic blood glucose control for type 1 diabetes: A trade-off between postprandial hyperglycemia and hypoglycemia","volume":"54","author":"S. Khodakaramzadeh","year":"2019","journal-title":"Biomed. Signal Process. Control"},{"key":"pone.0317662.ref004","first-page":"950","article-title":"A reinforcement learning based system for blood glucose control without carbohydrate estimation in type 1 diabetes: In silico validation","author":"C. Hettiarachchi","year":"2022","journal-title":"Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS"},{"issue":"1","key":"pone.0317662.ref005","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1109\/TBME.2017.2698036","article-title":"Model free iPID control for glycemia regulation of type-1 diabetes","volume":"65","author":"T. MohammadRidha","year":"2018","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"pone.0317662.ref006","doi-asserted-by":"crossref","first-page":"100937","DOI":"10.1016\/j.arcontrol.2024.100937","article-title":"A comprehensive review of models and nonlinear control strategies for blood glucose regulation in artificial pancreas","volume":"57","author":"I. S. Mughal","year":"2024","journal-title":"Annu. Rev. Control"},{"issue":"1","key":"pone.0317662.ref007","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4103\/2230-973X.176456","article-title":"Insulin delivery methods: Past, present and future","volume":"6","author":"R. Shah","year":"2016","journal-title":"Int. J. Pharm. Investig"},{"issue":"10","key":"pone.0317662.ref008","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1089\/dia.2019.0502","article-title":"Modeling carbohydrate counting error in type 1 diabetes Management","volume":"22","author":"C. Roversi","year":"2020","journal-title":"Diabetes Technol. Ther."},{"key":"pone.0317662.ref009","doi-asserted-by":"crossref","first-page":"101836","DOI":"10.1016\/j.artmed.2020.101836","article-title":"Reinforcement learning application in diabetes blood glucose control: A systematic review","volume":"104","author":"M. Tejedor","year":"2020","journal-title":"Artif. Intell. Med."},{"issue":"1","key":"pone.0317662.ref010","doi-asserted-by":"crossref","DOI":"10.1097\/MED.0000000000000599","article-title":"Advances in newer basal and bolus insulins: impact on type 1 diabetes","volume":"28","author":"J. Vliebergh","year":"2021","journal-title":"Curr. Opin. Endocrinol. Diabetes Obes."},{"issue":"5","key":"pone.0317662.ref011","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1586\/17434440.2013.827515","article-title":"The use of reinforcement learning algorithms to meet the challenges of an artificial pancreas","volume":"10","author":"M. K. Bothe","year":"2013","journal-title":"Expert Rev. Med. Devices"},{"issue":"4","key":"pone.0317662.ref012","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.1109\/TNNLS.2017.2654539","article-title":"Safe exploration algorithms for reinforcement learning controllers","volume":"29","author":"T. Mannucci","year":"2018","journal-title":"IEEE Trans. Neural Networks Learn. Syst"},{"issue":"40","key":"pone.0317662.ref013","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.ifacol.2023.01.058","article-title":"Deep reinforcement learning for closed-loop blood glucose control: two approaches","volume":"55","author":"F. Di Felice","year":"2022","journal-title":"IFAC-PapersOnLine"},{"key":"pone.0317662.ref014","article-title":"The 37 implementation details of proximal policy optimization","volume":"2022","author":"S. Huang","year":"2023","journal-title":"ICLR Blog Track"},{"key":"pone.0317662.ref015","article-title":"Reinforcement learning for blood glucose control: challenges and opportunities","author":"I. Fox","year":"2019","journal-title":"Reinf. Learn. Real Life Work. 36 th Int. Conf. Mach. Learn."},{"key":"pone.0317662.ref016","first-page":"508","article-title":"Deep reinforcement learning for closed-Loop blood glucose control","volume":"126","author":"I. Fox","year":"2020","journal-title":"Mach. Learn. Healthc. Conf."},{"issue":"2","key":"pone.0317662.ref017","doi-asserted-by":"crossref","first-page":"536","DOI":"10.1109\/JBHI.2020.3002022","article-title":"Toward a fully automated artificial pancreas system using a bioinspired reinforcement learning design: In silico validation","volume":"25","author":"S. Lee","year":"2021","journal-title":"IEEE J. Biomed. Heal. Informatics"},{"issue":"3","key":"pone.0317662.ref018","doi-asserted-by":"crossref","first-page":"4915","DOI":"10.1109\/LRA.2021.3070252","article-title":"Recovery RL: Safe reinforcement learning with learned recovery zones","volume":"6","author":"B. Thananjeyan","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"pone.0317662.ref019","first-page":"1","article-title":"Artificial pancreas control for diabetes using TD3 deep reinforcement learning","volume":"2022","author":"A. Mackey","year":"2022","journal-title":"in 2022 33rd Irish Signals and Systems Conference, ISSC"},{"key":"pone.0317662.ref020","doi-asserted-by":"crossref","DOI":"10.1016\/j.jbi.2022.104006","article-title":"Deep neural networks for neuro-oncology: Towards patient individualized design of chemo-radiation therapy for Glioblastoma patients","volume":"127","author":"A. Ebrahimi Zade","year":"2022","journal-title":"J. Biomed. Inform."},{"key":"pone.0317662.ref021","first-page":"887","article-title":"Improving sepsis treatment strategies by combining deep and kernel-based reinforcement learning","volume":"2018","author":"X. Peng","year":"2018","journal-title":"AMIA Annu Symp Proc."},{"key":"pone.0317662.ref022","first-page":"1","article-title":"Implementation matters in deep policy gradients: A case study on PPO and TRPO","author":"L. Engstrom","year":"2020","journal-title":"8th Int. Conf. Learn. Represent. ICLR 2020"},{"key":"pone.0317662.ref023","first-page":"1386","article-title":"Improving stochastic policy gradients in continuous control with deep reinforcement learning using the beta distribution","volume":"2","author":"P. W. Chou","year":"2017","journal-title":"34th Int. Conf. Mach. Learn. ICML 2017"},{"key":"pone.0317662.ref024","first-page":"5015","article-title":"The mirage of action-dependent baselines in reinforcement learning","author":"G. Tucker","year":"2018","journal-title":"in International conference on machine learning"},{"issue":"S1","key":"pone.0317662.ref025","first-page":"2009","article-title":"Statistical tools to analyze continuous glucose monitor data","volume":"11","author":"W. Clarke","journal-title":"Diabetes Technol. Ther."},{"issue":"11","key":"pone.0317662.ref026","first-page":"1143","article-title":"\u201cGuidelines for the diagnosis and treatment of type 1 diabetes mellitus in China (2021 edition)","volume":"14","author":"C. Diabetes Society","year":"2022","journal-title":"Chinese J. Diabetes"},{"key":"pone.0317662.ref027","unstructured":"J. Xie, \u201cSimglucose v0.2.1,\u201d 2018. https:\/\/github.com\/jxx123\/simglucose"},{"key":"pone.0317662.ref028","doi-asserted-by":"crossref","first-page":"120156","DOI":"10.1016\/j.eswa.2023.120156","article-title":"ARLPE: A meta reinforcement learning framework for glucose regulation in type 1 diabetics","volume":"228","author":"X. Yu","year":"2023","journal-title":"Expert Syst. Appl."},{"issue":"3","key":"pone.0317662.ref029","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1145\/3236386.3241340","article-title":"The mythos of model interpretability","volume":"16","author":"Z. C. Lipton","year":"2018","journal-title":"Queue"},{"key":"pone.0317662.ref030","first-page":"648","article-title":"A diabetes prediction model with visualized explainable artificial intelligence (XAI) technology BT \u2010 Advances in visual informatics","volume":"23","author":"Y. Zhao","year":"2023","journal-title":"in IVIC"},{"key":"pone.0317662.ref031","doi-asserted-by":"crossref","first-page":"100290","DOI":"10.1016\/j.health.2023.100290","article-title":"A predictive analytics model using machine learning algorithms to estimate the risk of shock development among dengue patients","volume":"5","author":"J. Chaw","year":"2023","journal-title":"Healthc. Anal."},{"key":"pone.0317662.ref032","doi-asserted-by":"crossref","first-page":"105756","DOI":"10.1109\/ACCESS.2021.3100007","article-title":"A Blood Glucose Control Framework Based on Reinforcement Learning with Safety and Interpretability: In Silico Validation","volume":"9","author":"M. H. Lim","year":"2021","journal-title":"IEEE Access"},{"key":"pone.0317662.ref033","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.inffus.2023.03.008","article-title":"A systematic review of trustworthy and explainable artificial intelligence in healthcare: Assessment of quality, bias risk, and data fusion","volume":"96","author":"A. S. Albahri","year":"2023","journal-title":"Inf. Fusion"},{"key":"pone.0317662.ref034","doi-asserted-by":"crossref","first-page":"102901","DOI":"10.1016\/j.artmed.2024.102901","article-title":"Systematic literature review on reinforcement learning in non-communicable disease interventions","volume":"154","author":"Y. Zhao","year":"2024","journal-title":"Artif. Intell. Med."},{"key":"pone.0317662.ref035","first-page":"4673","article-title":"Deep offline reinforcement learning for real-world treatment optimization applications","author":"M. Nambiar","year":"2023","journal-title":"Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min."},{"key":"pone.0317662.ref036","doi-asserted-by":"crossref","first-page":"104376","DOI":"10.1016\/j.jbi.2023.104376","article-title":"Offline reinforcement learning for safer blood glucose control in people with type 1 diabetes","volume":"142","author":"H. Emerson","year":"2023","journal-title":"J. Biomed. Inform."},{"key":"pone.0317662.ref037","doi-asserted-by":"crossref","first-page":"105839","DOI":"10.1016\/j.bspc.2023.105839","article-title":"G2P2C \u2014A modular reinforcement learning algorithm for glucose control by glucose prediction and planning in Type 1 Diabetes","volume":"90","author":"C. Hettiarachchi","year":"2023","journal-title":"Biomed. Signal Process. Control"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0317662","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,27]],"date-time":"2025-01-27T18:22:59Z","timestamp":1738002179000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0317662"}},"subtitle":[],"editor":[{"given":"Zeashan Hameed","family":"Khan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,1,27]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1,27]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0317662","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,27]]}}}