{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T04:34:15Z","timestamp":1777610055159,"version":"3.51.4"},"reference-count":40,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,3,1]],"date-time":"2023-03-01T00:00:00Z","timestamp":1677628800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Sen. Netw."],"published-print":{"date-parts":[[2023,8,31]]},"abstract":"<jats:p>As a critical branch of the Internet of Things (IoT) in the medicine industry, the Internet of Medical Things (IoMT) significantly improves the quality of healthcare due to its real-time monitoring and low medical cost. Benefiting from edge and cloud computing, IoMT is provided with more computing and storage resources near the terminal to meet the low-delay requirements of computation-intensive services. However, the service offloading from health monitoring units (HMUs) to edge servers generates additional energy consumption. Fortunately, artificial intelligence (AI), which has developed rapidly in recent years, has proved effective in some resource allocation applications. Taking both energy consumption and delay into account, we propose an energy-aware service offloading algorithm under an end-edge-cloud collaborative IoMT system with Asynchronous Advantage Actor-critic (A3C), named ECAC. Technically, ECAC uses the structural similarity between the natural distributed IoMT system and A3C, whose parameters are asynchronously updated. Besides, due to the typical delay-sensitivity mechanism and time-energy correction, ECAC can adjust dynamically to the diverse service types and system requirements. Finally, the effectiveness of ECAC for IoMT is proved on real data.<\/jats:p>","DOI":"10.1145\/3560265","type":"journal-article","created":{"date-parts":[[2022,8,29]],"date-time":"2022-08-29T12:03:40Z","timestamp":1661774620000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["Energy-saving Service Offloading for the Internet of Medical Things Using Deep Reinforcement Learning"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7191-8674","authenticated-orcid":false,"given":"Jielin","family":"Jiang","sequence":"first","affiliation":[{"name":"School of Computer and Software, Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, China, and The State Key Lab. for Novel Software Technology, Nanjing University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5305-3032","authenticated-orcid":false,"given":"Jiajie","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Computer and Software, Nanjing University of Information Science and Technology, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7656-0184","authenticated-orcid":false,"given":"Maqbool","family":"Khan","sequence":"additional","affiliation":[{"name":"Software Competence Center Hagenberg GmbH, Softwarepark, Austria, and SPCAI, Pak-Austria Fachhochschule-Institute of Applied Sciences and Technology, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9883-2750","authenticated-orcid":false,"given":"Yan","family":"Cui","sequence":"additional","affiliation":[{"name":"College of Mathematics and Information Science, Nanjing Normal University of Special Education, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2423-7542","authenticated-orcid":false,"given":"Wenmin","family":"Lin","sequence":"additional","affiliation":[{"name":"Institute of VR and Intelligent System, Alibaba Business School, Hangzhou Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,3]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCECE.2015.7129344"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2019.12.030"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-017-0015-z"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2743240"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/MWC.2019.1800411"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2018.2849014"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2020.3045653"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2018.2878876"},{"key":"e_1_3_1_10_2","first-page":"1123","article-title":"Distributed deep learning-based offloading for mobile edge computing networks","author":"Huang Liang","year":"2018","unstructured":"Liang Huang, Xu Feng, Anqi Feng, Yupin Huang, and Li Ping Qian. 2018. Distributed deep learning-based offloading for mobile edge computing networks. Mobile Networks and Applications 27 (2018), 1123\u20131130.","journal-title":"Mobile Networks and Applications"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dcan.2018.10.003"},{"key":"e_1_3_1_12_2","first-page":"307","volume-title":"Medical Imaging 2018: Computer-Aided Diagnosis","author":"Jnawali Kamal","year":"2018","unstructured":"Kamal Jnawali, Mohammad R. Arbabshirani, Navalgund Rao, and Alpen A. Patel. 2018. Deep 3D convolution neural network for CT brain hemorrhage classification. In Medical Imaging 2018: Computer-Aided Diagnosis, Vol. 10575. International Society for Optics and Photonics, SPIE, 307\u2013313."},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.5555\/168871"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2019.2935450"},{"key":"e_1_3_1_15_2","unstructured":"Volodymyr Mnih Koray Kavukcuoglu David Silver Alex Graves Ioannis Antonoglou Daan Wierstra and Martin Riedmiller. 2013. Playing Atari with deep reinforcement learning. Article arXiv:1312.5602 (2013). arxiv:1312.5602 arXiv:1312.5602"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2021.3058953"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2020.3020645"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2017162664"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2020.05.048"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2019.2904244"},{"key":"e_1_3_1_21_2","volume-title":"The RTT Distribution of TCP Flows on the Internet and Its Impact on TCP Based Flow Control","author":"Srinivas Shakkottai, Nevil Brownlee, Andre Broido, and Kc Claffy","year":"2004","unstructured":"Srinivas Shakkottai, Nevil Brownlee, Andre Broido, and Kc Claffy. 2004. The RTT Distribution of TCP Flows on the Internet and Its Impact on TCP Based Flow Control. Technical Report. Citeseer."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2019.2895593"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2018.2867191"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/LWC.2017.2740927"},{"key":"e_1_3_1_25_2","first-page":"1","article-title":"DIMA: Distributed cooperative microservice caching for Internet of Things in edge computing by deep reinforcement learning","author":"Tian Hao","year":"2021","unstructured":"Hao Tian, Xiaolong Xu, Tingyu Lin, Yong Cheng, Cheng Qian, Lei Ren, and Muhammad Bilal. 2021. DIMA: Distributed cooperative microservice caching for Internet of Things in edge computing by deep reinforcement learning. World Wide Web 24 (2021), 1\u201324.","journal-title":"World Wide Web"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-019-03011-4"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2014.6736752"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2020.05.042"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2020.2970550"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2018.2883991"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3408319"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2020.3040180"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/SmartIoT.2019.00074"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2016.2633522"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11036-020-01584-6"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2017.2786343"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2019.2947490"},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1109\/RNDM.2016.7608300","volume-title":"2016 8th International Workshop on Resilient Networks Design and Modeling (RNDM\u201916)","author":"Zhang Ke","year":"2016","unstructured":"Ke Zhang, Yuming Mao, Supeng Leng, Alexey Vinel, and Yan Zhang. 2016. Delay constrained offloading for mobile edge computing in cloud-enabled vehicular networks. In 2016 8th International Workshop on Resilient Networks Design and Modeling (RNDM\u201916). IEEE, 288\u2013294."},{"key":"e_1_3_1_39_2","unstructured":"Yunzhi Zhang Ignasi Clavera Boren Tsai and Pieter Abbeel. 2019. Asynchronous methods for model-based reinforcement learning. Article arXiv:1910.12453 (2019). arxiv:1910.12453 arXiv:1910.12453"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2963051"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2019.2905432"}],"container-title":["ACM Transactions on Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3560265","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3560265","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:00:33Z","timestamp":1750186833000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3560265"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3]]},"references-count":40,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,8,31]]}},"alternative-id":["10.1145\/3560265"],"URL":"https:\/\/doi.org\/10.1145\/3560265","relation":{},"ISSN":["1550-4859","1550-4867"],"issn-type":[{"value":"1550-4859","type":"print"},{"value":"1550-4867","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3]]},"assertion":[{"value":"2021-10-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-08-24","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-03-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}