{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T15:55:01Z","timestamp":1783439701459,"version":"3.54.6"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2019,12,19]],"date-time":"2019-12-19T00:00:00Z","timestamp":1576713600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"The Opening Fund of Acoustics Science and Technology Laboratory","award":["No.SSKF2018002"],"award-info":[{"award-number":["No.SSKF2018002"]}]},{"name":"China NSFC","award":["61702364"],"award-info":[{"award-number":["61702364"]}]},{"name":"The National Key R&D Program of China","award":["2018YFC0809803,2019YFB2101901"],"award-info":[{"award-number":["2018YFC0809803,2019YFB2101901"]}]},{"name":"Huawei Innovation Research Program","award":["HO2018095224"],"award-info":[{"award-number":["HO2018095224"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Sen. Netw."],"published-print":{"date-parts":[[2020,2,29]]},"abstract":"<jats:p>With the development of the Internet of Things (IoT) and the birth of various new IoT devices, the capacity of massive IoT devices is facing challenges. Fortunately, edge computing can optimize problems such as delay and connectivity by offloading part of the computational tasks to edge nodes close to the data source. Using this feature, IoT devices can save more resources while still maintaining the quality of service. However, since computation offloading decisions concern joint and complex resource management, we use multiple Deep Reinforcement Learning (DRL) agents deployed on IoT devices to guide their own decisions. Besides, Federated Learning (FL) is utilized to train DRL agents in a distributed fashion, aiming to make the DRL-based decision making practical and further decrease the transmission cost between IoT devices and Edge Nodes. In this article, we first study the problem of computation offloading optimization and prove the problem is an NP-hard problem. Then, based on DRL and FL, we propose an offloading algorithm that is different from the traditional method. Finally, we studied the effects of various parameters on the performance of the algorithm and verified the effectiveness of both the DRL and FL in the IoT system.<\/jats:p>","DOI":"10.1145\/3372025","type":"journal-article","created":{"date-parts":[[2019,12,20]],"date-time":"2019-12-20T13:33:12Z","timestamp":1576848792000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":66,"title":["Computation Offloading with Multiple Agents in Edge-Computing\u2013Supported IoT"],"prefix":"10.1145","volume":"16","author":[{"given":"Shihao","family":"Shen","sequence":"first","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiwen","family":"Han","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaofei","family":"Wang","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yan","family":"Wang","sequence":"additional","affiliation":[{"name":"Harbin Engineering University, Harbin, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,12,19]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2017.2750180"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183338"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2017.3641638"},{"key":"e_1_2_1_4_1","volume-title":"Keras: The Python Deep Learning library. https:\/\/keras.io.","year":"2017","unstructured":"Bigmoyan. 2017 . Keras: The Python Deep Learning library. https:\/\/keras.io. Bigmoyan. 2017. Keras: The Python Deep Learning library. https:\/\/keras.io."},{"key":"e_1_2_1_5_1","volume-title":"David Petrou, Daniel Ramage, and Jason Roselander.","author":"Bonawitz Keith","year":"2019","unstructured":"Keith Bonawitz , Hubert Eichner , Wolfgang Grieskamp , Dzmitry Huba , Alex Ingerman , Vladimir Ivanov , Chloe Kiddon , Jakub Konecny , Stefano Mazzocchi , H. Brendan McMahan , Timon Van Overveldt , David Petrou, Daniel Ramage, and Jason Roselander. 2019 . Towards federated learning at scale: System design. Arxiv Preprint Arxiv :1902.01046 (2019). Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konecny, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. 2019. Towards federated learning at scale: System design. Arxiv Preprint Arxiv:1902.01046 (2019)."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2018.2815360"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2015.2487344"},{"key":"e_1_2_1_8_1","volume-title":"Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning","author":"Chen Xianfu","year":"2018","unstructured":"Xianfu Chen , Honggang Zhang , Celimuge Wu , Shiwen Mao , Yusheng Ji , and Mehdi Bennis . 2018. Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning . IEEE Internet of Things Journal ( 2018 ), 1--1. DOI:https:\/\/doi.org\/10.1109\/JIOT.2018.2876279 Xianfu Chen, Honggang Zhang, Celimuge Wu, Shiwen Mao, Yusheng Ji, and Mehdi Bennis. 2018. Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet of Things Journal (2018), 1--1. DOI:https:\/\/doi.org\/10.1109\/JIOT.2018.2876279"},{"key":"e_1_2_1_9_1","unstructured":"ETSI. 2014. Mobile-Edge Computing \u2014Introductory Technical White Paper. https:\/\/portal.etsi.org\/Portals\/0\/TBpages\/MEC\/Docs\/Mobile-edge_Computing_-_Introductory_Technical_White_Paper_V1%2018-09-14.pdf.  ETSI. 2014. Mobile-Edge Computing \u2014Introductory Technical White Paper. https:\/\/portal.etsi.org\/Portals\/0\/TBpages\/MEC\/Docs\/Mobile-edge_Computing_-_Introductory_Technical_White_Paper_V1%2018-09-14.pdf."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/322077.322090"},{"key":"e_1_2_1_11_1","first-page":"1742","article-title":"On the interplay between global DVFS and scheduling tasks with precedence constraints","volume":"64","author":"Gerards Marco E. T.","year":"2014","unstructured":"Marco E. T. Gerards , Johann L. Hurink , and Jan Kuper . 2014 . On the interplay between global DVFS and scheduling tasks with precedence constraints . IEEE Trans. Comput. 64 , 6 (2014), 1742 -- 1754 . Marco E. T. Gerards, Johann L. Hurink, and Jan Kuper. 2014. On the interplay between global DVFS and scheduling tasks with precedence constraints. IEEE Trans. Comput. 64, 6 (2014), 1742--1754.","journal-title":"IEEE Trans. Comput."},{"key":"e_1_2_1_12_1","unstructured":"Google. 2017. Deploy machine learning models on mobile and IoT devices. https:\/\/www.tensorflow.org\/lite.  Google. 2017. Deploy machine learning models on mobile and IoT devices. https:\/\/www.tensorflow.org\/lite."},{"key":"e_1_2_1_13_1","volume-title":"Federated Learning: Collaborative Machine Learning without Centralized Training Data. https:\/\/research.googleblog.com\/2017\/04\/federated-learning-collaborative.html.","year":"2017","unstructured":"Google. 2017 . Federated Learning: Collaborative Machine Learning without Centralized Training Data. https:\/\/research.googleblog.com\/2017\/04\/federated-learning-collaborative.html. Google. 2017. Federated Learning: Collaborative Machine Learning without Centralized Training Data. https:\/\/research.googleblog.com\/2017\/04\/federated-learning-collaborative.html."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098208.3098210"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8667.2012.00781.x"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2015.2437951"},{"key":"e_1_2_1_17_1","volume-title":"Singh","author":"Jaakkola Tommi","year":"1994","unstructured":"Tommi Jaakkola , Michael I. Jordan , and Satinder P . Singh . 1994 . Convergence of stochastic iterative dynamic programming algorithms. In Advances in Neural Information Processing Systems . 703--710. Tommi Jaakkola, Michael I. Jordan, and Satinder P. Singh. 1994. Convergence of stochastic iterative dynamic programming algorithms. In Advances in Neural Information Processing Systems. 703--710."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCIS.2013.468"},{"key":"e_1_2_1_20_1","volume-title":"Garey","author":"Johnson David S.","year":"1979","unstructured":"David S. Johnson and Michael R . Garey . 1979 . Computers and Intractability: A Guide to the Theory of NP-completeness. Vol. 1 . W. H. Freeman San Francisco . David S. Johnson and Michael R. Garey. 1979. Computers and Intractability: A Guide to the Theory of NP-completeness. Vol. 1. W. H. Freeman San Francisco."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICOIN.2017.7899468"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3229064"},{"key":"e_1_2_1_23_1","volume-title":"Federated optimization: Distributed machine learning for on-device intelligence. Arxiv Preprint Arxiv:1610.02527","author":"Kone\u010dn\u1ef3 Jakub","year":"2016","unstructured":"Jakub Kone\u010dn\u1ef3 , H. Brendan McMahan , Daniel Ramage , and Peter Richt\u00e1rik . 2016. Federated optimization: Distributed machine learning for on-device intelligence. Arxiv Preprint Arxiv:1610.02527 ( 2016 ). Jakub Kone\u010dn\u1ef3, H. Brendan McMahan, Daniel Ramage, and Peter Richt\u00e1rik. 2016. Federated optimization: Distributed machine learning for on-device intelligence. Arxiv Preprint Arxiv:1610.02527 (2016)."},{"key":"e_1_2_1_24_1","volume-title":"Ananda Theertha Suresh, and Dave Bacon","author":"Kone\u010dn\u1ef3 Jakub","year":"2016","unstructured":"Jakub Kone\u010dn\u1ef3 , H. Brendan McMahan , Felix X. Yu , Peter Richt\u00e1rik , Ananda Theertha Suresh, and Dave Bacon . 2016 . Federated learning: Strategies for improving communication efficiency. Arxiv Preprint Arxiv :1610.05492 (2016). Jakub Kone\u010dn\u1ef3, H. Brendan McMahan, Felix X. Yu, Peter Richt\u00e1rik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. Arxiv Preprint Arxiv:1610.05492 (2016)."},{"key":"e_1_2_1_25_1","volume-title":"Letaief","author":"Mao Yuyi","year":"2017","unstructured":"Yuyi Mao , Changsheng You , Jun Zhang , Kaibin Huang , and Khaled B . Letaief . 2017 . A survey on mobile edge computing: The communication perspective. IEEE Communications Surveys 8 Tutorials 19, 4 (2017), 2322--2358. Yuyi Mao, Changsheng You, Jun Zhang, Kaibin Huang, and Khaled B. Letaief. 2017. A survey on mobile edge computing: The communication perspective. IEEE Communications Surveys 8 Tutorials 19, 4 (2017), 2322--2358."},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS","volume":"54","author":"McMahan Brendan","year":"2017","unstructured":"Brendan McMahan , Eider Moore , Daniel Ramage , Seth Hampson , and Blaise Ag\u00fcera y Arcas . 2017 . Communication-efficient learning of deep networks from decentralized data . In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017), (Fort Lauderdale, FL), (Proceedings of Machine Learning Research) , Vol. 54 . PMLR, 1273--1282. http:\/\/proceedings.mlr.press\/v54\/mcmahan17a.html. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Ag\u00fcera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017), (Fort Lauderdale, FL), (Proceedings of Machine Learning Research), Vol. 54. PMLR, 1273--1282. http:\/\/proceedings.mlr.press\/v54\/mcmahan17a.html."},{"key":"e_1_2_1_27_1","unstructured":"Regulation (EU) 2016\/679 of the European Parliament and of the Council. 2016. Mobile-Edge Computing -- Introductory Technical White Paper. https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?qid&equals;1564833514064&uri&equals;&equals;CELEX:32016R0679  Regulation (EU) 2016\/679 of the European Parliament and of the Council. 2016. Mobile-Edge Computing -- Introductory Technical White Paper. https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?qid&equals;1564833514064&uri&equals;&equals;CELEX:32016R0679"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/MWC.2015.7054714"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.424"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICBDSC.2016.7460395"},{"key":"e_1_2_1_31_1","volume-title":"Follownet: Robot navigation by following natural language directions with deep reinforcement learning. Arxiv Preprint Arxiv:1805.06150","author":"Shah Pararth","year":"2018","unstructured":"Pararth Shah , Marek Fiser , Aleksandra Faust , J. Chase Kew , and Dilek Hakkani-Tur . 2018 . Follownet: Robot navigation by following natural language directions with deep reinforcement learning. Arxiv Preprint Arxiv:1805.06150 (2018). Pararth Shah, Marek Fiser, Aleksandra Faust, J. Chase Kew, and Dilek Hakkani-Tur. 2018. Follownet: Robot navigation by following natural language directions with deep reinforcement learning. Arxiv Preprint Arxiv:1805.06150 (2018)."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2016.2579198"},{"key":"e_1_2_1_33_1","volume-title":"Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587","author":"Silver David","year":"2016","unstructured":"David Silver , Aja Huang , Chris J. Maddison , Arthur Guez , Laurent Sifre , George van den Driessche , Julian Schrittwieser , Ioannis Antonoglou , Veda Panneershelvam , Marc Lanctot , Sander Dieleman , Dominik Grewe , John Nham , Nal Kalchbrenner , Ilya Sutskever , Timothy Lillicrap , Madeleine Leach , Koray Kavukcuoglu , Thore Graepel , and Demis Hassabis . 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 ( 2016 ), 484. David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484."},{"key":"e_1_2_1_34_1","volume-title":"Barto","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G . Barto . 2018 . Reinforcement Learning : An Introduction. MIT press . Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT press."},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the 13th AAAI Conference on Artificial Intelligence, 2016","author":"van Hasselt Hado","year":"2016","unstructured":"Hado van Hasselt , Arthur Guez , and David Silver . 2016 . Deep reinforcement learning with double q-learning . In Proceedings of the 13th AAAI Conference on Artificial Intelligence, 2016 , (Phoenix, Arizona) , AAAI Press, 2094--2100. http:\/\/www.aaai.org\/ocs\/index.php\/AAAI\/AAAI16\/paper\/view\/12389 Hado van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double q-learning. In Proceedings of the 13th AAAI Conference on Artificial Intelligence, 2016, (Phoenix, Arizona), AAAI Press, 2094--2100. http:\/\/www.aaai.org\/ocs\/index.php\/AAAI\/AAAI16\/paper\/view\/12389"},{"key":"e_1_2_1_36_1","volume-title":"Human-level control through deep reinforcement learning. Nature 518, 7540","author":"Mnih Volodymyr","year":"2015","unstructured":"Volodymyr Mnih , Koray Kavukcuoglu , David Silver , Andrei A. Rusu , Joel Veness , Marc G. Bellemare , Alex Graves , Martin Riedmiller , Andreas K. Fidjeland , Georg Ostrovski , Stig Petersen , Charles Beattie , Amir Sadik , Ioannis Antonoglou , Helen King , Dharshan Kumaran , Daan Wierstra , Shane Legg , and Demis Hassabis . 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 ( 2015 ), 529--533. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529--533."},{"key":"e_1_2_1_37_1","volume-title":"Md Zakirul Alam Bhuiyan, and Qun Jin","author":"Wang Tian","year":"2018","unstructured":"Tian Wang , Guangxue Zhang , Anfeng Liu , Md Zakirul Alam Bhuiyan, and Qun Jin . 2018 . A secure IoT service architecture with an efficient balance dynamics based on cloud and edge computing. IEEE Internet of Things Journal ( 2018). Tian Wang, Guangxue Zhang, Anfeng Liu, Md Zakirul Alam Bhuiyan, and Qun Jin. 2018. A secure IoT service architecture with an efficient balance dynamics based on cloud and edge computing. IEEE Internet of Things Journal (2018)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2017.2694844"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2016.7565185"}],"container-title":["ACM Transactions on Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3372025","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3372025","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:44:19Z","timestamp":1750203859000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3372025"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,19]]},"references-count":39,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,2,29]]}},"alternative-id":["10.1145\/3372025"],"URL":"https:\/\/doi.org\/10.1145\/3372025","relation":{},"ISSN":["1550-4859","1550-4867"],"issn-type":[{"value":"1550-4859","type":"print"},{"value":"1550-4867","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,19]]},"assertion":[{"value":"2019-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-12-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}