{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T05:46:50Z","timestamp":1774590410231,"version":"3.50.1"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T00:00:00Z","timestamp":1693526400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T00:00:00Z","timestamp":1693526400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The unmanned aerial vehicle (UAV) assisted mobile edge computing (MEC) system leverages the high maneuverability of UAVs to provide efficient computing services to terminals. A dynamic deployment algorithm based on double deep Q-networks (DDQN) is suggested to address issues with energy limitation and obstacle avoidance when providing edge services to terminals by UAV. First, the energy consumption of the UAV and the fairness of the terminal\u2019s geographic location are jointly optimized in the case of multiple obstacles and multiple terminals on the ground. And the UAV can avoid obstacles. Furthermore, a double deep Q-network was introduced to address the slow convergence and risk of falling into local optima during the optimization problem training process. Also included in the learning process was a pseudo count exploration strategy. Finally, the improved DDQN algorithm achieves faster convergence and a higher average system reward, according to experimental results. Regarding the fairness of geographic locations of terminals, the improved DDQN algorithm outperforms Q-learning, DQN, and DDQN algorithms by 50%, 20%, and 15.38%, respectively, and the stability of the improved algorithm is also validated.<\/jats:p>","DOI":"10.1186\/s13677-023-00507-6","type":"journal-article","created":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T03:30:18Z","timestamp":1693539018000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Dynamic deployment method based on double deep Q-network in UAV-assisted MEC systems"],"prefix":"10.1186","volume":"12","author":[{"given":"Suqin","family":"Zhang","sequence":"first","affiliation":[]},{"given":"Lin","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Fei","family":"Xu","sequence":"additional","affiliation":[]},{"given":"Song","family":"Cheng","sequence":"additional","affiliation":[]},{"given":"Weiya","family":"Su","sequence":"additional","affiliation":[]},{"given":"Sen","family":"Wang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,9,1]]},"reference":[{"issue":"05","key":"507_CR1","first-page":"907","volume":"54","author":"W Shi","year":"2017","unstructured":"Shi W, Sun H, Cao J, Zhang Q, Liu W (2017) Edge computing\u202f: a new computing model in the era of internet of everything. Comput Res Dev 54(05):907\u2013924","journal-title":"Comput Res Dev"},{"issue":"6A","key":"507_CR2","first-page":"268","volume":"47","author":"M Zhao","year":"2020","unstructured":"Zhao M (2020) Overview of edge computing technology and its application. Comput Sci 47(6A):268\u2013272","journal-title":"Comput Sci"},{"issue":"05","key":"507_CR3","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1109\/MCOM.2016.7470933","volume":"54","author":"Y Zeng","year":"2016","unstructured":"Zeng Y, Zhang R, Lim T (2016) Wireless communications with unmanned aerial vehicles: Opportunities and challenges. IEEE Commun Mag 54(05):36\u201342","journal-title":"IEEE Commun Mag"},{"issue":"01","key":"507_CR4","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1109\/TCOMM.2019.2947921","volume":"68","author":"Y Zhou","year":"2019","unstructured":"Zhou Y, Pan C, Yeoh P (2019) Secure communications for uav-enabled mobile edge computing systems. IEEE Trans Commun 68(01):376\u2013388","journal-title":"IEEE Trans Commun"},{"issue":"09","key":"507_CR5","doi-asserted-by":"publisher","first-page":"4576","DOI":"10.1109\/TWC.2019.2927313","volume":"18","author":"Z Yang","year":"2019","unstructured":"Yang Z, Pan C, Wang K (2019) Energy efficient resource allocation in uav-enabled mobile edge computing networks. IEEE Trans Wirel Commun 18(09):4576\u20134589","journal-title":"IEEE Trans Wirel Commun"},{"issue":"02","key":"507_CR6","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1109\/MCOM.2017.1600587CM","volume":"55","author":"N Motlagh","year":"2017","unstructured":"Motlagh N, Bagga M, Taleb T (2017) Uav-based iot platform: A crowd surveillance use case. IEEE Commun Mag 55(02):128\u2013134","journal-title":"IEEE Commun Mag"},{"issue":"03","key":"507_CR7","first-page":"227","volume":"2","author":"C Dong","year":"2020","unstructured":"Dong C, Shen Y, Qu Y (2020) Survey of edge intelligent computing based on uav. J Intell Sci Technol 2(03):227\u2013239","journal-title":"J Intell Sci Technol"},{"issue":"05","key":"507_CR8","doi-asserted-by":"publisher","first-page":"5028","DOI":"10.1109\/TVT.2021.3075860","volume":"70","author":"Y Yuan","year":"2021","unstructured":"Yuan Y, Lei L, Vu T (2021) Energy minimization in uav-aided networks: Actor-critic learning for constrained scheduling optimization. IEEE Trans Veh Technol 70(05):5028\u20135042","journal-title":"IEEE Trans Veh Technol"},{"key":"507_CR9","doi-asserted-by":"publisher","unstructured":"Duong T, Nguyen L, Tuan H (2019) Learning-aided realtime performance optimisation of cognitive uav-assisted disaster communication. Waikoloa, HI, USA.IEEE. In: Proceedings of the 2019 IEEE Global Communications Conference. https:\/\/doi.org\/10.1109\/GLOBECOM38437.2019.9014313","DOI":"10.1109\/GLOBECOM38437.2019.9014313"},{"issue":"08","key":"507_CR10","doi-asserted-by":"publisher","first-page":"7957","DOI":"10.1109\/TVT.2019.2920284","volume":"68","author":"X Liu","year":"2019","unstructured":"Liu X, Liu Y, Chen Y (2019) Trajectory design and power control for multi-uav assisted wireless networks: A machine learning approach. IEEE Trans Veh Technol 68(08):7957\u20137969","journal-title":"IEEE Trans Veh Technol"},{"issue":"02","key":"507_CR11","doi-asserted-by":"publisher","first-page":"1741","DOI":"10.1109\/JIOT.2018.2875493","volume":"6","author":"J Wang","year":"2018","unstructured":"Wang J, Jiang C, Wei Z (2018) Joint uav hovering altitude and power control for space-air-ground iot networks. IEEE Internet Things J 6(02):1741\u20131753","journal-title":"IEEE Internet Things J"},{"issue":"12","key":"507_CR12","doi-asserted-by":"publisher","first-page":"1713","DOI":"10.1631\/FITEE.2000315","volume":"21","author":"P Huang","year":"2020","unstructured":"Huang P, Wang Y, Wang K (2020) Energy-efficient trajectory planning for a multi-uav-assisted mobile edge computing system. Front Inf Technol Electron Eng 21(12):1713\u20131725","journal-title":"Front Inf Technol Electron Eng"},{"key":"507_CR13","doi-asserted-by":"crossref","unstructured":"Lai C, Wang L, Han Z (2019) Data-driven 3d placement of uav base stations for arbitrarily distributed crowds. In: Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM)","DOI":"10.1109\/GLOBECOM38437.2019.9014210"},{"issue":"03","key":"507_CR14","doi-asserted-by":"publisher","first-page":"2049","DOI":"10.1109\/TVT.2017.2706308","volume":"67","author":"S Jeong","year":"2017","unstructured":"Jeong S, Simeone O, Kang J (2017) Mobile edge computing via a uav-mounted cloudlet: Optimization of bit allocation and path planning. IEEE Trans Veh Technol 67(03):2049\u20132063","journal-title":"IEEE Trans Veh Technol"},{"issue":"21","key":"507_CR15","doi-asserted-by":"publisher","first-page":"21751","DOI":"10.1109\/JIOT.2022.3182750","volume":"9","author":"MY Arafat","year":"2022","unstructured":"Arafat MY, Moh S (2022) Jrcs: Joint routing and charging strategy for logistics drones. IEEE Internet Things J 9(21):21751\u201321764","journal-title":"IEEE Internet Things J"},{"issue":"10","key":"507_CR16","doi-asserted-by":"publisher","first-page":"2150","DOI":"10.1109\/LWC.2022.3195787","volume":"11","author":"X Dai","year":"2022","unstructured":"Dai X, Duo B, Yuan X, Tang W (2022) Energy-efficient uav communications: A generalized propulsion energy consumption model. IEEE Wirel Commun Lett 11(10):2150\u20132154","journal-title":"IEEE Wirel Commun Lett"},{"issue":"01","key":"507_CR17","doi-asserted-by":"publisher","first-page":"136","DOI":"10.23919\/JCC.2022.01.011","volume":"19","author":"X Qi","year":"2022","unstructured":"Qi X, Yuan M, Zhang Q (2022) Joint power - trajectory scheduling optimization in a mobile uav-enabled network via alternating iteration. China Commun 19(01):136\u2013152","journal-title":"China Commun"},{"issue":"04","key":"507_CR18","doi-asserted-by":"publisher","first-page":"257","DOI":"10.23919\/JCC.2022.04.019","volume":"19","author":"H Hu","year":"2022","unstructured":"Hu H, Zhou X, Wang Q (2022) Online computation offloading and trajectory scheduling for uav-enabled wireless powered mobile edge computing. China Commun 19(04):257\u2013273","journal-title":"China Commun"},{"issue":"01","key":"507_CR19","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1109\/TCOMM.2020.3025910","volume":"69","author":"Y Xu","year":"2020","unstructured":"Xu Y, Zhang T, Yang D (2020) Joint resource and trajectory optimization for security in uav-assisted mec systems. IEEE Trans Commun 69(01):573\u2013588","journal-title":"IEEE Trans Commun"},{"issue":"01","key":"507_CR20","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1109\/LWC.2021.3122957","volume":"11","author":"D Wang","year":"2021","unstructured":"Wang D, Tian J, Zhang H (2021) Task offloading and trajectory scheduling for uav-enabled mec networks: an optimal transport theory perspective. IEEE Wirel Commun Lett 11(01):150\u2013154","journal-title":"IEEE Wirel Commun Lett"},{"issue":"04","key":"507_CR21","doi-asserted-by":"publisher","first-page":"2756","DOI":"10.1109\/TII.2019.2954334","volume":"16","author":"J Wang","year":"2019","unstructured":"Wang J, Jiang C, Zhang K (2019) Distributed q-learning aided heterogeneous network association for energy-efficient iiot. IEEE Trans Ind Inf 16(04):2756\u20132764","journal-title":"IEEE Trans Ind Inf"},{"key":"507_CR22","doi-asserted-by":"publisher","unstructured":"Lillicrap T, Hunt J, Pritzel A (2015) Continuous control with deep reinforcement learning.arXiv:1509.02971 [cs.LG]. https:\/\/doi.org\/10.48550\/arXiv.1509.02971","DOI":"10.48550\/arXiv.1509.02971"},{"issue":"05","key":"507_CR23","doi-asserted-by":"publisher","first-page":"5723","DOI":"10.1109\/TVT.2020.2982508","volume":"69","author":"Q Liu","year":"2020","unstructured":"Liu Q, Shi L, Sun L (2020) Path planning for uav-mounted mobile edge computing with deep reinforcement learning. IEEE Trans Veh Technol 69(05):5723\u20135728","journal-title":"IEEE Trans Veh Technol"},{"issue":"01","key":"507_CR24","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1109\/TCCN.2020.3027695","volume":"7","author":"L Wang","year":"2020","unstructured":"Wang L, Wang K, Pan C (2020) Multi-agent deep reinforcement learning-based trajectory planning for multi-uav assisted mobile edge computing. IEEE Trans Cogn Commun Netw 7(01):73\u201384","journal-title":"IEEE Trans Cogn Commun Netw"},{"issue":"04","key":"507_CR25","first-page":"2933","volume":"9","author":"S Yin","year":"2021","unstructured":"Yin S, Yu R (2021) Resource allocation and trajectory design in uav-aided cellular networks based on multiagent reinforcement learning. IEEE Internet J 9(04):2933\u20132943","journal-title":"IEEE Internet J"},{"issue":"01","key":"507_CR26","doi-asserted-by":"publisher","first-page":"47","DOI":"10.23919\/JCN.2021.000026","volume":"24","author":"Z Wu","year":"2021","unstructured":"Wu Z, Yang Z, Yang C (2021) Joint deployment and trajectory optimization in uav-assisted vehicular edge computing networks. J Commun Netw 24(01):47\u201358","journal-title":"J Commun Netw"},{"key":"507_CR27","doi-asserted-by":"publisher","unstructured":"Bor-Yaliniz R, El-Keyi A, Yanikomeroglu H (2016) Efficient 3-d placement of an aerial base station in next generation cellular networks.Kuala Lumpur, Malaysia. IEEE. In: Proceedings of the 2016 IEEE international conference on communications (ICC). https:\/\/doi.org\/10.1109\/ICC.2016.7510820","DOI":"10.1109\/ICC.2016.7510820"},{"issue":"02","key":"507_CR28","doi-asserted-by":"publisher","first-page":"1879","DOI":"10.1109\/JIOT.2018.2878876","volume":"6","author":"Q Hu","year":"2018","unstructured":"Hu Q, Cai Y, Yu G (2018) Joint offloading and trajectory design for uav-enabled mobile edge computing systems. IEEE Internet Things J 6(02):1879\u20131892","journal-title":"IEEE Internet Things J"},{"issue":"03","key":"507_CR29","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1109\/TETCI.2021.3083410","volume":"6","author":"H Chang","year":"2021","unstructured":"Chang H, Chen Y, Zhang B (2021) Multi-uav mobile edge computing and path planning platform based on reinforcement learning. IEEE Trans Emerg Top Comput Intell 6(03):489\u2013498","journal-title":"IEEE Trans Emerg Top Comput Intell"},{"issue":"03","key":"507_CR30","doi-asserted-by":"publisher","first-page":"277","DOI":"10.26599\/TST.2018.9010005","volume":"23","author":"Y Liu","year":"2018","unstructured":"Liu Y, Liu D, Yue D (2018) Bgmm: a body gauss-markov based mobility model for body area networks. Tsinghua Sci Technol 23(03):277\u2013287","journal-title":"Tsinghua Sci Technol"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00507-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-023-00507-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00507-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,19]],"date-time":"2023-11-19T07:31:31Z","timestamp":1700379091000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-023-00507-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,1]]},"references-count":30,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["507"],"URL":"https:\/\/doi.org\/10.1186\/s13677-023-00507-6","relation":{},"ISSN":["2192-113X"],"issn-type":[{"value":"2192-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,1]]},"assertion":[{"value":"21 April 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 August 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 September 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"130"}}