{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T15:51:29Z","timestamp":1780761089382,"version":"3.54.1"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2023,12,24]],"date-time":"2023-12-24T00:00:00Z","timestamp":1703376000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2023,12,24]],"date-time":"2023-12-24T00:00:00Z","timestamp":1703376000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Wireless Netw"],"published-print":{"date-parts":[[2024,12]]},"DOI":"10.1007\/s11276-023-03596-y","type":"journal-article","created":{"date-parts":[[2023,12,24]],"date-time":"2023-12-24T10:01:18Z","timestamp":1703412078000},"page":"7275-7291","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Deep reinforcement learning based computation offloading for xURLLC services with UAV-assisted IoT-based multi-access edge computing system"],"prefix":"10.1007","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9817-1806","authenticated-orcid":false,"given":"Nida","family":"Fatima","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paresh","family":"Saxena","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giovanni","family":"Giambene","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,12,24]]},"reference":[{"key":"3596_CR1","doi-asserted-by":"publisher","first-page":"127639","DOI":"10.1109\/ACCESS.2019.2939938","volume":"7","author":"A Ghosh","year":"2019","unstructured":"Ghosh, A., Maeder, A., Baker, M., & Chandramouli, D. (2019). 5g evolution: A view on 5g cellular technology beyond 3g pp release 15. IEEE access, 7, 127639\u2013127651.","journal-title":"IEEE access"},{"issue":"1","key":"3596_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11432-020-2955-6","volume":"64","author":"X You","year":"2021","unstructured":"You, X., Wang, C.-X., Huang, J., Gao, X., Zhang, Z., Wang, M., Huang, Y., Zhang, C., Jiang, Y., Wang, J., et al. (2021). Towards 6g wireless communication networks: Vision, enabling technologies, and new paradigm shifts. Science China Information Sciences, 64(1), 1\u201374.","journal-title":"Science China Information Sciences"},{"issue":"4","key":"3596_CR3","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1109\/JPROC.2020.2976703","volume":"108","author":"E Yaacoub","year":"2020","unstructured":"Yaacoub, E., & Alouini, M.-S. (2020). A key 6g challenge and opportunity\u2013connecting the base of the pyramid: A survey on rural connectivity. Proceedings of the IEEE, 108(4), 533\u2013582.","journal-title":"Proceedings of the IEEE"},{"issue":"3","key":"3596_CR4","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1038\/s41928-022-00728-8","volume":"5","author":"J Park","year":"2022","unstructured":"Park, J., Samarakoon, S., Shiri, H., Abdel-Aziz, M. K., Nishio, T., Elgabli, A., & Bennis, M. (2022). Extreme ultra-reliable and low-latency communication. Nature Electronics, 5(3), 133\u2013141.","journal-title":"Nature Electronics"},{"issue":"6","key":"3596_CR5","doi-asserted-by":"publisher","first-page":"4132","DOI":"10.1109\/TCOMM.2019.2898573","volume":"67","author":"C-F Liu","year":"2019","unstructured":"Liu, C.-F., Bennis, M., Debbah, M., & Poor, H. V. (2019). Dynamic task offloading and resource allocation for ultra-reliable low-latency edge computing. IEEE Transactions on Communications, 67(6), 4132\u20134150.","journal-title":"IEEE Transactions on Communications"},{"key":"3596_CR6","doi-asserted-by":"publisher","first-page":"101641","DOI":"10.1016\/j.phycom.2022.101641","volume":"52","author":"N Fatima","year":"2022","unstructured":"Fatima, N., Saxena, P., & Gupta, M. (2022). Integration of multi access edge computing with unmanned aerial vehicles: Current techniques, open issues and research directions. Physical Communication, 52, 101641.","journal-title":"Physical Communication"},{"issue":"2","key":"3596_CR7","doi-asserted-by":"publisher","first-page":"244","DOI":"10.1109\/MNET.011.2000493","volume":"35","author":"M Giordani","year":"2020","unstructured":"Giordani, M., & Zorzi, M. (2020). Non-terrestrial networks in the 6g era: Challenges and opportunities. IEEE Network, 35(2), 244\u2013251.","journal-title":"IEEE Network"},{"key":"3596_CR8","doi-asserted-by":"crossref","unstructured":"Saxena, P., Dreibholz, T., Skinnemoen, H., Alay, \u00d6., Vazquez-Castro, M. A., Ferlin, S., & Acar, G. (2020). Resilient hybrid satcom and terrestrial networking for unmanned aerial vehicles. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 418\u2013423. IEEE.","DOI":"10.1109\/INFOCOMWKSHPS50562.2020.9162943"},{"issue":"9","key":"3596_CR9","doi-asserted-by":"publisher","first-page":"6597","DOI":"10.1109\/JIOT.2021.3113872","volume":"9","author":"F Jiang","year":"2021","unstructured":"Jiang, F., Dong, L., Wang, K., Yang, K., & Pan, C. (2021). Distributed resource scheduling for large-scale mec systems: A multiagent ensemble deep reinforcement learning with imitation acceleration. IEEE Internet of Things Journal, 9(9), 6597\u20136610.","journal-title":"IEEE Internet of Things Journal"},{"issue":"09","key":"3596_CR10","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1109\/TC.1977.1674939","volume":"26","author":"PM Narendra","year":"1977","unstructured":"Narendra, P. M., & Fukunaga, K. (1977). A branch and bound algorithm for feature subset selection. IEEE Transactions on computers, 26(09), 917\u2013922.","journal-title":"IEEE Transactions on computers"},{"issue":"6","key":"3596_CR11","doi-asserted-by":"publisher","first-page":"833","DOI":"10.2307\/3010291","volume":"47","author":"DP Bertsekas","year":"1996","unstructured":"Bertsekas, D. P. (1996). Dynamic programming and optimal control. Journal of the Operational Research Society, 47(6), 833.","journal-title":"Journal of the Operational Research Society"},{"issue":"6","key":"3596_CR12","doi-asserted-by":"publisher","first-page":"4177","DOI":"10.1109\/TWC.2018.2821664","volume":"17","author":"S Bi","year":"2018","unstructured":"Bi, S., & Zhang, Y. J. (2018). Computation rate maximization for wireless powered mobile-edge computing with binary computation offloading. IEEE Transactions on Wireless Communications, 17(6), 4177\u20134190.","journal-title":"IEEE Transactions on Wireless Communications"},{"issue":"11","key":"3596_CR13","doi-asserted-by":"publisher","first-page":"2606","DOI":"10.1109\/JSAC.2017.2760186","volume":"35","author":"X Lyu","year":"2017","unstructured":"Lyu, X., Ni, W., Tian, H., Liu, R. P., Wang, X., Giannakis, G. B., & Paulraj, A. (2017). Optimal schedule of mobile edge computing for internet of things using partial information. IEEE Journal on Selected Areas in Communications, 35(11), 2606\u20132615.","journal-title":"IEEE Journal on Selected Areas in Communications"},{"key":"3596_CR14","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1016\/j.future.2019.08.032","volume":"102","author":"W-C Chien","year":"2020","unstructured":"Chien, W.-C., Weng, H.-Y., & Lai, C.-F. (2020). Q-learning based collaborative cache allocation in mobile edge computing. Future generation computer systems, 102, 603\u2013610.","journal-title":"Future generation computer systems"},{"issue":"3","key":"3596_CR15","doi-asserted-by":"publisher","first-page":"4005","DOI":"10.1109\/JIOT.2018.2876279","volume":"6","author":"X Chen","year":"2018","unstructured":"Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2018). Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet of Things Journal, 6(3), 4005\u20134018.","journal-title":"IEEE Internet of Things Journal"},{"issue":"3","key":"3596_CR16","doi-asserted-by":"publisher","first-page":"1628","DOI":"10.1109\/COMST.2017.2682318","volume":"19","author":"P Mach","year":"2017","unstructured":"Mach, P., & Becvar, Z. (2017). Mobile edge computing: A survey on architecture and computation offloading. IEEE communications surveys & tutorials, 19(3), 1628\u20131656.","journal-title":"IEEE communications surveys & tutorials"},{"key":"3596_CR17","doi-asserted-by":"publisher","first-page":"165462","DOI":"10.1109\/ACCESS.2021.3135464","volume":"9","author":"L Zhang","year":"2021","unstructured":"Zhang, L., Abderrahim, W., & Shihada, B. (2021). Heterogeneous traffic offloading in space-air-ground integrated networks. IEEE Access, 9, 165462\u2013165475.","journal-title":"IEEE Access"},{"key":"3596_CR18","doi-asserted-by":"crossref","unstructured":"Cao, Y., Dai, B., Mo, Y., & Xu, Y. (2020). Iqor: An intelligent qos-aware routing mechanism with deep reinforcement learning. In 2020 IEEE 45th Conference on Local Computer Networks (LCN), pp. 329\u2013332. IEEE.","DOI":"10.1109\/LCN48667.2020.9314768"},{"key":"3596_CR19","doi-asserted-by":"crossref","unstructured":"Zhang, L., Jabbari, B., & Ansari, N. (2022). Machine learning driven uav-assisted edge computing. In 2022 IEEE Wireless Communications and Networking Conference (WCNC), pp. 2220\u20132225. IEEE.","DOI":"10.1109\/WCNC51071.2022.9771769"},{"key":"3596_CR20","doi-asserted-by":"crossref","unstructured":"Lu, W., Mo, Y., Feng, Y., Gao, Y., Zhao, N., & Wu, Y. (2022). Nallanathan, A.: Secure transmission for multi-uav-assisted mobile edge computing based on reinforcement learning. IEEE Transactions on Network Science and Engineering.","DOI":"10.1109\/TNSE.2022.3185130"},{"key":"3596_CR21","unstructured":"Mobile-Edge Computing (MEC) Service Scenarios. [Online] Available At: https:\/\/www.etsi.org\/deliver\/etsi_gs\/mec-ieg\/001_099\/004\/01.01.01_60\/gs_mec-ieg004v010101p.pdf."},{"key":"3596_CR22","unstructured":"3GPP Release 18. [Online] Available at: https:\/\/www.3gpp.org\/specifications-technologies\/releases\/release-18."},{"key":"3596_CR23","unstructured":"Fujimoto, S., Hoof, H., & Meger, D. (2018). Addressing function approximation error in actor-critic methods. In International Conference on Machine Learning, pp. 1587\u20131596. PMLR."},{"key":"3596_CR24","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., & Silver, D. (2016). Deep reinforcement learning with double q-learning. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"3596_CR25","unstructured":"Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971."},{"issue":"10","key":"3596_CR26","doi-asserted-by":"publisher","first-page":"10863","DOI":"10.1109\/TVT.2022.3182647","volume":"71","author":"B Li","year":"2022","unstructured":"Li, B., Liu, Y., Tan, L., Pan, H., & Zhang, Y. (2022). Digital twin assisted task offloading for aerial edge computing and networks. IEEE Transactions on Vehicular Technology, 71(10), 10863\u201310877.","journal-title":"IEEE Transactions on Vehicular Technology"},{"key":"3596_CR27","doi-asserted-by":"crossref","unstructured":"Zhang, K., Cao, J., Wang, L., & Zhang, Y. (2022). Green offloading and trajectory scheduling of rechargeable uavs in aerial edge networks. In GLOBECOM 2022-2022 IEEE Global Communications Conference, pp. 1752\u20131757. IEEE.","DOI":"10.1109\/GLOBECOM48099.2022.10001727"},{"issue":"1","key":"3596_CR28","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1109\/JIOT.2022.3201017","volume":"10","author":"LP Qian","year":"2022","unstructured":"Qian, L. P., Zhang, H., Wang, Q., Wu, Y., & Lin, B. (2022). Joint multi-domain resource allocation and trajectory optimization in uav-assisted maritime iot networks. IEEE Internet of Things Journal, 10(1), 539\u2013552.","journal-title":"IEEE Internet of Things Journal"},{"key":"3596_CR29","doi-asserted-by":"crossref","unstructured":"Al-Hilo, A., Samir, M., Elhattab, M., Assi, C., & Sharafeddine, S. (2022). Ris-assisted uav for timely data collection in iot networks. IEEE Systems Journal.","DOI":"10.1109\/JSYST.2022.3215279"},{"key":"3596_CR30","doi-asserted-by":"crossref","unstructured":"Wang, H., Zhang, H., Liu, X., Long, K., & Nallanathan, A. (2022). Joint uav placement optimization, resource allocation, and computation offloading for thz band: A drl approach. IEEE Transactions on Wireless Communications.","DOI":"10.1109\/TWC.2022.3230407"},{"key":"3596_CR31","doi-asserted-by":"crossref","unstructured":"Zhang, X., & Wang, Y. (2022). Deepmecagent: Multi-agent computing resource allocation for uav-assisted mobile edge computing in distributed iot system. Applied Intelligence, 1\u201312","DOI":"10.1007\/s10489-022-03482-8"},{"issue":"19","key":"3596_CR32","doi-asserted-by":"publisher","first-page":"6499","DOI":"10.3390\/s21196499","volume":"21","author":"S Li","year":"2021","unstructured":"Li, S., Hu, X., & Du, Y. (2021). Deep reinforcement learning for computation offloading and resource allocation in unmanned-aerial-vehicle assisted edge computing. Sensors, 21(19), 6499.","journal-title":"Sensors"},{"key":"3596_CR33","doi-asserted-by":"publisher","first-page":"53708","DOI":"10.1109\/ACCESS.2021.3070908","volume":"9","author":"L Zhang","year":"2021","unstructured":"Zhang, L., Zhang, Z.-Y., Min, L., Tang, C., Zhang, H.-Y., Wang, Y.-H., & Cai, P. (2021). Task offloading and trajectory control for uav-assisted mobile edge computing using deep reinforcement learning. IEEE Access, 9, 53708\u201353719.","journal-title":"IEEE Access"},{"issue":"2","key":"3596_CR34","doi-asserted-by":"publisher","first-page":"111","DOI":"10.24138\/jcomss-2021-0154","volume":"18","author":"AA Baktayan","year":"2022","unstructured":"Baktayan, A. A., Al-Baltah, I. A., & Abd Ghani, A. A. (2022). Intelligent pricing model for task offloading in unmanned aerial vehicle mounted mobile edge computing for vehicular network. Journal of Communications Software and Systems, 18(2), 111\u2013123.","journal-title":"Journal of Communications Software and Systems"},{"issue":"3","key":"3596_CR35","doi-asserted-by":"publisher","first-page":"538","DOI":"10.1109\/LCOMM.2019.2891662","volume":"23","author":"J Xiong","year":"2019","unstructured":"Xiong, J., Guo, H., & Liu, J. (2019). Task offloading in uav-aided edge computing: Bit allocation and trajectory optimization. IEEE Communications Letters, 23(3), 538\u2013541.","journal-title":"IEEE Communications Letters"},{"issue":"3","key":"3596_CR36","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","volume":"27","author":"CE Shannon","year":"1948","unstructured":"Shannon, C. E. (1948). A mathematical theory of communication. The Bell system technical journal, 27(3), 379\u2013423.","journal-title":"The Bell system technical journal"},{"issue":"9","key":"3596_CR37","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1109\/MCOM.2013.6588654","volume":"51","author":"M Coldrey","year":"2013","unstructured":"Coldrey, M., Berg, J.-E., Manholm, L., Larsson, C., & Hansryd, J. (2013). Non-line-of-sight small cell backhauling using microwave technology. IEEE Communications Magazine, 51(9), 78\u201384. https:\/\/doi.org\/10.1109\/MCOM.2013.6588654","journal-title":"IEEE Communications Magazine"},{"issue":"2","key":"3596_CR38","doi-asserted-by":"publisher","first-page":"1879","DOI":"10.1109\/JIOT.2018.2878876","volume":"6","author":"Q Hu","year":"2018","unstructured":"Hu, Q., Cai, Y., Yu, G., Qin, Z., Zhao, M., & Li, G. Y. (2018). Joint offloading and trajectory design for uav-enabled mobile edge computing systems. IEEE Internet of Things Journal, 6(2), 1879\u20131892.","journal-title":"IEEE Internet of Things Journal"},{"key":"3596_CR39","volume-title":"Markov Decision Processes: Discrete Stochastic Dynamic Programming","author":"ML Puterman","year":"2014","unstructured":"Puterman, M. L. (2014). Markov Decision Processes: Discrete Stochastic Dynamic Programming. New Jersey: John Wiley & Sons."},{"issue":"3","key":"3596_CR40","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","volume":"8","author":"CJ Watkins","year":"1992","unstructured":"Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine learning, 8(3), 279\u2013292.","journal-title":"Machine learning"},{"issue":"7540","key":"3596_CR41","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529\u2013533.","journal-title":"Nature"},{"key":"3596_CR42","unstructured":"Konda, V., & Tsitsiklis, J. (1999). Actor-critic algorithms. Advances in neural information processing systems 12."},{"issue":"4","key":"3596_CR43","doi-asserted-by":"publisher","first-page":"838","DOI":"10.1137\/0330046","volume":"30","author":"BT Polyak","year":"1992","unstructured":"Polyak, B. T., & Juditsky, A. B. (1992). Acceleration of stochastic approximation by averaging. SIAM journal on control and optimization, 30(4), 838\u2013855.","journal-title":"SIAM journal on control and optimization"},{"key":"3596_CR44","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016). $$\\{$$TensorFlow$$\\}$$: a system for $$\\{$$Large-Scale$$\\}$$ machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 265\u2013283."}],"container-title":["Wireless Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11276-023-03596-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11276-023-03596-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11276-023-03596-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,23]],"date-time":"2024-11-23T08:15:11Z","timestamp":1732349711000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11276-023-03596-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,24]]},"references-count":44,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["3596"],"URL":"https:\/\/doi.org\/10.1007\/s11276-023-03596-y","relation":{},"ISSN":["1022-0038","1572-8196"],"issn-type":[{"value":"1022-0038","type":"print"},{"value":"1572-8196","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,24]]},"assertion":[{"value":"21 November 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 December 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no financial or proprietary interests in any material discussed in this article. This work is original and not have been published elsewhere in any form or language.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}