{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:21:34Z","timestamp":1760235694735,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2021,9,9]],"date-time":"2021-09-09T00:00:00Z","timestamp":1631145600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62071470","61971421"],"award-info":[{"award-number":["62071470","61971421"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005196","name":"Chinese Academy of Agricultural Sciences","doi-asserted-by":"publisher","award":["(CAAS\u2013ASTIP-2021-AII-01)"],"award-info":[{"award-number":["(CAAS\u2013ASTIP-2021-AII-01)"]}],"id":[{"id":"10.13039\/501100005196","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Today, vehicles are increasingly being connected to the Internet of Things, which enables them to obtain high-quality services. However, the numerous vehicular applications and time-varying network status make it challenging for onboard terminals to achieve efficient computing. Therefore, based on a three-stage model of local-edge clouds and reinforcement learning, we propose a task offloading algorithm for the Internet of Vehicles (IoV). First, we establish communication methods between vehicles and their cost functions. In addition, according to the real-time state of vehicles, we analyze their computing requirements and the price function. Finally, we propose an experience-driven offloading strategy based on multi-agent reinforcement learning. The simulation results show that the algorithm increases the probability of success for the task and achieves a balance between the task vehicle delay, expenditure, task vehicle utility and service vehicle utility under various constraints.<\/jats:p>","DOI":"10.3390\/s21186058","type":"journal-article","created":{"date-parts":[[2021,9,9]],"date-time":"2021-09-09T21:36:58Z","timestamp":1631223418000},"page":"6058","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Research on a Task Offloading Strategy for the Internet of Vehicles Based on Reinforcement Learning"],"prefix":"10.3390","volume":"21","author":[{"given":"Shuo","family":"Xiao","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shengzhi","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiayu","family":"Zhuang","sequence":"additional","affiliation":[{"name":"Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing 100080, China"},{"name":"Key Laboratory of Agri-Information Service Technology, Ministry of Agriculture, Beijing 100080, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tianyu","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiajia","family":"Liu","sequence":"additional","affiliation":[{"name":"Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing 100080, China"},{"name":"Key Laboratory of Agri-Information Service Technology, Ministry of Agriculture, Beijing 100080, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1446","DOI":"10.1007\/s11431-017-9338-1","article-title":"Intelligent and connected vehicles: Current status and future perspectives","volume":"61","author":"Yang","year":"2018","journal-title":"Sci. China Technol. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"04002","DOI":"10.1051\/matecconf\/202032504002","article-title":"Analysis of the Business Models of the Intelligent and Connected Vehicle Industry","volume":"Volume 325","author":"Zhao","year":"2020","journal-title":"MATEC Web of Conferences"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1109\/MITS.2017.2743167","article-title":"Autonomous vehicle-intersection coordination method in a connected vehicle environment","volume":"9","author":"Lin","year":"2017","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_4","first-page":"3701","article-title":"Internet of Vehicles: Architecture, protocols, and security","volume":"5","author":"Zeadally","year":"2017","journal-title":"IEEE Internet Things J."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3614","DOI":"10.1109\/JIOT.2020.2973420","article-title":"Toward Sustainable Micro-Level Fog-Federated Load Sharing in Internet of Vehicles","volume":"7","author":"Sharmin","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6046","DOI":"10.1109\/JIOT.2019.2958097","article-title":"Smart Collaborative Tracking for Ubiquitous Power IoT in Edge-Cloud Interplay Domain","volume":"7","author":"Song","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1109\/MCOM.2015.7060486","article-title":"Mobile code offloading: From concept to practice and beyond","volume":"53","author":"Flores","year":"2015","journal-title":"IEEE Commun. Mag."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1007\/s10107-015-0881-6","article-title":"A game-theoretic approach to computation offloading in mobile cloud computing","volume":"157","author":"Cardellini","year":"2016","journal-title":"Math. Program."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1060","DOI":"10.1109\/TCCN.2019.2930521","article-title":"Deep Reinforcement Learning for Intelligent Internet of Vehicles: An Energy-Efficient Computational Offloading Scheme","volume":"5","author":"Ning","year":"2019","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"6916","DOI":"10.1109\/TII.2020.3029766","article-title":"Smart Collaborative Balancing for Dependable Network Components in Cyber-Physical Systems","volume":"17","author":"Song","year":"2021","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.future.2019.01.012","article-title":"An edge computing-enabled computation offloading method with privacy preservation for internet of connected vehicles","volume":"96","author":"Xu","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Raza, S., Wang, S., Ahmed, M., and Anwar, M.R. (2019). A survey on vehicular edge computing: Architecture, applications, technical issues, and future directions. Wirel. Commun. Mob. Comput., 2019.","DOI":"10.1155\/2019\/3159762"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"102265","DOI":"10.1016\/j.adhoc.2020.102265","article-title":"Enhancing intelligence in traffic management systems to aid in vehicle traffic congestion problems in smart cities","volume":"107","author":"Meneguette","year":"2020","journal-title":"Ad Hoc Netw."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3941","DOI":"10.1109\/TVT.2016.2550105","article-title":"DaVe: Offloading delay-tolerant data traffic to connected vehicle networks","volume":"65","author":"Si","year":"2016","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, K., Mao, Y., Leng, S., Maharjan, S., and Zhang, Y. (2017, January 21\u201325). Optimal delay constrained offloading for vehicular edge computing networks. Proceedings of the 2017 IEEE International Conference on Communications (ICC), Paris, France.","DOI":"10.1109\/ICC.2017.7997360"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Hu, B., Chen, J., and Li, F. (2017, January 18\u201320). A dynamic service allocation algorithm in mobile edge computing. Proceedings of the 2017 IEEE International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Korea.","DOI":"10.1109\/ICTC.2017.8190951"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, B., Zhao, X., Han, S., and Chen, Z. (2018, January 15\u201316). New sdn-based architecture for integrated vehicular cloud computing networking. Proceedings of the 2018 IEEE International Conference on Selected Topics in Mobile and Wireless Networking (MoWNeT), Tianjin, China.","DOI":"10.1109\/MoWNet.2018.8428935"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"25408","DOI":"10.1109\/ACCESS.2017.2769878","article-title":"Distributed reputation management for secure and efficient vehicular edge computing and networks","volume":"5","author":"Huang","year":"2017","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, Y., Wang, S., Huang, J., and Yang, F. (2018, January 20\u201324). A computation offloading algorithm based on game theory for vehicular edge networks. Proceedings of the 2018 IEEE International Conference on Communications (ICC), Kansas City, MO, USA.","DOI":"10.1109\/ICC.2018.8422240"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3860","DOI":"10.1109\/TVT.2016.2532863","article-title":"Vehicular fog computing: A viewpoint of vehicles as the infrastructures","volume":"65","author":"Hou","year":"2016","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2618","DOI":"10.1109\/TII.2018.2799177","article-title":"Energy-efficient resource allocation for industrial cyber-physical IoT systems in 5G era","volume":"14","author":"Li","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1109\/TGCN.2019.2914736","article-title":"QoS-based robust power optimization for SWIPT NOMA system with statistical CSI","volume":"3","author":"Zhou","year":"2019","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1109\/MNET.011.2000613","article-title":"Enabling Heterogeneous Deterministic Networks with Smart Collaborative Theory","volume":"35","author":"Song","year":"2021","journal-title":"IEEE Netw."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, K., Mao, Y., Leng, S., Vinel, A., and Zhang, Y. (2016, January 13\u201315). Delay constrained offloading for mobile edge computing in cloud-enabled vehicular networks. Proceedings of the 2016 8th IEEE International Workshop on Resilient Networks Design and Modeling (RNDM), Halmstad, Sweden.","DOI":"10.1109\/RNDM.2016.7608300"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ren, J., Yu, G., Cai, Y., He, Y., and Qu, F. (2017, January 4\u20138). Partial offloading for latency minimization in mobile-edge computing. Proceedings of the GLOBECOM 2017\u20132017 IEEE Global Communications Conference, Singapore.","DOI":"10.1109\/GLOCOM.2017.8254550"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Luoto, P., Bennis, M., Pirinen, P., Samarakoon, S., Horneman, K., and Latva-Aho, M. (2017, January 12\u201315). Vehicle clustering for improving enhanced LTE-V2X network performance. Proceedings of the 2017 IEEE European Conference on Networks and Communications (EuCNC), Oulu, Finland.","DOI":"10.1109\/EuCNC.2017.7980735"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/MVT.2017.2668838","article-title":"Mobile-edge computing for vehicular networks: A promising network paradigm with predictive off-loading","volume":"12","author":"Zhang","year":"2017","journal-title":"IEEE Veh. Technol. Mag."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1016\/j.ins.2018.06.002","article-title":"Smart collaborative distribution for privacy enhancement in moving target defense","volume":"479","author":"Song","year":"2019","journal-title":"Inf. Sci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/OJCS.2020.2992630","article-title":"Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues","volume":"1","author":"Du","year":"2020","journal-title":"IEEE Open J. Comput. Soc."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1385","DOI":"10.1109\/TII.2019.2950109","article-title":"Smart Collaborative Automation for Receive Buffer Control in Multipath Industrial Networks","volume":"16","author":"Song","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/MNET.2019.1800453","article-title":"Integrating Licensed and Unlicensed Spectrum in the Internet of Vehicles with Mobile Edge Computing","volume":"33","author":"Wu","year":"2019","journal-title":"IEEE Netw."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Guo, S., Xiao, B., Yang, Y., and Yang, Y. (2016, January 10\u201314). Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing. Proceedings of the IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, San Francisco, CA, USA.","DOI":"10.1109\/INFOCOM.2016.7524497"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"131068","DOI":"10.1109\/ACCESS.2019.2940295","article-title":"A Computation Offloading Method for Edge Computing With Vehicle-to-Everything","volume":"7","author":"Xu","year":"2019","journal-title":"IEEE Access"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6058\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:59:59Z","timestamp":1760165999000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6058"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,9]]},"references-count":33,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21186058"],"URL":"https:\/\/doi.org\/10.3390\/s21186058","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,9,9]]}}}