{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T16:37:11Z","timestamp":1783701431355,"version":"3.55.0"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,8,8]],"date-time":"2023-08-08T00:00:00Z","timestamp":1691452800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,8]],"date-time":"2023-08-08T00:00:00Z","timestamp":1691452800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Natural Science Foundation of Chongqing, China","award":["No. CSTB2022NSCQ-MSX0368"],"award-info":[{"award-number":["No. CSTB2022NSCQ-MSX0368"]}]},{"name":"Young Project of Science and Technology Research Program of Chongqing Education Commission of China","award":["No. KJQN202200703"],"award-info":[{"award-number":["No. KJQN202200703"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In vehicular edge computing, the low-delay services are invoked by the vehicles from the edge clouds while the vehicles moving on the roads. Because of the insufficiency of computing capacity and storage resource for edge clouds, a single edge cloud cannot handle all the services, and thus the efficient service deployment strategy in multi edge clouds should be designed according to the service demands. Noticed that the service demands are dynamic in temporal, and the inter-relationship between services is a non-negligible factor for service deployment. In order to address the new challenges produced by these factors, a collaborative service on-demand dynamic deployment approach with deep reinforcement learning is proposed, which is named CODD-DQN. In our approach, the number of service request of each edge clouds are forecasted by a time-aware service demands prediction algorithm, and then the interacting services are discovered through the analysis of service invoking logs. On this basis, the service response time models are constructed to formulated the problem, aiming to minimize service response time with data transmission delay between services. Furthermore, a collaborative service dynamic deployment algorithm with DQN model is proposed to deploy the interacting services. Finally, the real-world dataset based experiments are conducted. The results show our approach can achieve lowest service response time than other algorithms for service deployment.<\/jats:p>","DOI":"10.1186\/s13677-023-00488-6","type":"journal-article","created":{"date-parts":[[2023,8,8]],"date-time":"2023-08-08T06:01:51Z","timestamp":1691474511000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Collaborative on-demand dynamic deployment via deep reinforcement learning for IoV service in multi edge clouds"],"prefix":"10.1186","volume":"12","author":[{"given":"Yuze","family":"Huang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beipeng","family":"Feng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuhui","family":"Cao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhenzhen","family":"Guo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miao","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Boren","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,8,8]]},"reference":[{"issue":"5","key":"488_CR1","doi-asserted-by":"publisher","first-page":"3701","DOI":"10.1109\/JIOT.2017.2690902","volume":"5","author":"J Contreras-Castillo","year":"2018","unstructured":"Contreras-Castillo J, Zeadally S, Ib\u00e1\u00f1ez JAG (2018) Internet of Vehicles: Architecture, Protocols, and Security. IEEE Internet Things J. 5(5):3701\u20133709","journal-title":"IEEE Internet Things J."},{"issue":"2","key":"488_CR2","doi-asserted-by":"publisher","first-page":"1093","DOI":"10.1109\/TVT.2018.2886010","volume":"68","author":"X Wang","year":"2019","unstructured":"Wang X, Ning Z, Hu X, Wang L, Hu B, Cheng J et al (2019) Optimizing Content Dissemination for Real-Time Traffic Management in Large-Scale Internet of Vehicle Systems. IEEE Trans Veh Technol. 68(2):1093\u20131105","journal-title":"IEEE Trans Veh Technol."},{"key":"488_CR3","first-page":"328","volume-title":"International Conference on Connected Vehicles and Expo, ICCVE 2015, October 19-23, 2015","author":"D Singh","year":"2015","unstructured":"Singh D, Singh M (2015) Internet of vehicles for smart and safe driving. International Conference on Connected Vehicles and Expo, ICCVE 2015, October 19-23, 2015. IEEE, Shenzhen, pp 328\u2013329"},{"issue":"4","key":"488_CR4","doi-asserted-by":"publisher","first-page":"2441","DOI":"10.1109\/JIOT.2018.2847249","volume":"5","author":"R Hussain","year":"2018","unstructured":"Hussain R, Kim D, Son J, Lee J, Kerrache CA, Benslimane A et al (2018) Secure and Privacy-Aware Incentives-Based Witness Service in Social Internet of Vehicles Clouds. IEEE Internet Things J. 5(4):2441\u20132448","journal-title":"IEEE Internet Things J."},{"key":"488_CR5","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1186\/s13677-020-00182-x","volume":"9","author":"M Zhang","year":"2020","unstructured":"Zhang M, Wang S, Gao Q (2020) A joint optimization scheme of content caching and resource allocation for internet of vehicles in mobile edge computing. J Cloud Comput. 9:33","journal-title":"J Cloud Comput."},{"issue":"5","key":"488_CR6","doi-asserted-by":"publisher","first-page":"165506","DOI":"10.1007\/s11704-021-0425-6","volume":"16","author":"L Wu","year":"2022","unstructured":"Wu L, Zhang R, Li Q, Ma C, Shi X (2022) A mobile edge computing-based applications execution framework for Internet of Vehicles. Frontiers Comput Sci. 16(5):165506","journal-title":"Frontiers Comput Sci."},{"issue":"2","key":"488_CR7","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1109\/JPROC.2019.2947490","volume":"108","author":"J Zhang","year":"2020","unstructured":"Zhang J, Letaief KB (2020) Mobile Edge Intelligence and Computing for the Internet of Vehicles. Proc IEEE. 108(2):246\u2013261","journal-title":"Proc IEEE."},{"key":"488_CR8","doi-asserted-by":"publisher","unstructured":"Chen Y, Zhao J, Zhou X et al (2023) A Distributed Game Theoretical Approach for Credibility-guaranteed Multimedia Data Offloading in MEC. Inf Sci. 644:119306. https:\/\/doi.org\/10.1016\/j.ins.2023.119306","DOI":"10.1016\/j.ins.2023.119306"},{"key":"488_CR9","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-83944-4","volume-title":"Mobile Edge Computing","author":"Y Zhang","year":"2022","unstructured":"Zhang Y (2022) Mobile Edge Computing, vol 9. Springer, Cham"},{"issue":"5","key":"488_CR10","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1109\/MNET.2019.1800309","volume":"33","author":"Z Ning","year":"2019","unstructured":"Ning Z, Huang J, Wang X, Rodrigues JJPC, Guo L (2019) Mobile Edge Computing-Enabled Internet of Vehicles: Toward Energy-Efficient Scheduling. IEEE Netw. 33(5):198\u2013205","journal-title":"IEEE Netw."},{"issue":"4","key":"488_CR11","doi-asserted-by":"publisher","first-page":"1002","DOI":"10.1109\/TPDS.2016.2604814","volume":"28","author":"S Wang","year":"2017","unstructured":"Wang S, Urgaonkar R, He T, Chan K, Zafer M, Leung KK (2017) Dynamic Service Placement for Mobile Micro-Clouds with Predicted Future Costs. IEEE Trans Parallel Distrib Syst. 28(4):1002\u20131016","journal-title":"IEEE Trans Parallel Distrib Syst."},{"issue":"1","key":"488_CR12","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1109\/MWC.001.1900273","volume":"27","author":"Y Hao","year":"2020","unstructured":"Hao Y, Chen M, Cao D, Zhao W, Petrov I, Antonenko VA et al (2020) Cognitive-Caching: Cognitive Wireless Mobile Caching by Learning Fine-Grained Caching-Aware Indicators. IEEE Wirel Commun. 27(1):100\u2013106","journal-title":"IEEE Wirel Commun."},{"issue":"10","key":"488_CR13","doi-asserted-by":"publisher","first-page":"4656","DOI":"10.1109\/TII.2018.2846549","volume":"14","author":"L Chen","year":"2018","unstructured":"Chen L, Zhou P, Gao L, Xu J (2018) Adaptive Fog Configuration for the Industrial Internet of Things. IEEE Trans Ind Inform. 14(10):4656\u20134664","journal-title":"IEEE Trans Ind Inform."},{"key":"488_CR14","first-page":"468","volume-title":"2018 IEEE Conference on Computer Communications, INFOCOM 2018, April 16-19, 2018","author":"L Wang","year":"2018","unstructured":"Wang L, Jiao L, He T, Li J, M\u00fchlh\u00e4user M (2018) Service Entity Placement for Social Virtual Reality Applications in Edge Computing. 2018 IEEE Conference on Computer Communications, INFOCOM 2018, April 16-19, 2018. IEEE, Honolulu, pp 468\u2013476"},{"issue":"3","key":"488_CR15","first-page":"65:1","volume":"53","author":"F A\u00eft-Salaht","year":"2021","unstructured":"A\u00eft-Salaht F, Desprez F, Lebre A (2021) An Overview of Service Placement Problem in Fog and Edge Computing. ACM Comput Surv 53(3):65:1-65:35","journal-title":"ACM Comput Surv"},{"key":"488_CR16","first-page":"10","volume-title":"2019 IEEE Conference on Computer Communications, INFOCOM 2019, April 29 - May 2, 2019","author":"K Poularakis","year":"2019","unstructured":"Poularakis K, Llorca J, Tulino AM, Taylor IJ, Tassiulas L (2019) Joint Service Placement and Request Routing in Multi-cell Mobile Edge Computing Networks. 2019 IEEE Conference on Computer Communications, INFOCOM 2019, April 29 - May 2, 2019. IEEE, Paris, pp 10\u201318"},{"key":"488_CR17","first-page":"2076","volume-title":"39th IEEE Conference on Computer Communications, INFOCOM 2020, July 6-9, 2020","author":"X Ma","year":"2020","unstructured":"Ma X, Zhou A, Zhang S, Wang S (2020) Cooperative Service Caching and Workload Scheduling in Mobile Edge Computing. 39th IEEE Conference on Computer Communications, INFOCOM 2020, July 6-9, 2020. IEEE, Toronto, pp 2076\u20132085"},{"key":"488_CR18","doi-asserted-by":"publisher","unstructured":"Chen Y, Zhao J, Hu J et al (2023) Distributed Task Offloading and Resource Purchasing in NOMA-enabled Mobile Edge Computing: Hierarchical Game Theoretical Approaches. ACM Trans Embed Comput Syst. early access. https:\/\/doi.org\/10.1145\/3597023","DOI":"10.1145\/3597023"},{"issue":"8","key":"488_CR19","doi-asserted-by":"publisher","first-page":"5552","DOI":"10.1109\/TII.2020.3041713","volume":"17","author":"Y Hao","year":"2021","unstructured":"Hao Y, Chen M, Gharavi H, Zhang Y, Hwang K (2021) Deep Reinforcement Learning for Edge Service Placement in Softwarized Industrial Cyber-Physical System. IEEE Trans Ind Informatics. 17(8):5552\u20135561","journal-title":"IEEE Trans Ind Informatics."},{"issue":"11","key":"488_CR20","doi-asserted-by":"publisher","first-page":"8964","DOI":"10.1109\/JIOT.2021.3056084","volume":"8","author":"R Wang","year":"2021","unstructured":"Wang R, Kan Z, Cui Y, Wu D, Zhen Y (2021) Cooperative Caching Strategy With Content Request Prediction in Internet of Vehicles. IEEE Internet Things J. 8(11):8964\u20138975","journal-title":"IEEE Internet Things J."},{"issue":"19","key":"488_CR21","doi-asserted-by":"publisher","first-page":"18607","DOI":"10.1109\/JIOT.2022.3161677","volume":"9","author":"Y Hui","year":"2022","unstructured":"Hui Y, Ma X, Su Z, Cheng N, Yin Z, Luan TH et al (2022) Collaboration as a Service: Digital-Twin-Enabled Collaborative and Distributed Autonomous Driving. IEEE Internet Things J. 9(19):18607\u201318619","journal-title":"IEEE Internet Things J."},{"key":"488_CR22","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1186\/s13677-022-00365-8","volume":"11","author":"H Chen","year":"2022","unstructured":"Chen H, Qin W, Wang L (2022) Task partitioning and offloading in IoT cloud-edge collaborative computing framework: a survey. J Cloud Comput. 11:86","journal-title":"J Cloud Comput."},{"key":"488_CR23","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/j.future.2022.09.007","volume":"139","author":"J Huang","year":"2023","unstructured":"Huang J, Gao H, Wan S et al (2023) AoI-aware energy control and computation offloading for industrial IoT. Futur Gener Comput Syst. 139:29\u201337","journal-title":"Futur Gener Comput Syst."},{"key":"488_CR24","doi-asserted-by":"publisher","unstructured":"Chen Y, Zhao J, Wu Y et al (2022) QoE-aware Decentralized Task Offloading and Resource Allocation for End-Edge-Cloud Systems: A Game-Theoretical Approach. IEEE Trans Mob Comput. early access.1\u201317. https:\/\/doi.org\/10.1109\/TMC.2022.3223119","DOI":"10.1109\/TMC.2022.3223119"},{"key":"488_CR25","doi-asserted-by":"publisher","unstructured":"Chen Y, Hu J, Zhao J, Min G (2023) QoS-Aware Computation Offloading in LEO Satellite Edge Computing for IoT: A Game-Theoretical Approach. Chin J Electron. early access. https:\/\/doi.org\/10.23919\/cje.2022.00.412","DOI":"10.23919\/cje.2022.00.412"},{"key":"488_CR26","first-page":"1","volume-title":"2020 IEEE International Conference on Communications, ICC 2020, June 7-11, 2020","author":"M LiWang","year":"2020","unstructured":"LiWang M, Gao Z, Hosseinalipour S, Dai H (2020) Multi-Task Offloading over Vehicular Clouds under Graph-based Representation. 2020 IEEE International Conference on Communications, ICC 2020, June 7-11, 2020. IEEE, Dublin, pp 1\u20137"},{"key":"488_CR27","doi-asserted-by":"publisher","unstructured":"Chen Y, Gu W, Xu J et al (2022) Dynamic Task Offloading for Digital Twin-empowered Mobile Edge Computing via Deep Reinforcement Learning. Chin Commun. early access.\u00a01\u201312. https:\/\/doi.org\/10.23919\/JCC.ea.2022-0372.202302","DOI":"10.23919\/JCC.ea.2022-0372.202302"},{"key":"488_CR28","doi-asserted-by":"publisher","first-page":"108975","DOI":"10.1016\/j.comnet.2022.108975","volume":"213","author":"P Hegyi","year":"2022","unstructured":"Hegyi P (2022) Service deployment design in latency-critical multi-cloud environment. Comput Netw. 213:108975","journal-title":"Comput Netw."},{"key":"488_CR29","doi-asserted-by":"publisher","first-page":"109208","DOI":"10.1016\/j.comnet.2022.109208","volume":"216","author":"D Lima","year":"2022","unstructured":"Lima D, Miranda H (2022) A geographical-aware state deployment service for Fog Computing. Comput Netw. 216:109208","journal-title":"Comput Netw."},{"issue":"2","key":"488_CR30","doi-asserted-by":"publisher","first-page":"1964","DOI":"10.1109\/TVT.2021.3133696","volume":"71","author":"J Huang","year":"2022","unstructured":"Huang J, Lv B, Wu Y et al (2022) Dynamic Admission Control and Resource Allocation for Mobile Edge Computing Enabled Small Cell Network. IEEE Trans Veh Technol. 71(2):1964\u20131973","journal-title":"IEEE Trans Veh Technol."},{"key":"488_CR31","doi-asserted-by":"crossref","unstructured":"Chen Y, Xing H, Ma Z, et\u00a0al (2022) Cost-Efficient Edge Caching for NOMA-enabled IoT Services. Chin Commun","DOI":"10.1155\/2022\/8072493"},{"key":"488_CR32","doi-asserted-by":"publisher","unstructured":"Huang J, Wan J, Lv B, Ye Q et al (2023) Joint Computation Offloading and Resource Allocation for Edge-Cloud Collaboration in Internet of Vehicles via Deep Reinforcement Learning. IEEE Syst J. 17(2):2500\u20132511. https:\/\/doi.org\/10.1109\/JSYST.2023.3249217","DOI":"10.1109\/JSYST.2023.3249217"},{"key":"488_CR33","doi-asserted-by":"publisher","unstructured":"Huang Y, Cao Y, Zhang M, Feng B, Guo Z (2022) CSO-DRL: A Collaborative Service Offloading Approach with Deep Reinforcement Learning in Vehicular Edge Computing. Sci Prog. 2022:1163177. https:\/\/doi.org\/10.1155\/2022\/1163177","DOI":"10.1155\/2022\/1163177"},{"key":"488_CR34","first-page":"558","volume-title":"Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, MobiCom 2017, October 16 - 20, 2017","author":"Y Huang","year":"2017","unstructured":"Huang Y, Huang J, Cheng B, Yao T, Chen J (2017) Poster: Interacting Data-Intensive Services Mining and Placement in Mobile Edge Clouds. Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, MobiCom 2017, October 16 - 20, 2017. ACM, Snowbird, pp 558\u2013560"},{"key":"488_CR35","doi-asserted-by":"publisher","first-page":"474","DOI":"10.1016\/j.future.2020.07.018","volume":"113","author":"Y Huang","year":"2020","unstructured":"Huang Y, Huang J, Liu C, Zhang C (2020) PFPMine: A parallel approach for discovering interacting data entities in data-intensive cloud workflows. Future Gener Comput Syst. 113:474\u2013487","journal-title":"Future Gener Comput Syst."},{"key":"488_CR36","volume-title":"Time Series Analysis: Forecasting and Control","author":"GEP Box","year":"2015","unstructured":"Box GEP, Jenkins GM (2015) Time Series Analysis: Forecasting and Control, 5th edn. Wiley, Hoboken","edition":"5"},{"issue":"3","key":"488_CR37","doi-asserted-by":"publisher","first-page":"1659","DOI":"10.1109\/COMST.2021.3073036","volume":"23","author":"W Chen","year":"2021","unstructured":"Chen W, Qiu X, Cai T, Dai H, Zheng Z, Zhang Y (2021) Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey. IEEE Commun Surv Tutorials. 23(3):1659\u20131692","journal-title":"IEEE Commun Surv Tutorials."},{"issue":"7540","key":"488_CR38","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG et al (2015) Human-level control through deep reinforcement learning. Nat. 518(7540):529\u2013533","journal-title":"Nat."},{"key":"488_CR39","doi-asserted-by":"publisher","unstructured":"Liu H, Li Y, Wang S (2022) Request Scheduling Combined with Load Balancing in Mobile Edge Computing. IEEE Internet of Things. 9(21):20841\u201320852. https:\/\/doi.org\/10.1109\/JIOT.2022.3176631","DOI":"10.1109\/JIOT.2022.3176631"},{"key":"488_CR40","volume-title":"Reinforcement Learning","author":"RS Suton","year":"2018","unstructured":"Suton RS, Barto AG (2018) Reinforcement Learning, 2nd edn. MIT Press, Cambridge","edition":"2"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00488-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-023-00488-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-023-00488-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,21]],"date-time":"2023-11-21T17:25:03Z","timestamp":1700587503000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-023-00488-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,8]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["488"],"URL":"https:\/\/doi.org\/10.1186\/s13677-023-00488-6","relation":{},"ISSN":["2192-113X"],"issn-type":[{"value":"2192-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,8]]},"assertion":[{"value":"28 December 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 July 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 August 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"119"}}