{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,27]],"date-time":"2025-11-27T06:43:19Z","timestamp":1764225799749,"version":"3.37.3"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,10,28]],"date-time":"2022-10-28T00:00:00Z","timestamp":1666915200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,28]],"date-time":"2022-10-28T00:00:00Z","timestamp":1666915200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62172134"],"award-info":[{"award-number":["62172134"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The exponential device growth in industrial Internet of things (IIoT) has a noticeable impact on the volume of data generated. Edge-cloud computing cooperation has been introduced to the IIoT to lessen the computational load on cloud servers and shorten the processing time for data. General programmable logic controllers (PLCs), which have been playing important roles in industrial control systems, start to gain the ability to process a large amount of industrial data and share the workload of cloud servers. This transforms them into edge-PLCs. However, the continuous influx of multiple types of concurrent production data streams against the limited capacity of built-in memory in PLCs brings a huge challenge. Therefore, the ability to reasonably allocate memory resources in edge-PLCs to ensure data utilization and real-time processing has become one of the core means of improving the efficiency of industrial processes. In this paper, to tackle dynamic changes in arrival data rate over time at each edge-PLC, we propose to optimize memory allocation with Q-learning distributedly. The simulation experiments verify that the method can effectively reduce the data loss probability while improving the system performance.<\/jats:p>","DOI":"10.1186\/s13677-022-00348-9","type":"journal-article","created":{"date-parts":[[2022,10,28]],"date-time":"2022-10-28T20:03:44Z","timestamp":1666987424000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Distributed reinforcement learning-based memory allocation for edge-PLCs in industrial IoT"],"prefix":"10.1186","volume":"11","author":[{"given":"Tingting","family":"Fu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanjun","family":"Peng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peng","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haksrun","family":"Lao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shaohua","family":"Wan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,10,28]]},"reference":[{"key":"348_CR1","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1186\/s13677-020-00168-9","volume":"9","author":"H Wu","year":"2020","unstructured":"Wu H, Li X, Deng Y (2020) Deep learning-driven wireless communication for edge-cloud computing: opportunities and challenges. J Cloud Comput 9:21","journal-title":"J Cloud Comput"},{"key":"348_CR2","doi-asserted-by":"publisher","unstructured":"Chen C, Li H, Li H, Fu R, Liu Y, Wan S (2022) Efficiency and fairness oriented dynamic task offloading in internet of vehicles. IEEE Trans Green Commun Netw 1. https:\/\/doi.org\/10.1109\/TGCN.2022.3167643","DOI":"10.1109\/TGCN.2022.3167643"},{"issue":"1","key":"348_CR3","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1186\/s13677-021-00256-4","volume":"10","author":"Q You","year":"2021","unstructured":"You Q, Tang B (2021) Efficient task offloading using particle swarm optimization algorithm in edge computing for industrial internet of things. J Cloud Comput 10(1):41","journal-title":"J Cloud Comput"},{"issue":"9","key":"348_CR4","doi-asserted-by":"publisher","first-page":"8099","DOI":"10.1109\/JIOT.2020.2996784","volume":"7","author":"H Wu","year":"2020","unstructured":"Wu H, Zhang Z, Guan C, Wolter K, Xu M (2020) Collaborate edge and cloud computing with distributed deep learning for smart city internet of things. IEEE Internet Things J 7(9):8099\u20138110. https:\/\/doi.org\/10.1109\/JIOT.2020.2996784","journal-title":"IEEE Internet Things J"},{"key":"348_CR5","doi-asserted-by":"publisher","unstructured":"Zhang Z, Wang N, Wu H, Tang C, Li R (2021) Mr-dro: A fast and efficient task offloading algorithm in heterogeneous edge\/cloud computing environments. IEEE Internet Things J 1\u20131. https:\/\/doi.org\/10.1109\/JIOT.2021.3126101","DOI":"10.1109\/JIOT.2021.3126101"},{"issue":"14","key":"348_CR6","doi-asserted-by":"publisher","first-page":"11514","DOI":"10.1109\/JIOT.2021.3053017","volume":"8","author":"H Wu","year":"2021","unstructured":"Wu H, Yan Y, Sun D, Wu H, Liu P (2021) Multi buffers multi objects optimal matching scheme for edge devices in iiot. IEEE Internet Things J 8(14):11514\u201311525. https:\/\/doi.org\/10.1109\/JIOT.2021.3053017","journal-title":"IEEE Internet Things J"},{"issue":"5","key":"348_CR7","doi-asserted-by":"publisher","first-page":"1830","DOI":"10.1007\/s12083-020-00934-1","volume":"13","author":"Y Peng","year":"2020","unstructured":"Peng Y, Liu P, Fu T (2020) Performance analysis of edge-plcs enabled industrial internet of things. Peer Peer Netw Appl 13(5):1830\u20131838","journal-title":"Peer Peer Netw Appl"},{"issue":"2","key":"348_CR8","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/j.dcan.2019.08.004","volume":"6","author":"S Safavat","year":"2020","unstructured":"Safavat S, Sapavath NN, Rawat DB (2020) Recent advances in mobile edge computing and content caching. Digit Commun Netw 6(2):189\u2013194. https:\/\/doi.org\/10.1016\/j.dcan.2019.08.004","journal-title":"Digit Commun Netw"},{"key":"348_CR9","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1186\/s13677-021-00237-7","volume":"10","author":"J Chen","year":"2021","unstructured":"Chen J, Du T, Xiao G (2021) A multi-objective optimization for resource allocation of emergent demands in cloud computing. J Cloud Comput 10:17","journal-title":"J Cloud Comput"},{"issue":"5","key":"348_CR10","doi-asserted-by":"publisher","first-page":"2795","DOI":"10.1109\/TNET.2015.2487344","volume":"24","author":"X Chen","year":"2016","unstructured":"Chen X, Jiao L, Li W, Fu X (2016) Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE\/ACM Trans Networking 24(5):2795\u20132808. https:\/\/doi.org\/10.1109\/TNET.2015.2487344","journal-title":"IEEE\/ACM Trans Networking"},{"issue":"8","key":"348_CR11","doi-asserted-by":"publisher","first-page":"4924","DOI":"10.1109\/TWC.2017.2703901","volume":"16","author":"C Wang","year":"2017","unstructured":"Wang C, Liang C, Yu FR, Chen Q, Tang L (2017) Computation offloading and resource allocation in wireless cellular networks with mobile edge computing. IEEE Trans Wirel Commun 16(8):4924\u20134938. https:\/\/doi.org\/10.1109\/TWC.2017.2703901","journal-title":"IEEE Trans Wirel Commun"},{"key":"348_CR12","doi-asserted-by":"publisher","unstructured":"Sadatdiynov K, Cui L, Zhang L, Huang JZ, Salloum S, Mahmud MS (2022) A review of optimization methods for computation offloading in edge computing networks. Digit Commun Netw. https:\/\/doi.org\/10.1016\/j.dcan.2022.03.003","DOI":"10.1016\/j.dcan.2022.03.003"},{"issue":"1","key":"348_CR13","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1109\/JIOT.2017.2780236","volume":"5","author":"L Liu","year":"2017","unstructured":"Liu L, Chang Z, Guo X, Mao S, Ristaniemi T (2017) Multiobjective optimization for computation offloading in fog computing. IEEE Internet Things J 5(1):283\u2013294","journal-title":"IEEE Internet Things J"},{"key":"348_CR14","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2021.108146","volume":"121","author":"S Wan","year":"2022","unstructured":"Wan S, Ding S, Chen C (2022) Edge computing enabled video segmentation for real-time traffic monitoring in internet of vehicles. Pattern Recog 121:108146. https:\/\/doi.org\/10.1016\/j.patcog.2021.108146","journal-title":"Pattern Recog"},{"issue":"2","key":"348_CR15","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1109\/TCC.2018.2789446","volume":"8","author":"H Wu","year":"2020","unstructured":"Wu H, Sun Y, Wolter K (2020) Energy-efficient decision making for mobile cloud offloading. IEEE Trans Cloud Comput 8(2):570\u2013584. https:\/\/doi.org\/10.1109\/TCC.2018.2789446","journal-title":"IEEE Trans Cloud Comput"},{"key":"348_CR16","doi-asserted-by":"publisher","unstructured":"Chen C, Zeng Y, Li H, Liu Y, Wan S (2022) A multi-hop task offloading decision model in mec-enabled internet of vehicles. IEEE Internet Things J 1. https:\/\/doi.org\/10.1109\/JIOT.2022.3143529","DOI":"10.1109\/JIOT.2022.3143529"},{"issue":"6","key":"348_CR17","first-page":"1171","volume":"3","author":"R Deng","year":"2016","unstructured":"Deng R, Lu R, Lai C, Luan TH, Liang H (2016) Optimal workload allocation in fog-cloud computing toward balanced delay and power consumption. IEEE Internet Things J 3(6):1171\u20131181","journal-title":"IEEE Internet Things J"},{"issue":"8","key":"348_CR18","first-page":"3571","volume":"65","author":"TQ Dinh","year":"2017","unstructured":"Dinh TQ, Tang J, La QD, Quek TQ (2017) Offloading in mobile edge computing: Task allocation and computational frequency scaling. IEEE Trans Commun 65(8):3571\u20133584","journal-title":"IEEE Trans Commun"},{"key":"348_CR19","doi-asserted-by":"publisher","unstructured":"Wei W, Yang R, Gu H, Zhao W, Chen C, Wan S (2021) Multi-objective optimization for resource allocation in vehicular cloud computing networks. IEEE Trans Intell Transp Syst 1\u201310. https:\/\/doi.org\/10.1109\/TITS.2021.3091321","DOI":"10.1109\/TITS.2021.3091321"},{"issue":"1","key":"348_CR20","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1016\/j.dcan.2018.10.003","volume":"5","author":"L Huang","year":"2019","unstructured":"Huang L, Feng X, Zhang C, Qian L, Wu Y (2019) Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing. Digit Commun Netw 5(1):10\u201317. https:\/\/doi.org\/10.1016\/j.dcan.2018.10.003","journal-title":"Digit Commun Netw"},{"key":"348_CR21","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2021.107108","volume":"103","author":"C Chen","year":"2021","unstructured":"Chen C, Zhang Y, Wang Z, Wan S, Pei Q (2021) Distributed computation offloading method based on deep reinforcement learning in icv. Applied Soft Computing 103:107108. https:\/\/doi.org\/10.1016\/j.asoc.2021.107108","journal-title":"Applied Soft Computing"},{"issue":"9","key":"348_CR22","doi-asserted-by":"publisher","first-page":"6103","DOI":"10.1109\/TII.2020.2974875","volume":"16","author":"S Deng","year":"2020","unstructured":"Deng S, Xiang Z, Zhao P, Taheri J, Gao H, Yin J, Zomaya AY (2020) Dynamical resource allocation in edge for trustable Internet-of-things systems: A reinforcement learning method. IEEE Trans Ind Inform 16(9):6103\u20136113. https:\/\/doi.org\/10.1109\/TII.2020.2974875","journal-title":"IEEE Trans Ind Inform"},{"issue":"3","key":"348_CR23","doi-asserted-by":"publisher","first-page":"1337","DOI":"10.1109\/TNSM.2020.3000274","volume":"17","author":"A Kaur","year":"2020","unstructured":"Kaur A, Kumar K (2020) Energy-efficient resource allocation in cognitive radio networks under cooperative multi-agent model-free reinforcement learning schemes. IEEE Trans Netw Serv Manag 17(3):1337\u20131348. https:\/\/doi.org\/10.1109\/TNSM.2020.3000274","journal-title":"IEEE Trans Netw Serv Manag"},{"issue":"2","key":"348_CR24","doi-asserted-by":"publisher","first-page":"729","DOI":"10.1109\/TWC.2019.2935201","volume":"19","author":"J Cui","year":"2020","unstructured":"Cui J, Liu Y, Nallanathan A (2020) Multi-agent reinforcement learning-based resource allocation for uav networks. IEEE Trans Wirel Commun 19(2):729\u2013743. https:\/\/doi.org\/10.1109\/TWC.2019.2935201","journal-title":"IEEE Trans Wirel Commun"},{"issue":"2","key":"348_CR25","doi-asserted-by":"publisher","first-page":"1041","DOI":"10.1109\/JIOT.2020.3009540","volume":"8","author":"J Baek","year":"2021","unstructured":"Baek J, Kaddoum G (2021) Heterogeneous task offloading and resource allocations via deep recurrent reinforcement learning in partial observable multifog networks. IEEE Internet Things J 8(2):1041\u20131056. https:\/\/doi.org\/10.1109\/JIOT.2020.3009540","journal-title":"IEEE Internet Things J"},{"issue":"1","key":"348_CR26","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.dcan.2018.10.006","volume":"5","author":"Q Li","year":"2019","unstructured":"Li Q, Lu C, Cao B, Zhang Q (2019) Caching resource management of mobile edge network based on stackelberg game. Digital Communications and Networks 5(1):18\u201323. https:\/\/doi.org\/10.1016\/j.dcan.2018.10.006","journal-title":"Digital Communications and Networks"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-022-00348-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-022-00348-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-022-00348-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,28]],"date-time":"2022-10-28T20:05:40Z","timestamp":1666987540000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-022-00348-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,28]]},"references-count":26,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["348"],"URL":"https:\/\/doi.org\/10.1186\/s13677-022-00348-9","relation":{},"ISSN":["2192-113X"],"issn-type":[{"type":"electronic","value":"2192-113X"}],"subject":[],"published":{"date-parts":[[2022,10,28]]},"assertion":[{"value":"6 August 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 October 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"73"}}