{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T09:50:59Z","timestamp":1781949059966,"version":"3.54.5"},"reference-count":48,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T00:00:00Z","timestamp":1781913600000},"content-version":"vor","delay-in-days":170,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,1,1]],"date-time":"2026-01-01T00:00:00Z","timestamp":1767225600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["International Journal of Intelligent Systems"],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:p>The rapid expansion of the Internet of Things (IoT) has greatly increased the number of sensors operating within cloud\u2013fog environments. As more users generate processing tasks, there is a critical need to deliver swift and efficient responses. To address this challenge, it is essential to develop a task scheduling framework that leverages available resources across both fog and cloud domains. This study focuses on the joint optimization of task scheduling and task result caching\u2014a combination that effectively manages computational resources but has been largely overlooked in prior research. While task scheduling has been well explored, result caching has received comparatively little attention, and their integrated application remains underdeveloped. By concurrently employing scheduling and caching strategies, we aim to minimize response times and operational costs. Specifically, frequently requested task results are cached to enable immediate replies to user queries. A key component is the algorithm that decides where to process tasks and where to cache their outputs. We propose a reinforcement\u2010learning\u2010based, Actor\u2013Critic real\u2010time scheduler designed for dynamic, nonepisodic fog\u2013cloud environments with multiple agents. Experimental evaluations demonstrate that our approach significantly outperforms baseline methods (LR\u2010MMT and LRR\u2010MMT) and recent state\u2010of\u2010the\u2010art approaches including Digital Twin, PPO\u2010GRU, A3C\u2010R2N2, and DDQN in reducing both latency and cost.<\/jats:p>","DOI":"10.1155\/int\/5268008","type":"journal-article","created":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T09:08:13Z","timestamp":1781946493000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Joint Planning for Task Scheduling and Task Result Caching in the Fog Using Reinforcement Learning"],"prefix":"10.1155","volume":"2026","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8959-3345","authenticated-orcid":false,"given":"Mohammad Hassan Nataj","family":"Solhdar","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7874-2563","authenticated-orcid":false,"given":"Mehdi","family":"Esnaashari","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,6,20]]},"reference":[{"key":"e_1_2_11_1_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2020.2985711"},{"key":"e_1_2_11_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/tii.2020.3023507"},{"key":"e_1_2_11_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/tii.2014.2300753"},{"key":"e_1_2_11_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12652-020-02448-3"},{"key":"e_1_2_11_5_2","doi-asserted-by":"publisher","DOI":"10.3390\/s21051832"},{"key":"e_1_2_11_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/tii.2020.2999310"},{"key":"e_1_2_11_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00607-022-01147-5"},{"key":"e_1_2_11_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2021.3123234"},{"key":"e_1_2_11_9_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13677-022-00338-x"},{"key":"e_1_2_11_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/tvt.2021.3068255"},{"key":"e_1_2_11_11_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.2509"},{"key":"e_1_2_11_12_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.995"},{"key":"e_1_2_11_13_2","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.1867"},{"key":"e_1_2_11_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12243-016-0527-6"},{"key":"e_1_2_11_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12083-021-01125-2"},{"key":"e_1_2_11_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12652-020-01768-8"},{"key":"e_1_2_11_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-020-03230-y"},{"key":"e_1_2_11_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-021-03334-z"},{"key":"e_1_2_11_19_2","doi-asserted-by":"publisher","DOI":"10.1155\/2021\/9923326"},{"key":"e_1_2_11_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/tii.2020.2978946"},{"key":"e_1_2_11_21_2","doi-asserted-by":"publisher","DOI":"10.32604\/csse.2022.019175"},{"key":"e_1_2_11_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpat.2022.102687"},{"key":"e_1_2_11_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/tpds.2021.3116863"},{"key":"e_1_2_11_24_2","doi-asserted-by":"publisher","DOI":"10.3390\/s21217308"},{"key":"e_1_2_11_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jnca.2019.04.010"},{"key":"e_1_2_11_26_2","article-title":"A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning","volume":"11","author":"Lu J.","year":"2024","journal-title":"IEEE Internet of Things Journal"},{"key":"e_1_2_11_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2024.107517"},{"key":"e_1_2_11_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2024.107925"},{"key":"e_1_2_11_29_2","doi-asserted-by":"crossref","unstructured":"PaczolayG.andHarmatiI. A New Advantage actor-critic Algorithm for multi-agent Environments Proceedings of the 2020 23rd International Symposium on Measurement and Control in Robotics (ISMCR) 2020 Iasi 1\u20136.","DOI":"10.1109\/ISMCR51255.2020.9263738"},{"key":"e_1_2_11_30_2","doi-asserted-by":"publisher","DOI":"10.3390\/s22041555"},{"key":"e_1_2_11_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/tetc.2018.2805718"},{"key":"e_1_2_11_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2021.3082532"},{"key":"e_1_2_11_33_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.2824"},{"key":"e_1_2_11_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/tmc.2020.3017079"},{"key":"e_1_2_11_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/tpds.2019.2893648"},{"key":"e_1_2_11_36_2","doi-asserted-by":"crossref","unstructured":"MaoH. AlizadehM. MenacheI. andKandulaS. Resource Management with Deep Reinforcement Learning Proceedings of the 15th ACM Workshop on Hot Topics in Networks 2016 Atlanta GA 50\u201356.","DOI":"10.1145\/3005745.3005750"},{"key":"e_1_2_11_37_2","doi-asserted-by":"crossref","unstructured":"PathakD. KrahenbuhlP. andDarrellT. Constrained Convolutional Neural Networks for Weakly Supervised Segmentation Proceedings of the 2015 IEEE International Conference on Computer Vision 2015 Santiago 1796\u20131804.","DOI":"10.1109\/ICCV.2015.209"},{"key":"e_1_2_11_38_2","doi-asserted-by":"publisher","DOI":"10.1023\/a:1022676722315"},{"key":"e_1_2_11_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10723-021-09560-4"},{"key":"e_1_2_11_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12652-020-02421-0"},{"key":"e_1_2_11_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2019.09.060"},{"key":"e_1_2_11_42_2","unstructured":"YeY. RenX. WangJ.et al. A New Approach for Resource Scheduling With Deep Reinforcement Learning 2018 https:\/\/arxiv.org\/abs\/1806.08122."},{"key":"e_1_2_11_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpat.2019.102042"},{"key":"e_1_2_11_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3418501"},{"key":"e_1_2_11_45_2","article-title":"Actor-Critic Algorithms","volume":"12","author":"Konda V.","year":"1999","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_11_46_2","doi-asserted-by":"crossref","unstructured":"ChengM. LiJ. andNazarianS. DRL-Cloud: Deep Reinforcement Learning-Based Resource Provisioning and Task Scheduling for Cloud Service Providers Proceedings of the 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC) 2018 Jeju 129\u2013134.","DOI":"10.1109\/ASPDAC.2018.8297294"},{"key":"e_1_2_11_47_2","doi-asserted-by":"crossref","unstructured":"ShenS. Van BeekV. andIosupA. Statistical Characterization of Business-Critical Workloads Hosted in Cloud Datacenters Proceedings of the 2015 15th IEEE\/ACM International Symposium on Cluster Cloud and Grid Computing 2015 Shenzhen 465\u2013474.","DOI":"10.1109\/CCGrid.2015.60"},{"key":"e_1_2_11_48_2","doi-asserted-by":"crossref","unstructured":"AksuD. \u00dcstebayS. AydinM. A. andAtmacaT. Intrusion Detection With Comparative Analysis of Supervised Learning Techniques and Fisher Score Feature Selection Algorithm Proceedings of the 2018 International Symposium on Computer and Information Sciences 2018 Pozna\u0144 141\u2013149.","DOI":"10.1007\/978-3-030-00840-6_16"}],"container-title":["International Journal of Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/int\/5268008","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1155\/int\/5268008","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/int\/5268008","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T09:08:16Z","timestamp":1781946496000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/int\/5268008"}},"subtitle":[],"editor":[{"given":"Richard","family":"Murray","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,1]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1]]}},"alternative-id":["10.1155\/int\/5268008"],"URL":"https:\/\/doi.org\/10.1155\/int\/5268008","archive":["Portico"],"relation":{},"ISSN":["0884-8173","1098-111X"],"issn-type":[{"value":"0884-8173","type":"print"},{"value":"1098-111X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1]]},"assertion":[{"value":"2025-07-10","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-23","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"5268008"}}