{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T04:49:16Z","timestamp":1777697356134,"version":"3.51.4"},"reference-count":32,"publisher":"SAGE Publications","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IDT"],"published-print":{"date-parts":[[2021,1,6]]},"abstract":"<jats:p>Caching contents at the edge of mobile networks is an efficient mechanism that can alleviate the backhaul links load and reduce the transmission delay. For this purpose, choosing an adequate caching strategy becomes an important issue. Recently, the tremendous growth of Mobile Edge Computing (MEC) empowers the edge network nodes with more computation capabilities and storage capabilities, allowing the execution of resource-intensive tasks within the mobile network edges such as running artificial intelligence (AI) algorithms. Exploiting users context information intelligently makes it possible to design an intelligent context-aware mobile edge caching. To maximize the caching performance, the suitable methodology is to consider both context awareness and intelligence so that the caching strategy is aware of the environment while caching the appropriate content by making the right decision. Inspired by the success of reinforcement learning (RL) that uses agents to deal with decision making problems, we present a modified reinforcement learning (mRL) to cache contents in the network edges. Our proposed solution aims to maximize the cache hit rate and requires a multi awareness of the influencing factors on cache performance. The modified RL differs from other RL algorithms in the learning rate that uses the method of stochastic gradient decent (SGD) beside taking advantage of learning using the optimal caching decision obtained from fuzzy rules.<\/jats:p>","DOI":"10.3233\/idt-190152","type":"journal-article","created":{"date-parts":[[2020,12,15]],"date-time":"2020-12-15T12:36:57Z","timestamp":1608035817000},"page":"537-552","source":"Crossref","is-referenced-by-count":1,"title":["Modified reinforcement learning based- caching system for mobile edge computing"],"prefix":"10.1177","volume":"14","author":[{"given":"Sarra","family":"Mehamel","sequence":"first","affiliation":[{"name":"University Mouloud Mammeri of Tizi-Ouzou, Algeria"},{"name":"Conservatoire National des Arts et M\u00e9tiers, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samia","family":"Bouzefrane","sequence":"additional","affiliation":[{"name":"Conservatoire National des Arts et M\u00e9tiers, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Soumya","family":"Banarjee","sequence":"additional","affiliation":[{"name":"Conservatoire National des Arts et M\u00e9tiers, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mehammed","family":"Daoui","sequence":"additional","affiliation":[{"name":"University Mouloud Mammeri of Tizi-Ouzou, Algeria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Valentina E.","family":"Balas","sequence":"additional","affiliation":[{"name":"Aurel Vlaicu University of Arad, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"issue":"6","key":"10.3233\/IDT-190152_ref2","doi-asserted-by":"crossref","first-page":"557","DOI":"10.3390\/app7060557","article-title":"Energy-efficient caching for mobile edge computing in 5g networks","volume":"7","author":"Luo","year":"2017","journal-title":"Appl. Sci."},{"key":"10.3233\/IDT-190152_ref3","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1109\/MNET.2018.1800109","article-title":"Deep reinforcement learning for mobile edge caching: review, new features, and open issues","volume":"32","author":"Hao","year":"2018","journal-title":"IEEE Network"},{"issue":"3","key":"10.3233\/IDT-190152_ref4","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1016\/j.cognition.2008.08.011","article-title":"Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective","volume":"113","author":"Botvinick","year":"2009","journal-title":"Cognition"},{"issue":"7","key":"10.3233\/IDT-190152_ref5","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/j.tics.2008.03.006","article-title":"Dialogues on prediction errors","volume":"12","author":"Niv","year":"2008","journal-title":"Trends Cogn. Sci. (Regul. Ed.)"},{"key":"10.3233\/IDT-190152_ref7","doi-asserted-by":"crossref","unstructured":"Suomala J, Suomala V. Modified Reinforcement Learning Infrastructure. 2nd International Conference on Applied Social Science Research (ICASSR 2014), pp. 95\u201397.","DOI":"10.2991\/icassr-14.2014.27"},{"issue":"8","key":"10.3233\/IDT-190152_ref8","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MCOM.2014.6871674","article-title":"Living on the edge: the role of proactive caching in 5g wireless networks","volume":"52","author":"Bastug","year":"2014","journal-title":"IEEE Commun. Mag."},{"issue":"6","key":"10.3233\/IDT-190152_ref11","doi-asserted-by":"crossref","first-page":"557","DOI":"10.3390\/app7060557","article-title":"Energy-efficient caching for mobile edge computing in 5g networks","volume":"7","author":"Luo","year":"2017","journal-title":"Applied Sciences"},{"issue":"3","key":"10.3233\/IDT-190152_ref12","doi-asserted-by":"crossref","first-page":"1473","DOI":"10.1109\/COMST.2015.2420097","article-title":"A survey of caching mechanisms in information-centric networking","volume":"17","author":"Zhang","year":"2015","journal-title":"In IEEE Communications Surveys & Tutorials"},{"key":"10.3233\/IDT-190152_ref13","unstructured":"Arif S, Hassan S, Abdullahi I. Cache Replacement Positions in Information-Centric Network. The 4th International Conference on Internet Applications, Protocols and Services (NETAPPS2015), Malaysia, 2014, pp. 54\u201358."},{"issue":"5","key":"10.3233\/IDT-190152_ref16","first-page":"3413","article-title":"Cache skip approach for information-centric network","volume":"11","author":"Ibrahim Abdullahi","year":"2016","journal-title":"ARPN Journal of Engineering and Applied Sciences"},{"key":"10.3233\/IDT-190152_ref17","doi-asserted-by":"crossref","unstructured":"Stefan Podlipnig, Laszlo Bszrmenyi. A survey of Web cache replacement strategies. ACM Computing Surveys (CSUR), 35(Issue 4) (December 2003), 374\u2013398.","DOI":"10.1145\/954339.954341"},{"key":"10.3233\/IDT-190152_ref19","doi-asserted-by":"crossref","unstructured":"Mehamel S, Slimani K, Bouzefrane S, Daoui M. Energy-efficient hardware caching decision using Fuzzy Logic in Mobile Edge Computing. The IEEE 6th International Conference on Future Internet of Things and Cloud Workshops (FiCloud\u20192018), August 2018, pp. 237\u2013242, Barcelona, Spain,","DOI":"10.1109\/W-FiCloud.2018.00045"},{"key":"10.3233\/IDT-190152_ref20","unstructured":"Mehamel S, Bouzefrane S, Slimani K, Daoui M. New caching system under uncertainty for Mobile Edge Computing. The Fourth IEEE International Conference on Fog and Mobile Edge Computing, June 2019, pp.to appear., Rome, Italy,"},{"issue":"1","key":"10.3233\/IDT-190152_ref22","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1109\/JSTSP.2017.2787979","article-title":"Optimal and scalable caching for 5g using reinforcement learning of space-time popularities","volume":"12","author":"Sadeghi","year":"2018","journal-title":"IEEE J. Sel. Topics Signal Process."},{"issue":"6","key":"10.3233\/IDT-190152_ref23","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1109\/JSAC.2018.2844985","article-title":"A reinforcement-learning approach to proactive caching in wireless networks","volume":"36","author":"Somuyiwa","year":"2018","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"1","key":"10.3233\/IDT-190152_ref26","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1109\/JSTSP.2017.2787979","article-title":"Optimal and scalable caching for 5g using reinforcement learning of space-time popularities","volume":"12","author":"Sadeghi","year":"2018","journal-title":"IEEE J. Sel. Top. Signal Process."},{"issue":"1","key":"10.3233\/IDT-190152_ref27","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/TVT.2017.2760281","article-title":"Integrated networking, caching, and computing for connected vehicles: a deep reinforcement learning approach","volume":"67","author":"He","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"10.3233\/IDT-190152_ref29","doi-asserted-by":"crossref","unstructured":"Zhang F, et al. Edgebuffer: Caching and prefetching content at the edge in the mobilityfirst future internet architecture. Proc. IEEE 16th Int. Symp. World Wireless Mobile Multimedia Netw. Jun. 2015, pp. 1\u20139.","DOI":"10.1109\/WoWMoM.2015.7158137"},{"key":"10.3233\/IDT-190152_ref30","unstructured":"Mahmood A, Casetti C, Chiasserini CF, Giaccone P, Harri J. Mobility-aware edge caching for connected cars. 12th Annual Conference on Wireless On-demand Network Systems and Services (WONS), Cortina d\u2019Ampezzo, 2016, pp. 1\u20138."},{"issue":"8","key":"10.3233\/IDT-190152_ref31","doi-asserted-by":"crossref","first-page":"1791","DOI":"10.1109\/TMC.2017.2780834","article-title":"Cooperative edge caching in user-centric clustered mobile networks","volume":"17","author":"Zhang","year":"2018","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"10.3233\/IDT-190152_ref32","doi-asserted-by":"crossref","unstructured":"Psaras I, Chai WK, Pavlou G. Probabilistic in-network caching for information-centric networks. Proc. of the Second Edition of the ICN Workshop on Information-Centric Networking, 2012, pp. 55\u201360.","DOI":"10.1145\/2342488.2342501"},{"key":"10.3233\/IDT-190152_ref34","doi-asserted-by":"crossref","unstructured":"Cho K, Lee M, Park K, Kwon TT, Choi Y, Pack S. WAVE: Popularity-based and collaborative in-network caching for contentoriented networks. Proc of IEEE Infocom Wkshps 2012, March 2012, pp. 316\u2013321.","DOI":"10.1109\/INFCOMW.2012.6193512"},{"key":"10.3233\/IDT-190152_ref36","doi-asserted-by":"crossref","unstructured":"Nikolaos Laoutaris, Hao Che, Ioannis Stavrakakis. The LCD interconnection of LRU caches and its analysis. Performance Evaluation, 63(7) (July 2006), 609\u2013634.","DOI":"10.1016\/j.peva.2005.05.003"},{"key":"10.3233\/IDT-190152_ref38","unstructured":"Wong TM, Wilkes J. My cache or yours? Making storage more exclusive. Usenix Association Proceedings of the General Track, 2002, pp. 161\u2013175."},{"issue":"4","key":"10.3233\/IDT-190152_ref42","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1155\/2005\/513531","article-title":"Supporting user mobility through cache relocation","volume":"1","author":"Kwong Yuen","year":"2005","journal-title":"Mobile Information Systems"},{"issue":"1","key":"10.3233\/IDT-190152_ref43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2007\/371572","article-title":"Cooperative proxy caching for wireless base stations","volume":"3","author":"W. James","year":"2007","journal-title":"Mobile Information Systems"},{"key":"10.3233\/IDT-190152_ref44","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.ejor.2005.09.007","article-title":"The Karush-Kuhn-Tucker optimality conditions in an optimization problem with interval-valued objective function","volume":"176","author":"Wu","year":"2007","journal-title":"European Journal of Operational Research"},{"key":"10.3233\/IDT-190152_ref45","doi-asserted-by":"crossref","unstructured":"Johannes Jahn. Karush-Kuhn-Tucker conditions in set optimization. Journal of Optimization Theory and Applications, 172(3) (2017), 707.","DOI":"10.1007\/s10957-017-1066-7"},{"key":"10.3233\/IDT-190152_ref46","doi-asserted-by":"crossref","unstructured":"Xavier Gabaix. Zipf\u2019s law for cities: an explanation. The Quarterly Journal of Economics, 114(Issue 3) (August 1999), 739\u2013767.","DOI":"10.1162\/003355399556133"},{"key":"10.3233\/IDT-190152_ref51","doi-asserted-by":"crossref","unstructured":"Amar Abane, Mehammed Daoui, Samia Bouzefrane, Paul Muhlethaler. A lightweight forwarding strategy for named data networking in low-end IoT. Journal of Network and Computer Applications, Elsevier, 148 (2019), 102445.","DOI":"10.1016\/j.jnca.2019.102445"},{"key":"10.3233\/IDT-190152_ref52","doi-asserted-by":"crossref","unstructured":"Diab Hassan, KashaniAli, Nasri Ahmad. Cache replacement engine: A fuzzy logic approach. Proceedings of the 2009 International Conference on the Current Trends in Information Technology, CTIT 2009.","DOI":"10.1109\/CTIT.2009.5423141"},{"issue":"3","key":"10.3233\/IDT-190152_ref54","doi-asserted-by":"crossref","first-page":"1610","DOI":"10.1109\/TWC.2019.2894403","article-title":"Multi-Agent Reinforcement learning for efficient content caching in mobile D2D networks","volume":"18","author":"Jiang","year":"2019","journal-title":"IEEE Transactions on Wireless Communications"}],"container-title":["Intelligent Decision Technologies"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/IDT-190152","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:22:47Z","timestamp":1777454567000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/IDT-190152"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,6]]},"references-count":32,"journal-issue":{"issue":"4"},"URL":"https:\/\/doi.org\/10.3233\/idt-190152","relation":{},"ISSN":["1872-4981","1875-8843"],"issn-type":[{"value":"1872-4981","type":"print"},{"value":"1875-8843","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,6]]}}}