{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T20:47:23Z","timestamp":1761598043901,"version":"3.37.3"},"reference-count":19,"publisher":"Wiley","license":[{"start":{"date-parts":[[2019,2,27]],"date-time":"2019-02-27T00:00:00Z","timestamp":1551225600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"crossref","award":["2018YFC0809803","61702364"],"award-info":[{"award-number":["2018YFC0809803","61702364"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2018YFC0809803","61702364"],"award-info":[{"award-number":["2018YFC0809803","61702364"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Wireless Communications and Mobile Computing"],"published-print":{"date-parts":[[2019,2,27]]},"abstract":"<jats:p>Edge caching is a promising method to deal with the traffic explosion problem towards future network. In order to satisfy the demands of user requests, the contents can be proactively cached locally at the proximity to users (e.g., base stations or user device). Recently, some learning-based edge caching optimizations are discussed. However, most of the previous studies explore the influence of dynamic and constant expanding action and caching space, leading to unpracticality and low efficiency. In this paper, we study the edge caching optimization problem by utilizing the Double Deep Q-network (Double DQN) learning framework to maximize the hit rate of user requests. Firstly, we obtain the Device-to-Device (D2D) sharing model by considering both online and offline factors and then we formulate the optimization problem, which is proved as NP-hard. Then the edge caching replacement problem is derived by Markov decision process (MDP). Finally, an edge caching strategy based on Double DQN is proposed. The experimental results based on large-scale actual traces show the effectiveness of the proposed framework.<\/jats:p>","DOI":"10.1155\/2019\/2561069","type":"journal-article","created":{"date-parts":[[2019,2,27]],"date-time":"2019-02-27T18:36:25Z","timestamp":1551292585000},"page":"1-12","source":"Crossref","is-referenced-by-count":9,"title":["Edge Caching for D2D Enabled Hierarchical Wireless Networks with Deep Reinforcement Learning"],"prefix":"10.1155","volume":"2019","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0225-4122","authenticated-orcid":true,"given":"Wenkai","family":"Li","sequence":"first","affiliation":[{"name":"College of Intelligence and Computing, Tianjin University, 300350 Tianjin, China"}]},{"given":"Chenyang","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Intelligence and Computing, Tianjin University, 300350 Tianjin, China"}]},{"given":"Ding","family":"Li","sequence":"additional","affiliation":[{"name":"College of Intelligence and Computing, Tianjin University, 300350 Tianjin, China"}]},{"given":"Bin","family":"Hu","sequence":"additional","affiliation":[{"name":"Department of Network Engineering, Technical College for Deaf, Tianjin University of Technology, 300000 Tianjin, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7223-1030","authenticated-orcid":true,"given":"Xiaofei","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Intelligence and Computing, Tianjin University, 300350 Tianjin, China"}]},{"given":"Jianji","family":"Ren","sequence":"additional","affiliation":[{"name":"Institute of Computer Science and Technology, Henan Polytechnic University, 454150 Jiaozuo, Henan, China"}]}],"member":"311","reference":[{"key":"2","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2016.2545413"},{"key":"4","doi-asserted-by":"publisher","DOI":"10.1109\/TCOMM.2017.2713384"},{"key":"5","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2015.2442529"},{"key":"6","doi-asserted-by":"publisher","DOI":"10.1109\/TCOMM.2016.2536728"},{"key":"7","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2016.2614952"},{"key":"8","doi-asserted-by":"publisher","DOI":"10.1109\/mcom.2014.6736753"},{"key":"9","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2016.7537179"},{"key":"10","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2016.7565185"},{"key":"11","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2013.6495773"},{"key":"14","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2013.2281606"},{"key":"19","doi-asserted-by":"publisher","DOI":"10.1109\/90.944339"},{"key":"20","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2002.801752"},{"key":"21","doi-asserted-by":"publisher","DOI":"10.1109\/TCOMM.2016.2545655"},{"key":"23","doi-asserted-by":"publisher","DOI":"10.1109\/mcom.2014.6871674"},{"key":"24","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2008.918081"},{"key":"30","doi-asserted-by":"publisher","DOI":"10.1007\/s11036-017-0927-5"},{"key":"26","doi-asserted-by":"publisher","DOI":"10.1109\/tnet.2015.2487344"},{"year":"2001","key":"27"},{"key":"29","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2018.2844658"}],"container-title":["Wireless Communications and Mobile Computing"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/wcmc\/2019\/2561069.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/wcmc\/2019\/2561069.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/wcmc\/2019\/2561069.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,4,2]],"date-time":"2019-04-02T21:11:03Z","timestamp":1554239463000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.hindawi.com\/journals\/wcmc\/2019\/2561069\/"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,27]]},"references-count":19,"alternative-id":["2561069","2561069"],"URL":"https:\/\/doi.org\/10.1155\/2019\/2561069","relation":{},"ISSN":["1530-8669","1530-8677"],"issn-type":[{"type":"print","value":"1530-8669"},{"type":"electronic","value":"1530-8677"}],"subject":[],"published":{"date-parts":[[2019,2,27]]}}}