{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:36:58Z","timestamp":1760240218062,"version":"build-2065373602"},"reference-count":27,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2019,4,8]],"date-time":"2019-04-08T00:00:00Z","timestamp":1554681600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["17H01730"],"award-info":[{"award-number":["17H01730"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The demand for mobile data communication has been increasing owing to the diversification of its purposes and the increase in the number of mobile devices accessing mobile networks. Users are experiencing a degradation in communication quality due to mobile network congestion. Therefore, improving the bandwidth utilization efficiency of cellular infrastructure is crucial. We previously proposed a mobile data offloading protocol (MDOP) for improving the bandwidth utilization efficiency. Although this method balances a load of evolved node B by taking into consideration the content delay tolerance, accurately balancing the load is challenging. In this paper, we apply deep reinforcement learning to MDOP to solve the temporal locality of a traffic. Moreover, we examine and evaluate the concrete processing while considering a delay tolerance. A comparison of the proposed method and bandwidth utilization efficiency of MDOP showed that the proposed method reduced the network traffic in excess of the control target value by 35% as compared with the MDOP. Furthermore, the proposed method improved the data transmission ratio by the delay tolerance range. Consequently, the proposed method improved the bandwidth utilization efficiency by learning how to provide the bandwidth to the user equipment when MDOP cannot be used to appropriately balance a load.<\/jats:p>","DOI":"10.3390\/s19071674","type":"journal-article","created":{"date-parts":[[2019,4,8]],"date-time":"2019-04-08T11:54:52Z","timestamp":1554724492000},"page":"1674","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Delay-Tolerance-Based Mobile Data Offloading Using Deep Reinforcement Learning"],"prefix":"10.3390","volume":"19","author":[{"given":"Daisuke","family":"Mochizuki","sequence":"first","affiliation":[{"name":"Graduate School of Integrated Science and Technology, Shizuoka University, 3-5-1 Johoku, Naka-ku, Hamamatsu-shi, Shizuoka 432-8011, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7165-7531","authenticated-orcid":false,"given":"Yu","family":"Abiko","sequence":"additional","affiliation":[{"name":"Graduate School of Integrated Science and Technology, Shizuoka University, 3-5-1 Johoku, Naka-ku, Hamamatsu-shi, Shizuoka 432-8011, Japan"}]},{"given":"Takato","family":"Saito","sequence":"additional","affiliation":[{"name":"NTT DOCOMO, INC., Urban Sensing Research Group, Research Laboratories, 3-6 Hikari-no-oka, Yokosuka-shi, Kanagawa 239-8536, Japan"}]},{"given":"Daizo","family":"Ikeda","sequence":"additional","affiliation":[{"name":"NTT DOCOMO, INC., Urban Sensing Research Group, Research Laboratories, 3-6 Hikari-no-oka, Yokosuka-shi, Kanagawa 239-8536, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3921-4298","authenticated-orcid":false,"given":"Hiroshi","family":"Mineno","sequence":"additional","affiliation":[{"name":"Graduate School of Integrated Science and Technology, Shizuoka University, 3-5-1 Johoku, Naka-ku, Hamamatsu-shi, Shizuoka 432-8011, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2019,4,8]]},"reference":[{"key":"ref_1","unstructured":"Cisco (2017). Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2016\u20132021, Cisco."},{"key":"ref_2","unstructured":"Ministry of Internal Affairs and Communications (2017). Information and Communications in Japan ICT Service Usage Trends 2017."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bulut, E., and Szymanski, B.K. (2015, January 8\u201312). Understanding user behavior via mobile data analysis. Proceedings of the ICC Workshops, IEEE, London, UK.","DOI":"10.1109\/ICCW.2015.7247402"},{"key":"ref_4","unstructured":"3GPP (2010). Evolved Universal Terrestrial Radio Access Network (E-UTRAN); Self-Configuring and Self-Optimizing Network (SON) Use Cases and Solutions, 3GPP. 3GPP TR 36.902, 9.2.0."},{"key":"ref_5","first-page":"13","article-title":"Proposal of mobile data offloading protocol to reduce temporal locality of mobile data traffic","volume":"58","author":"Nishioka","year":"2017","journal-title":"IPSJ J."},{"key":"ref_6","first-page":"168","article-title":"Evaluation of mobile data offloading protocol with place wise offloading","volume":"59","author":"Machida","year":"2018","journal-title":"IPSJ J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/COMST.2014.2369742","article-title":"Data offloading techniques in cellular networks: A survey","volume":"17","author":"Rebecchi","year":"2015","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.dcan.2015.02.001","article-title":"The challenges of M2M massive access in wireless cellular networks","volume":"1","author":"Brial","year":"2015","journal-title":"Digit. Commun. Netw."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MCOM.2017.1600247CM","article-title":"A unified urban mobile cloud computing offloading mechanism for smart cities","volume":"55","author":"Mazza","year":"2017","journal-title":"IEEE Commun. Mag."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Timmer, Y., Pettersson, J., Hannu, H., Wang, M., and Johansson, I. (2014, January 18). Network assisted rate adaptation for conversational video over LTE, concept and performance evaluation. Proceedings of the 2014 ACM SINGCOMM Workshop on Capacity Sharing Workshop (ACM), Chicago, IL, USA.","DOI":"10.1145\/2630088.2630096"},{"key":"ref_11","unstructured":"3GPP (2015). Study on System Enhancements for User Plane Congestion Management, 3GPP. 3GPP TR 23.705."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Maia, A.M., de Castro, M.F., and Vieira, D. (2014, January 2\u20135). A dynamic lte uplink packet scheduler for machine-to-machine communication. Proceedings of the 2014 IEEE 25th Annual International Symposium on Personal, Indoor, and Mobile Radio Communication (PIMRC), Washington, DC, USA.","DOI":"10.1109\/PIMRC.2014.7136425"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Arulkumaran, K., Deisenroth, M.P., Brundage, M., and Bharath, A.A. (arXiv, 2017). A brief survey of deep reinforcement learning, arXiv.","DOI":"10.1109\/MSP.2017.2743240"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1561\/2200000071","article-title":"An introduction to deep reinforcement learning","volume":"11","author":"Henderson","year":"2018","journal-title":"Found. Trends Mach. Learn."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of Go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ghadimi, E., Calabrese, F.D., Peters, G., and Soldati, P. (2017, January 21\u201325). A reinforcement learning approach to power control and rate adaptation in cellular networks. Proceedings of the 2017 IEEE International Conference on Communications (ICC), Paris, France.","DOI":"10.1109\/ICC.2017.7997440"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1625","DOI":"10.1587\/transcom.2017CQP0014","article-title":"A deep reinforcement learning based approach for cost- and energy-aware multi-flow mobile data offloading","volume":"7","author":"Zhang","year":"2018","journal-title":"IEICE Trans. Commun."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Bennis, M., and Niyato, D. (2010, January 6\u201310). A Q-learning based approach to interference avoidance in self-organized femtocell networks. Proceedings of the GLOBECOM Workshops (GC Workshps); IEEE, Miami, FL, USA.","DOI":"10.1109\/GLOCOMW.2010.5700414"},{"key":"ref_20","first-page":"5","article-title":"Deep reinforcement learning with double Q-learning","volume":"2","author":"Guez","year":"2016","journal-title":"Assoc. Adv. Artif. Intell."},{"key":"ref_21","unstructured":"Watkins, C.J.C.H. (1989). Learning from Delayed Rewards. [Ph.D. Thesis, King\u2019s College]."},{"key":"ref_22","unstructured":"Wang, Z., Schaul, T., Hessel, M., Van Hasselt, H., Lanctot, M., and De Freitas, N. (arXiv, 2015). Dueling network architectures for deep reinforcement learning, arXiv."},{"key":"ref_23","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the ICML, Lille, France."},{"key":"ref_24","unstructured":"(2019, January 11). Space Time Engineering. Available online: https:\/\/www.spacetime-eng.com\/en\/."},{"key":"ref_25","unstructured":"3GPP (2016). LTE; Evolved Universal Terrestrial Radio Access (E-UTRA); Radio Frequency (RF) System Scenarios, 3GPP. 3GPP TR 36.942, 13.0.0."},{"key":"ref_26","unstructured":"NGMN Alliance (2019, April 08). NGMN Radio Access Performance Evaluation Methodology. Available online: https:\/\/www.ngmn.org\/fileadmin\/user_upload\/NGMN_Radio_Access_Performance_Evaluation_Methodology.pdf."},{"key":"ref_27","unstructured":"Statista (2019, April 08). Mobile Data Traffic Worldwide by Application Category from 2016 to 2021 (in Terabytes per Month). Available online: https:\/\/www.statista.com\/statistics\/454951\/mobile-data-traffic-worldwide-by-application-category\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/7\/1674\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:43:44Z","timestamp":1760186624000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/7\/1674"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,8]]},"references-count":27,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2019,4]]}},"alternative-id":["s19071674"],"URL":"https:\/\/doi.org\/10.3390\/s19071674","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,4,8]]}}}