{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T09:12:41Z","timestamp":1780045961199,"version":"3.53.1"},"reference-count":41,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2022,11,26]],"date-time":"2022-11-26T00:00:00Z","timestamp":1669420800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Culture, Sports and Tourism","award":["R2022020116"],"award-info":[{"award-number":["R2022020116"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Hardware bottlenecks can throttle smart device (SD) performance when executing computation-intensive and delay-sensitive applications. Hence, task offloading can be used to transfer computation-intensive tasks to an external server or processor in Mobile Edge Computing. However, in this approach, the offloaded task can be useless when a process is significantly delayed or a deadline has expired. Due to the uncertain task processing via offloading, it is challenging for each SD to determine its offloading decision (whether to local or remote and drop). This study proposes a deep-reinforcement-learning-based offloading scheduler (DRL-OS) that considers the energy balance in selecting the method for performing a task, such as local computing, offloading, or dropping. The proposed DRL-OS is based on the double dueling deep Q-network (D3QN) and selects an appropriate action by learning the task size, deadline, queue, and residual battery charge. The average battery level, drop rate, and average latency of the DRL-OS were measured in simulations to analyze the scheduler performance. The DRL-OS exhibits a lower average battery level (up to 54%) and lower drop rate (up to 42.5%) than existing schemes. The scheduler also achieves a lower average latency of 0.01 to &gt;0.25 s, despite subtle case-wise differences in the average latency.<\/jats:p>","DOI":"10.3390\/s22239212","type":"journal-article","created":{"date-parts":[[2022,11,28]],"date-time":"2022-11-28T08:13:09Z","timestamp":1669623189000},"page":"9212","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["DRL-OS: A Deep Reinforcement Learning-Based Offloading Scheduler in Mobile Edge Computing"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4361-653X","authenticated-orcid":false,"given":"Ducsun","family":"Lim","sequence":"first","affiliation":[{"name":"The Department of Computer and Software, Hanyang University, 222 Wangsimni-ro, Seoul 04763, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9688-6691","authenticated-orcid":false,"given":"Wooyeob","family":"Lee","sequence":"additional","affiliation":[{"name":"The Department of Computer and Software, Hanyang University, 222 Wangsimni-ro, Seoul 04763, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3426-3792","authenticated-orcid":false,"given":"Won-Tae","family":"Kim","sequence":"additional","affiliation":[{"name":"The Department of Computer Science and Engineering, Korea University of Technology and Education, Cheonan-si 31253, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Inwhee","family":"Joe","sequence":"additional","affiliation":[{"name":"The Department of Computer and Software, Hanyang University, 222 Wangsimni-ro, Seoul 04763, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"9073","DOI":"10.1109\/TVT.2018.2865211","article-title":"Delay-tolerant data traffic to software-defined vehicular networks with mobile edge computing in smart city","volume":"67","author":"Li","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1815","DOI":"10.1109\/JIOT.2019.2957351","article-title":"Edge cache-assisted secure low-latency millimeter-wave transmission","volume":"7","author":"Hao","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"69200","DOI":"10.1109\/ACCESS.2020.2986329","article-title":"Wearables and the Internet of Things (IoT), applications, opportunities, and challenges: A Survey","volume":"8","author":"Dian","year":"2020","journal-title":"IEEE Access."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2322","DOI":"10.1109\/COMST.2017.2745201","article-title":"A survey on mobile edge computing: The communication perspective","volume":"19","author":"Mao","year":"2017","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/j.comcom.2020.05.001","article-title":"A smartphone perspective on computation offloading\u2014A survey","volume":"159","author":"Nguyen","year":"2020","journal-title":"Comput. Commun."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"771","DOI":"10.1109\/TMC.2018.2847337","article-title":"Dynamic computation offloading for mobile cloud computing: A stochastic game-theoretic approach","volume":"18","author":"Zheng","year":"2018","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hamdan, S., Ayyash, M., and Almajali, S. (2020). Edge-computing architectures for internet of things applications: A survey. Sensors, 20.","DOI":"10.3390\/s20226441"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2961","DOI":"10.1109\/COMST.2018.2849509","article-title":"Survey on multi-access edge computing for internet of things realization","volume":"20","author":"Porambage","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"6571","DOI":"10.1109\/ACCESS.2020.3049094","article-title":"Planning the future of smart cities with swarms of fully autonomous unmanned aerial vehicles using a novel framework","volume":"9","author":"Kuru","year":"2021","journal-title":"IEEE Access."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Eom, H., Juste, P.S., Figueiredo, R., Tickoo, O., Illikkal, R., and Iyer, R. (2013, January 9\u201312). Machine learning-based runtime scheduler for mobile offloading framework. Proceedings of the 6th International Conference on Utility and Cloud Computing, Washington, DC, USA.","DOI":"10.1109\/UCC.2013.21"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4924","DOI":"10.1109\/TWC.2017.2703901","article-title":"Computation offloading and resource allocation in wireless cellular networks with mobile edge computing","volume":"16","author":"Wang","year":"2017","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"149623","DOI":"10.1109\/ACCESS.2019.2947053","article-title":"A deep learning approach for energy efficient computational offloading in mobile edge computing","volume":"7","author":"Ali","year":"2019","journal-title":"IEEE Access."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1985","DOI":"10.1109\/TMC.2020.3036871","article-title":"Deep reinforcement learning for task offloading in mobile edge computing systems","volume":"21","author":"Tang","year":"2020","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, J., Mao, Y., Zhang, J., and Letaief, K.B. (2016, January 10\u201315). Delay-optimal computation task scheduling for mobile-edge computing systems. Proceedings of the IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain.","DOI":"10.1109\/ISIT.2016.7541539"},{"key":"ref_15","unstructured":"Jia, M., Cao, J., and Yang, L. (May, January 27). Heuristic offloading of concurrent tasks for computation-intensive applications in mobile cloud computing. Proceedings of the IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Toronto, Canada."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1481","DOI":"10.1109\/LCOMM.2017.2690678","article-title":"Latency aware workload offloading in the cloudlet network","volume":"21","author":"Sun","year":"2017","journal-title":"IEEE Commun. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3864","DOI":"10.1109\/JIOT.2019.2892398","article-title":"Adaptive service offloading for revenue maximization in mobile edge computing with delay-constraint","volume":"6","author":"Samanta","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1109\/WCL.2013.122113.130825","article-title":"Energy-efficient link selection and transmission scheduling in mobile cloud computing","volume":"3","author":"Xiang","year":"2014","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4569","DOI":"10.1109\/TWC.2013.072513.121842","article-title":"Energy-optimal mobile cloud computing under stochastic wireless channel","volume":"12","author":"Zhang","year":"2013","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2516","DOI":"10.1109\/TMC.2015.2405539","article-title":"Offloading in mobile cloudlet systems with intermittent connectivity","volume":"14","author":"Zhang","year":"2015","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2651","DOI":"10.1109\/TNET.2018.2873002","article-title":"An efficient computation offloading management scheme in the densely deployed small cell networks with mobile edge computing","volume":"26","author":"Guo","year":"2018","journal-title":"IEEE ACM Trans. Netw."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"6398","DOI":"10.1109\/TVT.2018.2799620","article-title":"Mobile edge computing empowered energy efficient task offloading in 5G","volume":"67","author":"Yang","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_23","first-page":"89","article-title":"Joint optimization of radio and computational resources for multicell mobile-edge computing","volume":"1","author":"Sardellitti","year":"2015","journal-title":"IEEE Trans. Signal Inf. Process. Netw."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2603","DOI":"10.1109\/TCOMM.2018.2799937","article-title":"Energy-efficient admission of delay-sensitive tasks for mobile edge computing","volume":"66","author":"Lyu","year":"2018","journal-title":"IEEE Trans. Commun."},{"key":"ref_25","unstructured":"Eshraghi, N., and Liang, B. (May, January 29). Joint offloading decision and resource allocation with uncertain task computing requirement. Proceedings of the IEEE Conference on Computer Communications, Paris, France."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2762","DOI":"10.1109\/TNET.2018.2876941","article-title":"A distributed computation offloading strategy in small-cell networks integrated with mobile edge computing","volume":"26","author":"Yang","year":"2018","journal-title":"IEEE ACM Trans. Netw."},{"key":"ref_27","unstructured":"Alpaydin, E. (2020). Introduction to Machine Learning, MIT Press."},{"key":"ref_28","first-page":"6382","article-title":"Multi-agent actor-critic for mixed cooperative-competitive environments","volume":"30","author":"Lowe","year":"2017","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_30","unstructured":"Hausknecht, M., and Stone, P. (2015). Deep reinforcement learning in parameterized action space. arXiv."},{"key":"ref_31","first-page":"30","article-title":"Residual algorithms: Reinforcement learning with function approximation","volume":"Volume 1995","author":"Baird","year":"1995","journal-title":"Machine Learning Proceedings"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016). Deep reinforcement learning with double q-learning. arXiv.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_33","unstructured":"Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016, January 19\u201324). Dueling network architectures for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2276","DOI":"10.1109\/TMC.2016.2613864","article-title":"Offloading in HetNet: A coordination of interference mitigation, user association, and resource allocation","volume":"16","author":"Oo","year":"2016","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Dos Anjos, J.C.S., Gross, J.L.G., Matteussi, K.J., Gonz\u00e1lez, G.V., Leithardt, V.R.Q., and Geyer, C.F.R. (2021). An algorithm to minimize energy consumption and elapsed time for IoT workloads in a hybrid architecture. Sensors, 21.","DOI":"10.3390\/s21092914"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"5031","DOI":"10.1109\/TVT.2019.2904244","article-title":"Collaborative cloud and edge computing for latency minimization","volume":"68","author":"Ren","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1109\/TMC.2018.2831230","article-title":"Energy-efficient dynamic computation offloading and cooperative task scheduling in mobile cloud computing","volume":"18","author":"Guo","year":"2018","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1027","DOI":"10.1109\/TGCN.2018.2846524","article-title":"Delay and energy efficiency tradeoff for information pushing system","volume":"2","author":"Nath","year":"2018","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., and Silver, D. (2018, January 2\u20137). Rainbow: Combining improvements in deep reinforcement learning. Proceedings of the AAAI Thirty-second AAAI conference on artificial intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11796"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Sheng, S., Chen, P., Chen, Z., Wu, L., and Yao, Y. (2021). Deep reinforcement learning-based task scheduling in iot edge computing. Sensors, 21.","DOI":"10.3390\/s21051666"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Venieris, S.I., Panopoulos, I., and Venieris, I.S. (2021, January 23\u201327). OODIn: An optimised on-device inference framework for heterogeneous mobile devices. Proceedings of the IEEE International Conference on Smart Computing (SMARTCOMP), Irvine, CA, USA.","DOI":"10.1109\/SMARTCOMP52413.2021.00021"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/23\/9212\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:27:37Z","timestamp":1760146057000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/23\/9212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,26]]},"references-count":41,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["s22239212"],"URL":"https:\/\/doi.org\/10.3390\/s22239212","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,26]]}}}