{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T04:28:04Z","timestamp":1772252884924,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,1,19]],"date-time":"2022-01-19T00:00:00Z","timestamp":1642550400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008205","name":"Auckland University of Technology","doi-asserted-by":"publisher","award":["16941452"],"award-info":[{"award-number":["16941452"]}],"id":[{"id":"10.13039\/100008205","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The Fifth Generation (5G) mobile networks use millimeter waves (mmWaves) to offer gigabit data rates. However, unlike microwaves, mmWave links are prone to user and topographic dynamics. They easily get blocked and end up forming irregular cell patterns for 5G. This in turn causes too early, too late, or wrong handoffs (HOs). To mitigate HO challenges, sustain connectivity, and avert unnecessary HO, we propose an HO scheme based on a jump Markov linear system (JMLS) and deep reinforcement learning (DRL). JMLS is widely known to account for abrupt changes in system dynamics. DRL likewise emerges as an artificial intelligence technique for learning highly dimensional and time-varying behaviors. We combine the two techniques to account for time-varying, abrupt, and irregular changes in mmWave link behavior by predicting likely deterioration patterns of target links. The prediction is optimized by meta training techniques that also reduce training sample size. Thus, the JMLS\u2013DRL platform formulates intelligent and versatile HO policies for 5G. When compared to a signal and interference noise ratio (SINR) and DRL-based HO scheme, our HO scheme becomes more reliable in selecting reliable target links. In particular, our proposed scheme is able to reduce wasteful HO to less than 5% within 200 training episodes compared to the DRL-based HO scheme that needs more than 200 training episodes to get to less than 5%. It supports longer dew time between HOs and high sum rates by ably averting unnecessary HOs with almost half the HOs compared to a DRL-based HO scheme.<\/jats:p>","DOI":"10.3390\/s22030746","type":"journal-article","created":{"date-parts":[[2022,1,19]],"date-time":"2022-01-19T21:01:51Z","timestamp":1642626111000},"page":"746","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Enhancing Handover for 5G mmWave Mobile Networks Using Jump Markov Linear System and Deep Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4839-905X","authenticated-orcid":false,"given":"Masoto","family":"Chiputa","sequence":"first","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Auckland University of Technology, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3993-1120","authenticated-orcid":false,"given":"Minglong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Auckland University of Technology, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5861-0475","authenticated-orcid":false,"given":"G. G. Md. Nawaz","family":"Ali","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Systems, Bradley University, Peoria, IL 61625, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5375-8961","authenticated-orcid":false,"given":"Peter Han Joo","family":"Chong","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Auckland University of Technology, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9358-7733","authenticated-orcid":false,"given":"Hakilo","family":"Sabit","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Auckland University of Technology, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4309-3396","authenticated-orcid":false,"given":"Arun","family":"Kumar","sequence":"additional","affiliation":[{"name":"Department of Computer Science & Engineering, National Institute of Technology, Rourkela 769008, Odisha, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hui","family":"Li","sequence":"additional","affiliation":[{"name":"Shenzhen Graduate School, Peking University, Shenzhen 518055, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Rebato, M., Polese, M., and Zorzi, M. (2018, January 9\u201313). Multi-Sector and Multi-Panel Performance in 5G mmWave Cellular Networks. Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/GLOCOM.2018.8647528"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1109\/JPROC.2014.2299397","article-title":"Millimeter-Wave Cellular Wireless Networks: Potentials and Challenges","volume":"102","author":"Rangan","year":"2014","journal-title":"Proc. IEEE"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4377","DOI":"10.1109\/JIOT.2018.2876298","article-title":"Joint load balancing and offloading in vehicular edge computing and networks","volume":"6","author":"Dai","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Mwanje, S., Zia, N., and Mitschele-Thiel, A. (2012, January 28\u201331). Self-Organized Handover Parameter Configuration for LTE. Proceedings of the 9th International Symposium on Wireless Communication Systems (ISWCS\u201912), Paris, France.","DOI":"10.1109\/ISWCS.2012.6328323"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Shubyn, B., and Maksymyuk, T. (2019, January 2\u20136). Intelligent Handover Management in 5G Mobile Networks based on Recurrent Neural Networks. Proceedings of the 2019 3rd International Conference on Advanced Information and Communications Technologies (AICT), Lviv, Ukraine.","DOI":"10.1109\/AIACT.2019.8847734"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Joud, M., Garc\u00eda-Lozano, M., and Ruiz, S. (2018, January 6\u20138). User specific cell clustering to improve mobility robustness in 5G ultra-dense cellular networks. Proceedings of the 2018 14th Annual Conference on Wireless On-Demand Network Systems and Services (WONS), Isola, France.","DOI":"10.23919\/WONS.2018.8311661"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"8402","DOI":"10.1109\/TIT.2013.2281606","article-title":"Femtocaching: Wireless content delivery through distributed caching helpers","volume":"59","author":"Shanmugam","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1109\/TRO.2010.2044948","article-title":"A probabilistic particle-control approximation of chance-constrained stochastic predictive control","volume":"26","author":"Blackmore","year":"2010","journal-title":"IEEE Trans. Robot."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.sysconle.2014.10.008","article-title":"On control of discrete-time state-dependent jump linear systems with probabilistic constraints: A receding horizon approach","volume":"74","author":"Chitraganti","year":"2014","journal-title":"Syst. Control Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1444","DOI":"10.1109\/JSAC.2017.2698859","article-title":"How Much Spectrum is too Much in Millimeter Wave Wireless Access","volume":"35","author":"Du","year":"2017","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"953","DOI":"10.1109\/TITS.2017.2771519","article-title":"Dependable content distribution in D2D-based cooperative vehicular networks: A big data-integrated coalition game approach","volume":"19","author":"Zhou","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"768","DOI":"10.1109\/TII.2017.2733001","article-title":"Social big-data-based content dissemination in Internet of Vehicles","volume":"14","author":"Zhou","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1287","DOI":"10.1109\/TC.2018.2818144","article-title":"Cloudlets activation scheme for scalable mobile edge computing with transmission power control and virtual machine migration","volume":"67","author":"Rodrigues","year":"2018","journal-title":"IEEE Trans. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Maksymyuk, T., Han, L., Larionov, S., Shubyn, B., Luntovskyy, A., and Klymash, M. (March, January 26). Intelligent Spectrum Management in 5G Mobile Networks based on Recurrent Neural Networks. Proceedings of the 15th IEEE International Conference The Experience of Designing and Application of CADSystems (IEEE CADSM\u20192019), Polyana, Ukraine.","DOI":"10.1109\/CADSM.2019.8779301"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"810","DOI":"10.1109\/TC.2016.2620469","article-title":"Hybrid method for minimizing service delay in edge cloud computing through vm migration and transmission power control","volume":"66","author":"Rodrigues","year":"2017","journal-title":"IEEE Trans. Comput."},{"key":"ref_16","unstructured":"Edwards, C. (1973). Advanced Calculus of Several Variables, Dover Publications."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1080\/1055678031000098773","article-title":"Pennonacode for convex nonlinear and semi definite programming","volume":"8","author":"Kocvara","year":"2003","journal-title":"Optim. Methods Softw."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum likelihood from incomplete data via the EM algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"J. R. Stat. Soc. Ser. B (Methodol.)"},{"key":"ref_19","first-page":"126","article-title":"A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models","volume":"4","author":"Bilmes","year":"1998","journal-title":"Int. Comput. Sci. Inst."},{"key":"ref_20","unstructured":"Costa, O.L.V., Fragoso, M.D., and Marques, R.P. (2006). Discrete-Time Markov Jump Linear Systems, Springer Science & Business Media."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5087","DOI":"10.1109\/TVT.2019.2905432","article-title":"Energy-efficient edge computing service provisioning for vehicular networks: A consensus admm approach","volume":"68","author":"Zhou","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"8472","DOI":"10.1109\/TVT.2019.2927634","article-title":"Workload scheduling in vehicular networks with edge cloud capabilities","volume":"68","author":"Sorkhoh","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_23","first-page":"529","article-title":"On certain integrals of Lipschitz-Hankel type involving products of Bessel functions","volume":"A247","author":"Eason","year":"1955","journal-title":"Phil. Trans. R. Soc. Lond."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gaw\u0142owicz, P., and Zubow, A. (2019, January 25\u201329). Ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research. Proceedings of the ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, Miami Beach, FL, USA.","DOI":"10.1145\/3345768.3355908"},{"key":"ref_25","unstructured":"Glen, S. (2021, November 20). Kaiser-Meyer-Olkin (KMO) Test for Sampling Adequacy, From StatisticsHowTo.com: Elementary Statistics for the Rest of Us!. Available online: https:\/\/www.statisticshowto.com\/kaiser-meyer-olkin\/."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3319","DOI":"10.1109\/TWC.2020.2972352","article-title":"ViterbiNet: A Deep Learning Based Viterbi Algorithm for Symbol Detection","volume":"19","author":"Shlezinger","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_27","first-page":"1092","article-title":"Road tracking using deep reinforcement learning for self-driving car applications","volume":"108","author":"Han","year":"2018","journal-title":"Futur. Gener. Comput. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yang, P., Chen, L., Zhang, H., Yang, J., Wang, R., and Li, Z. (2021). Joint Optical and Wireless Resource Allocation for Cooperative Transmission in C-RAN. Sensors, 21.","DOI":"10.3390\/s21010217"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"802","DOI":"10.1109\/TCCN.2019.2961655","article-title":"Handover Management for mmWave Networks with Proactive Performance Prediction Using Camera Images and Deep Reinforcement Learning","volume":"6","author":"Koda","year":"2020","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ho, T.M., and Nguyen, K.-K. (2021, January 14\u201323). Deep Q-Learning for Joint Server Selection, Offloading, and Handover in Multi-access Edge Computing. Proceedings of the ICC 2021\u2014IEEE International Conference on Communications, Montreal, QC, Canada.","DOI":"10.1109\/ICC42927.2021.9500353"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"13124","DOI":"10.1109\/TVT.2020.3020400","article-title":"Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning","volume":"69","author":"Guo","year":"2020","journal-title":"IEEE Trans. Veh. Technol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/746\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:03:59Z","timestamp":1760133839000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/746"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,19]]},"references-count":31,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["s22030746"],"URL":"https:\/\/doi.org\/10.3390\/s22030746","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202111.0514.v1","asserted-by":"object"}]},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,19]]}}}