{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:58:16Z","timestamp":1760241496566,"version":"build-2065373602"},"reference-count":34,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2018,3,29]],"date-time":"2018-03-29T00:00:00Z","timestamp":1522281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Networks will continue to become increasingly heterogeneous as we move toward 5G. Meanwhile, the intelligent programming of the core network makes the available radio resource be more changeable rather than static. In such a dynamic and heterogeneous network environment, how to help terminal users select optimal networks to access is challenging. Prior implementations of network selection are usually applicable for the environment with static radio resources, while they cannot handle the unpredictable dynamics in 5G network environments. To this end, this paper considers both the fluctuation of radio resources and the variation of user demand. We model the access network selection scenario as a multiagent coordination problem, in which a bunch of rationally terminal users compete to maximize their benefits with incomplete information about the environment (no prior knowledge of network resource and other users\u2019 choices). Then, an adaptive learning based strategy is proposed, which enables users to adaptively adjust their selections in response to the gradually or abruptly changing environment. The system is experimentally shown to converge to Nash equilibrium, which also turns out to be both Pareto optimal and socially optimal. Extensive simulation results show that our approach achieves significantly better performance compared with two learning and non-learning based approaches in terms of load balancing, user payoff and the overall bandwidth utilization efficiency. In addition, the system has a good robustness performance under the condition with non-compliant terminal users.<\/jats:p>","DOI":"10.3390\/e20040236","type":"journal-article","created":{"date-parts":[[2018,3,29]],"date-time":"2018-03-29T12:51:56Z","timestamp":1522327916000},"page":"236","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["An Adaptive Learning Based Network Selection Approach for 5G Dynamic Environments"],"prefix":"10.3390","volume":"20","author":[{"given":"Xiaohong","family":"Li","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Tianjin University, Tianjin 300000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ru","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Tianjin University, Tianjin 300000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianye","family":"Hao","sequence":"additional","affiliation":[{"name":"School of Software, Tianjin University, Tianjin 300000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,3,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1109\/MCOM.2014.6736752","article-title":"Cellular architecture and key technologies for 5G wireless communication networks","volume":"52","author":"Wang","year":"2014","journal-title":"Commun. Mag. IEEE"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1065","DOI":"10.1109\/JSAC.2014.2328098","article-title":"What will 5G be?","volume":"32","author":"Andrews","year":"2014","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1109\/MWC.2014.6812298","article-title":"Emerging technologies and research challenges for 5G wireless networks","volume":"21","author":"Chin","year":"2014","journal-title":"IEEE Wirel. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1212","DOI":"10.1109\/SURV.2012.010912.00081","article-title":"Game theory-based network selection: Solutions and challenges","volume":"14","author":"Trestian","year":"2012","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Martinez-Morales, J.D., Pineda-Rico, U., and Stevens-Navarro, E. (2010, January 8\u201310). Performance comparison between MADM algorithms for vertical handoff in 4G networks. Proceedings of the 2010 7th International Conference on Electrical Engineering Computing Science and Automatic Control (CCE), Tuxtla Gutierrez, Mexico.","DOI":"10.1109\/ICEEE.2010.5608646"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lahby, M., Cherkaoui, L., and Adib, A. (2013, January 6\u20138). An enhanced-TOPSIS based network selection technique for next generation wireless networks. Proceedings of the 2013 20th International Conference on Telecommunications (ICT), Casablanca, Morocco.","DOI":"10.1109\/ICTEL.2013.6632067"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lahby, M., and Adib, A. (2013, January 23\u201325). Network selection mechanism by using M-AHP\/GRA for heterogeneous networks. Proceedings of the 2013 6th Joint IFIP Wireless and Mobile Networking Conference (WMNC), Dubai, UAE.","DOI":"10.1109\/WMNC.2013.6549009"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1109\/SURV.2012.010912.00044","article-title":"Mathematical modeling for network selection in heterogeneous wireless networks\u2014A tutorial","volume":"15","author":"Wang","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2008","DOI":"10.1109\/TVT.2008.2004588","article-title":"Dynamics of network selection in heterogeneous wireless networks: An evolutionary game approach","volume":"58","author":"Niyato","year":"2009","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1109\/TVT.2015.2394431","article-title":"Traffic-aware online network selection in heterogeneous wireless networks","volume":"65","author":"Wu","year":"2016","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_11","first-page":"306","article-title":"Distributed learning of equilibria with incomplete, dynamic, and uncertain information in wireless communication networks","volume":"4","author":"Xu","year":"2016","journal-title":"Prov. Med. J. Retrosp. Med. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1007\/s11036-017-0885-y","article-title":"Dynamic Provider Selection & Power Resource Management in Competitive Wireless Communication Markets","volume":"23","author":"Vamvakas","year":"2018","journal-title":"Mob. Netw. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Tsiropoulou, E.E., Katsinis, G.K., Filios, A., and Papavassiliou, S. (2014). On the Problem of Optimal Cell Selection and Uplink Power Control in Open Access Multi-service Two-Tier Femtocell Networks. Lecture Notes in Computer Science Description, Springer.","DOI":"10.1007\/978-3-319-07425-2_9"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1145\/2517329","article-title":"Achieving socially optimal outcomes in multiagent systems with reinforcement social learning","volume":"8","author":"Hao","year":"2013","journal-title":"ACM Trans. Auton. Adapt. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2427","DOI":"10.1109\/TMC.2012.207","article-title":"Network selection and resource allocation games for wireless access networks","volume":"12","author":"Malanchini","year":"2013","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Aryafar, E., Keshavarz-Haddad, A., Wang, M., and Chiang, M. (2013, January 14\u201319). RAT selection games in HetNets. Proceedings of the 2013 IEEE INFOCOM, Turin, Italy.","DOI":"10.1109\/INFCOM.2013.6566889"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Monsef, E., Keshavarz-Haddad, A., Aryafar, E., Saniie, J., and Chiang, M. (May, January 26). Convergence properties of general network selection games. Proceedings of the 2015 IEEE Conference on Computer Communications (INFOCOM), Hong Kong, China.","DOI":"10.1109\/INFOCOM.2015.7218522"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement learning: A survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/TSMCC.2007.913919","article-title":"A comprehensive survey of multiagent reinforcement learning","volume":"38","author":"Busoniu","year":"2008","journal-title":"IEEE Trans. Syst. Man Cybern. C"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Barve, S.S., and Kulkarni, P. (2012, January 18\u201320). Dynamic channel selection and routing through reinforcement learning in cognitive radio networks. Proceedings of the 2012 IEEE International Conference on Computational Intelligence Computing Research, Coimbatore, India.","DOI":"10.1109\/ICCIC.2012.6510175"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Xu, Y., Chen, J., Ma, L., and Lang, G. (2014, January 18\u201321). Q-Learning Based Network Selection for WCDMA\/WLAN Heterogeneous Wireless Networks. Proceedings of the 2014 IEEE 79th Vehicular Technology Conference (VTC Spring), Seoul, Korea.","DOI":"10.1109\/VTCSpring.2014.7023063"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Kittiwaytang, K., Chanloha, P., and Aswakul, C. (2010, January 28\u201330). CTM-Based Reinforcement Learning Strategy for Optimal Heterogeneous Wireless Network Selection. Proceedings of the 2010 Second International Conference on Computational Intelligence, Modelling and Simulation (CIMSiM), Tuban, Indonesia.","DOI":"10.1109\/CIMSiM.2010.47"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/MVT.2013.2269187","article-title":"5G on the horizon: Key challenges for the radio-access network","volume":"8","author":"Demestichas","year":"2013","journal-title":"IEEE Veh. Technol. Mag."},{"key":"ref_24","first-page":"58","article-title":"Green Base Station Solutions and Technology","volume":"9","author":"Chen","year":"2011","journal-title":"Zte Commun."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Oh, E., and Krishnamachari, B. (2010, January 6\u201310). Energy savings through dynamic base station switching in cellular wireless access networks. Proceedings of the 2010 IEEE Global Telecommunications Conference (GLOBECOM 2010), Miami, FL, USA.","DOI":"10.1109\/GLOCOM.2010.5683654"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.engappai.2016.11.008","article-title":"The dynamics of reinforcement social learning in networked cooperative multiagent systems","volume":"58","author":"Hao","year":"2017","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1007\/s11277-007-9431-8","article-title":"Multi-access management in heterogeneous networks","volume":"48","author":"Sachs","year":"2009","journal-title":"Wirel. Pers. Commun."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Brockwell, P.J., and Davis, R.A. (2016). Introduction to Time Series and Forecasting, Springer.","DOI":"10.1007\/978-3-319-29854-2"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"041145","DOI":"10.1103\/PhysRevE.85.041145","article-title":"Dynamics of Boltzmann q learning in two-player two-action games","volume":"85","author":"Kianercy","year":"2012","journal-title":"Phys. Rev. E"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhu, K., Niyato, D., and Wang, P. (2010, January 18\u201321). Network Selection in Heterogeneous Wireless Networks: Evolution with Incomplete Information. Proceedings of the 2010 IEEE Wireless Communications and NETWORKING Conference (WCNC), Sydney, Australia.","DOI":"10.1109\/WCNC.2010.5506371"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MCOM.2004.1321381","article-title":"Ethernet pons: A survey of dynamic bandwidth allocation (dba) algorithms","volume":"42","author":"Mcgarry","year":"2004","journal-title":"IEEE Commun. Mag."},{"key":"ref_32","unstructured":"Malialis, K., Devlin, S., and Kudenko, D. (2016, January 9\u201313). Resource abstraction for reinforcement learning in multiagent congestion problems. Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, Singapore."},{"key":"ref_33","first-page":"666","article-title":"Online optimization for scheduling preemptable tasks on IaaS cloud systems","volume":"2","author":"Li","year":"2013","journal-title":"Int. J. Comput. Sci. Mob. Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3127498","article-title":"Efficient and Robust Emergence of Norms through Heuristic Collective Learning","volume":"12","author":"Hao","year":"2017","journal-title":"ACM Trans. Auton. Adapt. Syst."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/4\/236\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:59:00Z","timestamp":1760194740000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/4\/236"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,29]]},"references-count":34,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2018,4]]}},"alternative-id":["e20040236"],"URL":"https:\/\/doi.org\/10.3390\/e20040236","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2018,3,29]]}}}