{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T15:27:47Z","timestamp":1791041267784,"version":"4.1.0"},"reference-count":27,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2018,1,26]],"date-time":"2018-01-26T00:00:00Z","timestamp":1516924800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Multi-armed bandit (MAB) models are a viable approach to describe the problem of best wireless network selection by a multi-Radio Access Technology (multi-RAT) device, with the goal of maximizing the quality perceived by the final user. The classical MAB model does not allow, however, to properly describe the problem of wireless network selection by a multi-RAT device, in which a device typically performs a set of measurements in order to collect information on available networks, before a selection takes place. The MAB model foresees in fact only one possible action for the player, which is the selection of one among different arms at each time step; existing arm selection algorithms thus mainly differ in the rule according to which a specific arm is selected. This work proposes a new MAB model, named measure-use-MAB (muMAB), aiming at providing a higher flexibility, and thus a better accuracy in describing the network selection problem. The muMAB model extends the classical MAB model in a twofold manner; first, it foresees two different actions: to measure and to use; second, it allows actions to span over multiple time steps. Two new algorithms designed to take advantage of the higher flexibility provided by the muMAB model are also introduced. The first one, referred to as measure-use-UCB1 (muUCB1) is derived from the well known UCB1 algorithm, while the second one, referred to as Measure with Logarithmic Interval (MLI), is appositely designed for the new model so to take advantage of the new measure action, while aggressively using the best arm. The new algorithms are compared against existing ones from the literature in the context of the muMAB model, by means of computer simulations using both synthetic and captured data. Results show that the performance of the algorithms heavily depends on the Probability Density Function (PDF) of the reward received on each arm, with different algorithms leading to the best performance depending on the PDF. Results highlight, however, that as the ratio between the time required for using an arm and the time required to measure increases, the proposed algorithms guarantee the best performance, with muUCB1 emerging as the best candidate when the arms are characterized by similar mean rewards, and MLI prevailing when an arm is significantly more rewarding than others. This calls thus for the introduction of an adaptive approach capable of adjusting the behavior of the algorithm or of switching algorithm altogether, depending on the acquired knowledge on the PDF of the reward on each arm.<\/jats:p>","DOI":"10.3390\/a11020013","type":"journal-article","created":{"date-parts":[[2018,1,26]],"date-time":"2018-01-26T07:31:51Z","timestamp":1516951911000},"page":"13","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["muMAB: A Multi-Armed Bandit Model for Wireless Network Selection"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0650-3114","authenticated-orcid":false,"given":"Stefano","family":"Boldrini","sequence":"first","affiliation":[{"name":"Amadeus S.A.S., 485 Route du Pin Montard, 06902 Sophia Antipolis CEDEX, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9286-8744","authenticated-orcid":false,"given":"Luca","family":"De Nardis","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, Electronics and Telecommunications, Sapienza University of Rome, Via Eudossiana 18, 00184 Rome, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0611-5637","authenticated-orcid":false,"given":"Giuseppe","family":"Caso","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, Electronics and Telecommunications, Sapienza University of Rome, Via Eudossiana 18, 00184 Rome, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9657-5172","authenticated-orcid":false,"given":"Mai","family":"Le","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, Electronics and Telecommunications, Sapienza University of Rome, Via Eudossiana 18, 00184 Rome, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jocelyn","family":"Fiorina","sequence":"additional","affiliation":[{"name":"Laboratoire des Signaux et Syst\u00e8mes (L2S), CentraleSup\u00e9lec-CNRS-Universit\u00e9 Paris-Sud, Universit\u00e9 Paris-Saclay, 3, rue Joliot Curie, 91192 Gif-sur-Yvette, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1523-5083","authenticated-orcid":false,"given":"Maria-Gabriella","family":"Di Benedetto","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, Electronics and Telecommunications, Sapienza University of Rome, Via Eudossiana 18, 00184 Rome, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,1,26]]},"reference":[{"key":"ref_1","unstructured":"(2018, January 24). 5G: A Technology Vision. Available online: http:\/\/www.huawei.com\/5gwhitepaper\/."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Matinmikko, M., Roivainen, A., Latva-aho, M., and Hiltunen, K. (2017, January 20\u201322). Interference Study of Micro Licensing for 5G Micro Operator Small Cell Deployments. Proceedings of the 12th EAI International Conference on Cognitive Radio Oriented Wireless Networks (CROWNCOM), Lisbon, Portugal.","DOI":"10.1007\/978-3-319-76207-4_22"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1212","DOI":"10.1109\/SURV.2012.010912.00081","article-title":"Game Theory-Based Network Selection: Solutions and Challenges","volume":"14","author":"Trestian","year":"2012","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1109\/SURV.2012.010912.00044","article-title":"Mathematical Modeling for Network Selection in Heterogeneous Wireless Networks\u2014A Tutorial","volume":"15","author":"Wang","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2389","DOI":"10.1109\/TVT.2011.2140386","article-title":"Enhanced Group Handover Scheme in Multiaccess Networks","volume":"60","author":"Lee","year":"2011","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Farrugia, R.A., Galea, C., Zammit, S., and Muscat, A. (2013). Objective Video Quality Metrics for HDTV Services: A Survey. EuroCon, 2013.","DOI":"10.1109\/EUROCON.2013.6624982"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Boldrini, S., Di Benedetto, M.G., Tosti, A., and Fiorina, J. (2015). Cognitive radio and Networking for Heterogeneous Wireless Networks. Automatic Best Wireless Network Selection Based on Key Performance Indicators, Springer. Chapter by Boldrini.","DOI":"10.1007\/978-3-319-01718-1_7"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Tsiropoulou, E.E., Katsinis, G.K., Filios, A., and Papavassiliou, S. (2014, January 22\u201327). On the Problem of Optimal Cell Selection and Uplink Power Control in Open Access Multi-service Two-Tier Femtocell Networks. Proceedings of the 13th International Conference on Ad-Hoc Networks and Wireless (ADHOC-NOW 2014), Benidorm, Spain.","DOI":"10.1007\/978-3-319-07425-2_9"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Vamvakas, P., Tsiropoulou, E.E., and Papavassiliou, S. (2017). Dynamic provider selection and power resource management in competitive wireless communication markets. Mob. Netw. Appl., 1\u201314.","DOI":"10.1007\/s11036-017-0885-y"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2427","DOI":"10.1109\/TMC.2012.207","article-title":"Network Selection and Resource Allocation Games for Wireless Access Networks","volume":"12","author":"Malanchini","year":"2013","journal-title":"IEEE Trans. Mobile Comput."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5048","DOI":"10.1109\/TWC.2013.090513.122033","article-title":"Wireless Access Network Selection Game with Negative Network Externality","volume":"12","author":"Yang","year":"2013","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1111\/j.2517-6161.1980.tb01111.x","article-title":"Multi-armed bandits and the Gittins index","volume":"42","author":"Whittle","year":"1980","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_13","unstructured":"Gittins, J.C. (1989). Multi-Armed Bandit Allocation Indices, John Wiley & Sons."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Hero, A., Castanon, D., Cochran, D., and Kastella, K. (2008). Multi-Armed Bandit Problems. Foundations and Applications of Sensor Management, Springer International Publishing AG.","DOI":"10.1007\/978-0-387-49819-5"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/MWC.2017.1700090","article-title":"Toward Context-Aware Dynamic Spectrum Management for 5G","volume":"24","author":"Caso","year":"2017","journal-title":"IEEE Wirel. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1023\/A:1013689704352","article-title":"Finite-time analysis of the multiarmed bandit problem","volume":"47","author":"Auer","year":"2002","journal-title":"Mach. Learn."},{"key":"ref_17","first-page":"437","article-title":"Multi-armed bandit algorithms and empirical evaluation","volume":"Volume 3720","author":"Vermorel","year":"2005","journal-title":"Proceedings of the 16th European Conference on Machine Learning"},{"key":"ref_18","unstructured":"Agarwal, A., Hsu, D., Kale, S., Langford, J., Li, L., and Schapire, R.E. (2014, January 21\u201326). Taming the monster: a fast and simple algorithm for contextual bandits. Proceedings of the 31st International Conference on Machine Learning, Beijing, China."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1109\/TVT.2015.2394431","article-title":"Traffic-Aware Online Network Selection in Heterogeneous Wireless Networks","volume":"65","author":"Wu","year":"2016","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/0196-8858(85)90002-8","article-title":"Asymptotically efficient adaptive allocation rules","volume":"6","author":"Lai","year":"1985","journal-title":"Adv. Appl. Math."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Hassan, H., Elkhazeen, K., Raahemiafar, K., and Fernando, X. (2015, January 3\u20136). Optimization of control parameters using averaging of handover indicator and received power for minimizing ping-pong handover in LTE. Proceedings of the IEEE 28th Canadian Conference on Electrical and Computer Engineering (CCECE), Halifax, NS, Canada.","DOI":"10.1109\/CCECE.2015.7129166"},{"key":"ref_22","unstructured":"Cesa-Bianchi, N., and Fischer, P. (1998, January 24\u201327). Finite-time regret bounds of the multi-armed bandit problem. Proceedings of the 15th International Conference on Machine Learning (ICML 1998), Madison, WI, USA."},{"key":"ref_23","unstructured":"Watkins, C.J.C.H. (1989). Learning from Delayed Rewards. [Ph.D. Thesis, Cambridge University]."},{"key":"ref_24","unstructured":"Vermorel, J. (2018, January 24). Multi-Armed Bandit Data. Available online: https:\/\/sourceforge.net\/projects\/bandit\/."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1109\/TMC.2010.65","article-title":"Cognitive medium access: Exploration, exploitation, and competition","volume":"10","author":"Lai","year":"2011","journal-title":"IEEE Trans. Mobile Comput."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Mu, M., Mauthe, A., and Garcia, F. (2008, January 16\u201319). A utility-based QoS model for emerging multimedia applications. Proceedings of the 2nd International Conference on Next Generation Mobile Applications, Services and Technologies (NGMAST\u201908), Cardiff, UK.","DOI":"10.1109\/NGMAST.2008.24"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Boldrini, S., Fiorina, J., and Di Benedetto, M.G. (2013, January 8\u20139). Introducing strategic measure actions in multi-armed bandits. Proceedings of the IEEE 24th International Symposium on Personal, Indoor and Mobile Radio Communications-Workshop on Cognitive Radio Medium Access Control and Network Solutions (MACNET\u201913), London, UK.","DOI":"10.1109\/PIMRCW.2013.6707833"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/11\/2\/13\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:52:41Z","timestamp":1760194361000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/11\/2\/13"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,1,26]]},"references-count":27,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2018,2]]}},"alternative-id":["a11020013"],"URL":"https:\/\/doi.org\/10.3390\/a11020013","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,1,26]]}}}