{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,9]],"date-time":"2025-09-09T22:12:05Z","timestamp":1757455925337,"version":"3.37.3"},"reference-count":20,"publisher":"Springer Science and Business Media LLC","issue":"11-12","license":[{"start":{"date-parts":[[2020,8,27]],"date-time":"2020-08-27T00:00:00Z","timestamp":1598486400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,8,27]],"date-time":"2020-08-27T00:00:00Z","timestamp":1598486400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Ann. Telecommun."],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This paper describes the theoretical principles and experimental results of reinforcement learning algorithms embedded into IoT devices (Internet of Things), in order to tackle the problem of radio collision mitigation in ISM unlicensed bands. Multi-armed bandit (MAB) learning algorithms are used here to improve both the IoT network capability to support the expected massive number of objects and the energetic autonomy of the IoT devices. We first illustrate the efficiency of the proposed approach in a proof-of-concept, based on USRP software radio platforms operating on real radio signals. It shows how collisions with other RF signals are diminished for IoT devices that use MAB learning. Then we describe the first implementation of such algorithms on LoRa devices operating in a real LoRaWAN network at 868 MHz. We named this solution IoTligent. IoTligent does not add neither processing overhead, so it can be run into the IoT devices, nor network overhead, so that it requires no change to LoRaWAN protocol. Real-life experiments done in a real LoRa network show that IoTligent devices\u2019 battery life can be extended by a factor of 2, in the scenarios we faced during our experiment. Finally we submit IoTligent devices to very constrained conditions that are expected in the future with the growing number of IoT devices, by generating an artificial IoT massive radio traffic in anechoic chamber. We show that IoTligent devices can cope with spectrum scarcity that will occur at that time in unlicensed bands.<\/jats:p>","DOI":"10.1007\/s12243-020-00795-y","type":"journal-article","created":{"date-parts":[[2020,8,27]],"date-time":"2020-08-27T19:03:51Z","timestamp":1598555031000},"page":"711-727","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Decentralized spectrum learning for radio collision mitigation in ultra-dense IoT networks: LoRaWAN case study and experiments"],"prefix":"10.1007","volume":"75","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9639-1648","authenticated-orcid":false,"given":"Christophe","family":"Moy","sequence":"first","affiliation":[]},{"given":"Lilian","family":"Besson","sequence":"additional","affiliation":[]},{"given":"Guillaume","family":"Delbarre","sequence":"additional","affiliation":[]},{"given":"Laurent","family":"Toutain","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,8,27]]},"reference":[{"key":"795_CR1","unstructured":"Sornin N, Luis M, Eirich T and Beylot AL (2015) \u201cLoRaWAN specification\u201d, technical report, LoRa Alliance, Inc."},{"key":"795_CR2","volume-title":"\"The technical challenge of future IoT networks and their consequences on modem\u2019s and SoC\u2019s design\", R\u00e9seaux et services conference","author":"C Fourtet","year":"2015","unstructured":"Fourtet C (2015) \"The technical challenge of future IoT networks and their consequences on modem\u2019s and SoC\u2019s design\", R\u00e9seaux et services conference. R&S, Paris"},{"key":"795_CR3","first-page":"9","volume-title":"IoTligent: first world-wide Implementation of decentralized spectrum learning for IoT wireless networks","author":"C Moy","year":"2019","unstructured":"Moy C (2019) IoTligent: first world-wide Implementation of decentralized spectrum learning for IoT wireless networks. URSI AP-RASC, New Delhi, pp 9\u201314"},{"key":"795_CR4","first-page":"6","volume-title":"Multi-armed bandit based policies for cognitive radio\u2019s decision making issues","author":"W Jouini","year":"2009","unstructured":"Jouini W, Ernst D, Moy C, Palicot J (2009) Multi-armed bandit based policies for cognitive radio\u2019s decision making issues. Signal Circuits and Systems Conference, Jerba, pp 6\u20138"},{"key":"795_CR5","doi-asserted-by":"crossref","unstructured":"Jouini W, Ernst D, Moy C and Palicot J, (2010)\u201cUpper confidence bound based decision making strategies and dynamic spectrum access,\u201d IEEE ICC, International Conference on Communications, Cape Town, South Africa.","DOI":"10.1109\/ICC.2010.5502014"},{"issue":"1","key":"795_CR6","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1016\/0196-8858(85)90002-8","volume":"6","author":"TL Lai","year":"1985","unstructured":"Lai TL, Robbins H (1985) Asymptotically efficient adaptive allocation rules. Adv Appl Math 6(1):4\u201322","journal-title":"Adv Appl Math"},{"key":"795_CR7","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1023\/A:1013689704352","volume":"47","author":"P Auer","year":"2002","unstructured":"Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Mach Learn 47:2\u20133","journal-title":"Mach Learn"},{"key":"795_CR8","volume-title":"\u201cReinforcement learning real experiments for opportunistic spectrum access\u201d, Karlsruhe Workshop on Software Radio","author":"C Moy","year":"2014","unstructured":"Moy C (2014) \u201cReinforcement learning real experiments for opportunistic spectrum access\u201d, Karlsruhe Workshop on Software Radio. Karlsruhe, Germany"},{"key":"795_CR9","doi-asserted-by":"crossref","unstructured":"Bonnefoi R, Besson L, Moy C, Kaufman E, Palicot J (2017) Multi-armed bandit learning in IoT networks: learning helps even in non-stationary settings. CrownCom, Lisbon","DOI":"10.1007\/978-3-319-76207-4_15"},{"key":"795_CR10","doi-asserted-by":"crossref","unstructured":"Anandkumar A, Michael N, Tang AK, and Swami A, (2011) \u201cDistributed algorithms for learning and cognitive medium access with logarithmic regret\u201d, IEEE J Selected Areas Commun 29(4)","DOI":"10.1109\/JSAC.2011.110406"},{"issue":"5","key":"795_CR11","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1090\/S0002-9904-1952-09620-8","volume":"58","author":"H Robbins","year":"1952","unstructured":"Robbins H (1952) Some aspects of the sequential design of experiments. Bull Am Math Soc 58(5):527\u2013535","journal-title":"Bull Am Math Soc"},{"key":"795_CR12","doi-asserted-by":"crossref","unstructured":"Zhao Q, Sadler B, \"A survey of dynamic spectrum access\", in IEEE Signal Processing and Magazine, 2007.","DOI":"10.1109\/ICASSP.2007.367328"},{"issue":"1","key":"795_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1561\/2200000024","volume":"5","author":"S Bubeck","year":"2012","unstructured":"Bubeck S, Cesa-Bianchi N (2012) Regret analysis of stochastic and non-stochastic multi-armed bandit problems. Found Trends\u00ae Mach Learn 5(1):1\u2013122","journal-title":"Found Trends\u00ae Mach Learn"},{"key":"795_CR14","unstructured":"Besson L (2019), \"Multi-players algorithms for Internet of Things networks\u201d, PhD thesis, CentraleSup\u00e9lec."},{"key":"795_CR15","doi-asserted-by":"crossref","unstructured":"Thompson WR, (1933)\u201cOn the likelihood that one unknown probability exceeds another in view of the evidence of two samples,\u201d Biometrika 5","DOI":"10.2307\/2332286"},{"key":"795_CR16","doi-asserted-by":"crossref","unstructured":"Moy C, Palicot J, and Darak SJ, (2016) \u201cProof-of-concept system for opportunistic spectrum access in multi-user decentralized networks\u201d, EAI Endorsed Transactions on Cognitive Communications 2.","DOI":"10.4108\/eai.5-9-2016.151647"},{"key":"795_CR17","unstructured":"Besson L, \u201cSMPyBandits: an open-source research framework for single and multi-players multi-arms bandits (MAB) algorithms in Python\u201d. Code on https:\/\/GitHub.com\/SMPyBandits\/SMPyBandits and documentation on https:\/\/SMPyBandits.GitHub.io\/. Accessed: 2020"},{"key":"795_CR18","volume-title":"MALIN: multi-armed bandit learning for Iot networks with GRC: a TestBed implementation and demonstration that Learning Helps","author":"L Besson","year":"2018","unstructured":"Besson L, Bonnefoi R, Moy C (2018) MALIN: multi-armed bandit learning for Iot networks with GRC: a TestBed implementation and demonstration that Learning Helps. ICT, France"},{"key":"795_CR19","unstructured":"LoRaWAN\u2122, (2017) v1.1 Specification, LoRa Alliance Inc, https:\/\/LoRa-alliance.org\/sites\/default\/files\/2018-04\/lorawantm_specification_-v1.1.pdf. Accessed: 2020"},{"key":"795_CR20","first-page":"29","volume-title":"Decentralized spectrum learning for IoT wireless networks collision mitigation","author":"C Moy","year":"2019","unstructured":"Moy C, Besson L (2019) Decentralized spectrum learning for IoT wireless networks collision mitigation. First International Workshop on Intelligent Systems for the Internet of Things, Santorini Island, pp 29\u201331"}],"container-title":["Annals of Telecommunications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12243-020-00795-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12243-020-00795-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12243-020-00795-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,27]],"date-time":"2021-08-27T00:07:11Z","timestamp":1630022831000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12243-020-00795-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,27]]},"references-count":20,"journal-issue":{"issue":"11-12","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["795"],"URL":"https:\/\/doi.org\/10.1007\/s12243-020-00795-y","relation":{},"ISSN":["0003-4347","1958-9395"],"issn-type":[{"type":"print","value":"0003-4347"},{"type":"electronic","value":"1958-9395"}],"subject":[],"published":{"date-parts":[[2020,8,27]]},"assertion":[{"value":"10 July 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 August 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 August 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}