{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T04:29:22Z","timestamp":1772166562886,"version":"3.50.1"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,8,16]],"date-time":"2023-08-16T00:00:00Z","timestamp":1692144000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,16]],"date-time":"2023-08-16T00:00:00Z","timestamp":1692144000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003711","name":"Ministry of Science and Technology","doi-asserted-by":"publisher","award":["MOST 108-2221-E-343-002-"],"award-info":[{"award-number":["MOST 108-2221-E-343-002-"]}],"id":[{"id":"10.13039\/501100003711","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Wireless Com Network"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Many previous studies showed that some nodes should be in a sleeping state while the traffic is high. These nodes can wake up periodically to transmit data while they have data to transmit. The system throughput will be increased while each node can change between wake-up and sleeping states periodically. If a node can estimate the active rate and wake up at the optimal time, the collision probability decreases. This study proposes a Q-learning-based distributed queuing medium access control (QL-based DQMAC) protocol for Internet-of-Things (IoT) networks. In the proposed QL-based DQMAC, we derive the optimal number of contention IoT nodes. Each node calculates the active rate by itself through the Q-learning algorithm. Then, each node determines whether it will be active or in sleeping mode in the next contention period according to the active rate. Finding the optimal IoT nodes in each contention period decreases the probability of collision. The energy consumption due to the contention and delay for MAC contention is reduced owing to the lower number of contentions. Protocol comparison with other DQMAC protocols shows that the proposed QL-based DQMAC protocol achieves higher performance in IoT networks.<\/jats:p>","DOI":"10.1186\/s13638-023-02288-7","type":"journal-article","created":{"date-parts":[[2023,8,16]],"date-time":"2023-08-16T10:02:48Z","timestamp":1692180168000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Q-learning-based distributed queuing Mac protocol for Internet-of-Things networks"],"prefix":"10.1186","volume":"2023","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4746-1125","authenticated-orcid":false,"given":"Chien-Min","family":"Wu","sequence":"first","affiliation":[]},{"given":"Yu-Hsiu","family":"Chang","sequence":"additional","affiliation":[]},{"given":"Cheng-Tai","family":"Tsai","sequence":"additional","affiliation":[]},{"given":"Cheng-Chun","family":"Hou","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,8,16]]},"reference":[{"issue":"3","key":"2288_CR1","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1109\/MWC.2017.1600404","volume":"24","author":"AA Khan","year":"2017","unstructured":"A.A. Khan, M.H. Rehmani, A. Rachedi, Cognitive-radio-based internet of things: applications, architectures, spectrum related functionalities, and future research directions. IEEE Trans. Wirel. Commun. 24(3), 17\u201325 (2017)","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"2288_CR2","unstructured":"R.N.S.D.A. Vasavi, R. Priya, The internet of everything: a survey, in 2021 13th International Conference on Computational Intelligence and Communication Networks (CICN), 28 October 2021, Lima, Peru (2021)"},{"issue":"7","key":"2288_CR3","doi-asserted-by":"publisher","first-page":"1465","DOI":"10.1016\/j.future.2013.01.010","volume":"29","author":"J Gubbi","year":"2013","unstructured":"J. Gubbi, R. Buyya, S. Marusic, M. Palaniswam, Internet of things (iot): a vision, architectural elements, and future directions. Future Gener. Comput. Syst. 29(7), 1465\u20131660 (2013)","journal-title":"Future Gener. Comput. Syst."},{"issue":"1","key":"2288_CR4","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1108\/LHT-12-2018-0200","volume":"38","author":"EA Shammar","year":"2020","unstructured":"E.A. Shammar, A.T. Zahary, The internet of things (iot): a survey of techniques, operating systems, and trends. Library Hi Tech 38(1), 5\u201366 (2020)","journal-title":"Library Hi Tech"},{"key":"#cr-split#-2288_CR5.1","doi-asserted-by":"crossref","unstructured":"W. Xu, G. Campbell, A distributed queuing random access protocol for a broadcast channel, in SIGCOMM '93: Conference Proceedings on Communications Architectures, Protocols and Applications, 13-17 September 1993","DOI":"10.1145\/166237.166263"},{"key":"#cr-split#-2288_CR5.2","unstructured":"California, San Francisco, USA (1993)"},{"key":"2288_CR6","doi-asserted-by":"publisher","first-page":"763","DOI":"10.1109\/JIOT.2019.2945327","volume":"7","author":"W Wu","year":"2020","unstructured":"W. Wu, Y. Li, Y. Zhang, B. Wang, W. Wang, Distributed queueing-based random access protocol for loRa networks. IEEE Internet of Things J. 7, 763\u2013772 (2020)","journal-title":"IEEE Internet of Things J."},{"key":"2288_CR7","unstructured":"W. Ye, J. Heidemann, D. Estrin, An energy-efficient mac protocol for wireless sensor networks, in Proceedings Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies, 23\u201327 June 2002; New York, NY, USA (2002)"},{"key":"2288_CR8","doi-asserted-by":"crossref","unstructured":"M. Buettner, G.V. Yee, E. Anderson, R. Han, X-mac: a short preamble mac protocol for duty-cycled wireless sensor networks, in SenSys06: ACM Conference on Embedded Network Sensor Systems, 31 October 2006\u20133 November 2006; Boulder, Colorado, USA (2006)","DOI":"10.1145\/1182807.1182838"},{"key":"2288_CR9","doi-asserted-by":"crossref","unstructured":"J. Polastre, J. Hill, D. Culler, Versatile low power media access for wireless sensor networks, in SenSys \u201904: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems, 3\u20135 November 2004; MD, Baltimore, USA (2004)","DOI":"10.1145\/1031495.1031508"},{"key":"2288_CR10","first-page":"1","volume":"178","author":"R Ali","year":"2019","unstructured":"R. Ali, Y.A. Qadri, Y.B. Zikria, T. Umer, B.S. Kim, S. Kim, Q-learning-enabled channel access in next-generation dense wireless networks for iot-based ehealth systems. EURASIP J. Wirel. Commun. Netw. 178, 1\u201312 (2019)","journal-title":"EURASIP J. Wirel. Commun. Netw."},{"key":"2288_CR11","unstructured":"P. Pierrick, B. Conche, G. Serge, Distributed ensembles of reinforcement learning agents for electricity control, in 2022 IEEE International Conference on Omni-layer Intelligent Systems (COINS), 18 August 2022; Barcelona, Spain (2022)"},{"key":"2288_CR12","doi-asserted-by":"crossref","unstructured":"S. Amuru, Y. Xiao, M.V.D. Schaar, R.M. Buehrer, To send or not to send: learning mac contention, in 2015 IEEE Global Communications Conference (GLOBECOM), 6\u201310 December 2015; San Diego, CA, USA (2015)","DOI":"10.1109\/GLOCOM.2015.7417224"},{"key":"2288_CR13","unstructured":"Z. Liu, I. Elhanany, Rl-mac: a qos-aware reinforcement learning based mac protocol for wireless sensor networks, in 2006 IEEE International Conference on Networking, Sensing and Control, 23\u201325 April 2006; Ft. Lauderdale, FL, USA (2006)"},{"key":"2288_CR14","unstructured":"C. Wu, S. Ohzahata, Y. Ji, T. Kato, A mac protocol for delaysensitive vanet applications with self-learning contention scheme, in 2014 IEEE 11th Consumer Communications and Networking Conference (CCNC), 10\u201313 January 2014; Las Vegas, Nevada, USA (2014)"},{"key":"2288_CR15","unstructured":"F.C. Keras et al., https:\/\/keras.io (2015)"},{"key":"#cr-split#-2288_CR16.1","doi-asserted-by":"crossref","unstructured":"A.T. Ahmed, A.N. Rashid, S. Khalid, Energy model of wireless sensor network, in 2021 14th International Conference on Developments in eSystems Engineering (DeSE), 07-10 December 2021","DOI":"10.1109\/DeSE54285.2021.9719361"},{"key":"#cr-split#-2288_CR16.2","unstructured":"Sharjah, United Arab Emirates (2021)"},{"key":"2288_CR17","doi-asserted-by":"publisher","first-page":"2029","DOI":"10.1109\/ACCESS.2016.2557758","volume":"4","author":"A Laya","year":"2016","unstructured":"A. Laya, C. Kalalas, F. Vazquez-Gallego, L. Alonso, J. Alonso-Zarate, Goodbye, aloha. IEEE Access 4, 2029\u20132044 (2016)","journal-title":"IEEE Access"},{"issue":"3","key":"2288_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3484201","volume":"18","author":"F Gazi","year":"2022","unstructured":"F. Gazi, N. Ahmed, S. Misra, W. Wei, Reinforcement learning-based mac protocol for underwater multimedia sensor networks. ACM Trans. Sens. Netw. 18(3), 1\u201325 (2022)","journal-title":"ACM Trans. Sens. Netw."},{"key":"2288_CR19","doi-asserted-by":"publisher","first-page":"348","DOI":"10.3390\/fi14120348","volume":"14","author":"V Jorge","year":"2022","unstructured":"V. Jorge, B. Javier, Q-learning for opportunistic networks. Future Internet 14, 348 (2022)","journal-title":"Future Internet"},{"key":"2288_CR20","doi-asserted-by":"publisher","first-page":"115418","DOI":"10.1109\/ACCESS.2019.2935255","volume":"7","author":"H Jiang","year":"2019","unstructured":"H. Jiang, R. Gui, L. Chen, Z. Wu, J. Dang, J. Zhou, An improved sarsa \u03bb reinforcement learning algorithm for wireless communication systems. IEEE Access 7, 115418\u2013115427 (2019)","journal-title":"IEEE Access"},{"issue":"6","key":"2288_CR21","doi-asserted-by":"publisher","first-page":"1277","DOI":"10.1109\/JSAC.2019.2904329","volume":"37","author":"Y Yu","year":"2019","unstructured":"Y. Yu, T. Wang, S.C. Liew, Deep-reinforcement learning multiple access for heterogeneous wireless networks. IEEE J. Sel. Areas Commun. 37(6), 1277\u20131290 (2019)","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"2288_CR22","first-page":"1","volume":"24","author":"L Jianjun","year":"2022","unstructured":"L. Jianjun, L. Lu, W. Ying, Qos-oriented media access control using reinforcement learning for next-generation wlans. Comput. Netw. 24, 1\u201312 (2022)","journal-title":"Comput. Netw."},{"key":"2288_CR23","doi-asserted-by":"crossref","unstructured":"M.P. Mota, A. Valcarce, J.M. Gorce, J. Hoydis, The emergence of wireless mac protocols with multi-agent reinforcement learning, in 2021 IEEE Globecom Workshops (GC Wkshps), 24 January 2022; Madrid, Spain (2021)","DOI":"10.1109\/GCWkshps52748.2021.9681991"},{"issue":"9","key":"2288_CR24","doi-asserted-by":"publisher","first-page":"9136","DOI":"10.1109\/TVT.2019.2929035","volume":"68","author":"HY Chen","year":"2019","unstructured":"H.Y. Chen, M.L. Ku, S.J. Jou, C.C. Huang, A q-learning approach with collective contention estimation for bandwidth-efficient and fair access control in IEEE 802.11p vehicular networks. IEEE Trans. Veh. Technol. 68(9), 9136\u20139150 (2019)","journal-title":"IEEE Trans. Veh. Technol."},{"key":"2288_CR25","volume-title":"Reinforcement Learning: An Introduction","author":"RS Sutton","year":"1988","unstructured":"R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, 1988)"},{"key":"2288_CR26","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","volume":"8","author":"CJCH Watkins","year":"1992","unstructured":"C.J.C.H. Watkins, P. Dayan, Q-learning. Mach. Learn. 8, 279\u2013292 (1992)","journal-title":"Mach. Learn."},{"key":"2288_CR27","doi-asserted-by":"crossref","unstructured":"A. Pressas, Z. Sheng, D. Ali, F. Tian, M. Nekovee, Contention-based learning mac protocol for broadcast vehicle-to-vehicle communication, in 2017 IEEE Vehicular Networking Conference (VNC), 27\u201329 November 2017; Torino, Italy (2017)","DOI":"10.1109\/VNC.2017.8275614"},{"issue":"3","key":"2288_CR28","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1109\/49.840210","volume":"18","author":"G Giuseppe","year":"2000","unstructured":"G. Giuseppe, Performance analysis of the IEEE 802.11 distributed coordination function. IEEE J. Sel. Areas Commun. 18(3), 535\u2013547 (2000)","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"2288_CR29","doi-asserted-by":"crossref","unstructured":"Z. Sadreddini, O. Makul, T. \u00c7avdar, F.B. G\u00fcnay, Performance analysis of licensed shared access based secondary users activity on cognitive radio networks, in 2018 Electric Electronics, Computer Science, Biomedical Engineerings\u2019 Meeting (EBBT), 18\u201319 April 2018; Istanbul, Turkey (2018)","DOI":"10.1109\/EBBT.2018.8391442"},{"issue":"33","key":"2288_CR30","first-page":"1","volume":"22","author":"KH Lee","year":"2022","unstructured":"K.H. Lee, D. Kim, Cfx: contention-free channel access for IEEE 802.11ax. Sens. Netw. 22(33), 1\u201320 (2022)","journal-title":"Sens. Netw."},{"issue":"9","key":"2288_CR31","doi-asserted-by":"publisher","first-page":"1701","DOI":"10.1109\/49.872957","volume":"18","author":"L Alonso","year":"2000","unstructured":"L. Alonso, R. Agusti, O. Sallent, A near-optimum mac protocol based on the distributed queuing random access protocol (dqrap) for a cdma mobile communication system. IEEE J. Sel. Areas Commun. 18(9), 1701\u20131718 (2000)","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"1","key":"2288_CR32","doi-asserted-by":"publisher","first-page":"763","DOI":"10.1109\/JIOT.2019.2945327","volume":"7","author":"W Wu","year":"2020","unstructured":"W. Wu, Y. Li, B. Zhang, Y. Wang, W. Wang, Distributed queuing based random access protocol for lora networks. IEEE Internet Things J. 7(1), 763\u2013772 (2020)","journal-title":"IEEE Internet Things J."},{"key":"2288_CR33","doi-asserted-by":"publisher","first-page":"133653","DOI":"10.1109\/ACCESS.2019.2941229","volume":"7","author":"B Jang","year":"2019","unstructured":"B. Jang, M. Kim, G. Harerimana, J.W. Kim, Q-learning algorithms: a comprehensive classification and applications. IEEE Access 7, 133653\u2013133667 (2019)","journal-title":"IEEE Access"}],"container-title":["EURASIP Journal on Wireless Communications and Networking"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13638-023-02288-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13638-023-02288-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13638-023-02288-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,20]],"date-time":"2023-11-20T03:25:43Z","timestamp":1700450743000},"score":1,"resource":{"primary":{"URL":"https:\/\/jwcn-eurasipjournals.springeropen.com\/articles\/10.1186\/s13638-023-02288-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,16]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2288"],"URL":"https:\/\/doi.org\/10.1186\/s13638-023-02288-7","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-1554570\/v1","asserted-by":"object"}]},"ISSN":["1687-1499"],"issn-type":[{"value":"1687-1499","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,16]]},"assertion":[{"value":"14 April 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 August 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 August 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"77"}}