{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T12:23:21Z","timestamp":1740140601405,"version":"3.37.3"},"reference-count":22,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,9,15]],"date-time":"2021-09-15T00:00:00Z","timestamp":1631664000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,9,15]],"date-time":"2021-09-15T00:00:00Z","timestamp":1631664000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Wireless Com Network"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The efficient use of resources in wireless communications has always been a major issue. In the Internet of Things (IoT), the energy resource becomes more critical. The transmission policy with the aid of a coordinator is not a viable solution in an IoT network, since a node should report its state to the coordinator for scheduling and it causes serious signaling overhead. Machine learning algorithms can provide the optimal distributed transmission mechanism with little overhead. A node can learn by itself by utilizing the machine learning algorithm and make the optimal transmission decision on its own. In this paper, we propose a novel learning Medium Access Control (MAC) protocol with learning nodes. Nodes learn the optimal transmission policy, i.e., minimizing the data and energy queue levels, using the Q-learning algorithm. The performance evaluation shows that the proposed scheme enhances the queue states and throughput.<\/jats:p>","DOI":"10.1186\/s13638-021-02047-6","type":"journal-article","created":{"date-parts":[[2021,9,15]],"date-time":"2021-09-15T09:02:44Z","timestamp":1631696564000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Learning nodes: machine learning-based energy and data management strategy"],"prefix":"10.1186","volume":"2021","author":[{"given":"Yunmin","family":"Kim","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3985-7808","authenticated-orcid":false,"given":"Tae-Jin","family":"Lee","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,9,15]]},"reference":[{"issue":"1","key":"2047_CR1","first-page":"2787","volume":"54","author":"L Atzori","year":"2012","unstructured":"L. Atzori, A. Iera, G. Moralbito, Internet of Things: a survey. Comput. Netw. 54(1), 2787\u20132805 (2012)","journal-title":"Comput. Netw."},{"issue":"7","key":"2047_CR2","doi-asserted-by":"publisher","first-page":"1645","DOI":"10.1016\/j.future.2013.01.010","volume":"29","author":"J Gubbi","year":"2013","unstructured":"J. Gubbi, R. Buyya, S. Marusic, M. Palaniswamia, Internet of Things (IoT): a vision, architectural elements, and future directions. Future Gener. Comput. Syst. 29(7), 1645\u20131660 (2013)","journal-title":"Future Gener. Comput. Syst."},{"issue":"6","key":"2047_CR3","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1109\/MCOM.2015.7120024","volume":"53","author":"P Kamalinejad","year":"2015","unstructured":"P. Kamalinejad, C. Mahapatra, Z. Sheng, S. Mirabbasi, V.C.M. Leung, Y.L. Guan, Wireless energy harvesting for the Internet of Things. IEEE Commun. Mag. 53(6), 102\u2013108 (2015)","journal-title":"IEEE Commun. Mag."},{"issue":"3","key":"2047_CR4","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1109\/SURV.2011.060710.00094","volume":"13","author":"S Sudevalayam","year":"2011","unstructured":"S. Sudevalayam, P. Kulkarni, Energy harvesting sensor nodes: survey and implications. IEEE Commun. Surv. Tutor. 13(3), 443\u2013461 (2011)","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"2047_CR5","doi-asserted-by":"crossref","unstructured":"A.\u00a0Biason, M.\u00a0Zorzi, Transmission policies for an energy harvesting device with a data queue. In Proceedings of International Conference on Computing, Networking and Communications (ICNC) (2015), pp. 189\u2013195","DOI":"10.1109\/ICCNC.2015.7069339"},{"key":"2047_CR6","doi-asserted-by":"crossref","unstructured":"D. Liu, J. Lin, J. Wang, X. Chen, Y. Chen, Dynamic power allocation for a hybrid energy harvesting transmitter with multiuser in fading channels. In Proceedings of IEEE Vehicular Technology Conference (VTC) (2016), pp. 1\u20135","DOI":"10.1109\/VTCFall.2016.7880974"},{"issue":"1","key":"2047_CR7","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1109\/TCOMM.2011.112811.100349","volume":"60","author":"J Yang","year":"2012","unstructured":"J. Yang, S. Ulukus, Optimal packet scheduling in an energy harvesting communication system. IEEE Trans. Commun. 60(1), 220\u2013230 (2012)","journal-title":"IEEE Trans. Commun."},{"issue":"2","key":"2047_CR8","doi-asserted-by":"publisher","first-page":"1393","DOI":"10.1109\/TWC.2015.2489642","volume":"15","author":"DD Testa","year":"2016","unstructured":"D.D. Testa, N. Michelusi, M. Zorzi, Optimal transmission policies for two-user energy harvesting device networks with limited state-of-charge knowledge. IEEE Trans. Wirel. Commun. 15(2), 1393\u20131405 (2016)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"8","key":"2047_CR9","doi-asserted-by":"publisher","first-page":"1732","DOI":"10.1109\/JSAC.2011.110921","volume":"29","author":"O Ozel","year":"2011","unstructured":"O. Ozel, K. Tutuncuoglu, J. Yang, S. Ulukus, A. Yener, Transmission with energy harvesting nodes in fading wireless channels: optimal policies. IEEE J. Sel. Areas Commun. 29(8), 1732\u20131743 (2011)","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"2047_CR10","doi-asserted-by":"crossref","unstructured":"Q.\u00a0Bai, J.A.\u00a0Nossek, Joint optimization of transmission and reception policies for energy harvesting nodes. In Proceedings of International Symposium of Wireless Communication Systems (2015)","DOI":"10.1109\/ISWCS.2015.7454376"},{"issue":"4","key":"2047_CR11","first-page":"1355","volume":"63","author":"N Michelus","year":"2015","unstructured":"N. Michelus, M. Zorzi, Optimal adaptive random multiaccess in energy harvesting wireless sensor networks. IEEE Trans. Commun. 63(4), 1355\u20131372 (2015)","journal-title":"IEEE Trans. Commun."},{"issue":"6","key":"2047_CR12","doi-asserted-by":"publisher","first-page":"4079","DOI":"10.1109\/TWC.2017.2691777","volume":"16","author":"MK Sharma","year":"2017","unstructured":"M.K. Sharma, C.R. Murthy, On the design of dual energy harvesting communication links with retransmission. IEEE Trans. Wirel. Commun. 16(6), 4079\u20134093 (2017)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"12","key":"2047_CR13","doi-asserted-by":"publisher","first-page":"5487","DOI":"10.1109\/TCOMM.2017.2734882","volume":"65","author":"A Yadav","year":"2017","unstructured":"A. Yadav, M. Goonewardena, W. Ajib, O.A. Dobre, H. Elbiaze, Energy management for energy harvesting wireless sensors with adaptive retransmission. IEEE Trans. Commun. 65(12), 5487\u20135498 (2017)","journal-title":"IEEE Trans. Commun."},{"key":"2047_CR14","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"D. Silver et al., Mastering the game of go with deep neural networks and tree search. Nature 529, 484\u2013489 (2016)","journal-title":"Nature"},{"issue":"12","key":"2047_CR15","doi-asserted-by":"publisher","first-page":"2687","DOI":"10.1109\/JSAC.2015.2478717","volume":"33","author":"WHR Chan","year":"2015","unstructured":"W.H.R. Chan, P. Zhang, I. Nevat, S.G. Nagarajan, A.C. Valera, H.-X. Tan, N. Gautam, Adaptive duty cycling in sensor networks with energy harvesting using continuous-time Markov Chain and fluid models. IEEE J. Sel. Areas Commun. 33(12), 2687\u20132700 (2015)","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"2047_CR16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/WCL.2012.112012.120754","volume":"2","author":"KJ Prabuchandran","year":"2013","unstructured":"K.J. Prabuchandran, S.K. Meena, S. Bhatnagar, Q-learning based energy management policies for a single sensor node with finite buffer. IEEE Wirel. Commun. Lett. 2, 1 (2013)","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"2047_CR17","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1109\/TCOMM.2015.2415777","volume":"63","author":"S Padakandla","year":"2015","unstructured":"S. Padakandla, K.J. Prabuchandran, S. Bhatnagar, Energy sharing for multiple sensor nodes with finite buffers. IEEE Trans. Commun. 63, 5 (2015)","journal-title":"IEEE Trans. Commun."},{"key":"2047_CR18","doi-asserted-by":"publisher","first-page":"48742","DOI":"10.1109\/ACCESS.2021.3068407","volume":"9","author":"F Ahmed","year":"2021","unstructured":"F. Ahmed, H.-S. Cho, A time-slotted data gathering medium access control protocol using Q-learning for underwater acoustic sensor networks. IEEE Access 9, 48742\u201348752 (2021)","journal-title":"IEEE Access"},{"key":"2047_CR19","first-page":"7","volume":"18","author":"B Yang","year":"2019","unstructured":"B. Yang, X. Cao, Z. Han, L. Qian, A machine learning enabled MAC framework for heterogeneous Internet-of-Things networks. IEEE Trans. Wirel. Commun. 18, 7 (2019)","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"2047_CR20","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511841224","volume-title":"Wireless Communication","author":"A Goldsmith","year":"2005","unstructured":"A. Goldsmith, Wireless Communication (Cambridge University, Cambridge, 2005)"},{"key":"2047_CR21","unstructured":"C.J. Watkins, Learning from delayed rewards. Ph.D. dissertation, Cambridge University, Cambridge, UK (1989)"},{"issue":"3\u20134","key":"2047_CR22","first-page":"279","volume":"8","author":"CJ Watkins","year":"1992","unstructured":"C.J. Watkins, P. Dayan, Q-learning. Mach. Learn. 8(3\u20134), 279\u2013292 (1992)","journal-title":"Mach. Learn."}],"container-title":["EURASIP Journal on Wireless Communications and Networking"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13638-021-02047-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13638-021-02047-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13638-021-02047-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,15]],"date-time":"2021-09-15T09:17:15Z","timestamp":1631697435000},"score":1,"resource":{"primary":{"URL":"https:\/\/jwcn-eurasipjournals.springeropen.com\/articles\/10.1186\/s13638-021-02047-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,15]]},"references-count":22,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["2047"],"URL":"https:\/\/doi.org\/10.1186\/s13638-021-02047-6","relation":{},"ISSN":["1687-1499"],"issn-type":[{"type":"electronic","value":"1687-1499"}],"subject":[],"published":{"date-parts":[[2021,9,15]]},"assertion":[{"value":"27 March 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 August 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 September 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"176"}}