{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,27]],"date-time":"2026-01-27T21:01:27Z","timestamp":1769547687969,"version":"3.49.0"},"reference-count":48,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2019,12,19]],"date-time":"2019-12-19T00:00:00Z","timestamp":1576713600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Introduction of high-level talents and overseas returnee\u2019s scientific fund","award":["GXL015"],"award-info":[{"award-number":["GXL015"]}]},{"DOI":"10.13039\/501100001809","name":"National natural science foundation of China","doi-asserted-by":"publisher","award":["61801225"],"award-info":[{"award-number":["61801225"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Wireless body area networks (WBANs) have attracted great attention from both industry and academia as a promising technology for continuous monitoring of physiological signals of the human body. As the sensors in WBANs are typically battery-driven and inconvenient to recharge, an energy efficient resource allocation scheme is essential to prolong the lifetime of the networks, while guaranteeing the rigid requirements of quality of service (QoS) of the WBANs in nature. As a possible alternative solution to address the energy efficiency problem, energy harvesting (EH) technology with the capability of harvesting energy from ambient sources can potentially reduce the dependence on the battery supply. Consequently, in this paper, we investigate the resource allocation problem for EH-powered WBANs (EH-WBANs). Our goal is to maximize the energy efficiency of the EH-WBANs with the joint consideration of transmission mode, relay selection, allocated time slot, transmission power, and the energy constraint of each sensor. In view of the characteristic of the EH-WBANs, we formulate the energy efficiency problem as a discrete-time and finite-state Markov decision process (DFMDP), in which allocation strategy decisions are made by a hub that does not have complete and global network information. Owing to the complexity of the problem, we propose a modified Q-learning (QL) algorithm to obtain the optimal allocation strategy. The numerical results validate the effectiveness of the proposed scheme as well as the low computation complexity of the proposed modified Q-learning (QL) algorithm.<\/jats:p>","DOI":"10.3390\/s20010044","type":"journal-article","created":{"date-parts":[[2019,12,23]],"date-time":"2019-12-23T03:15:01Z","timestamp":1577070901000},"page":"44","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":60,"title":["Reinforcement Learning (RL)-Based Energy Efficient Resource Allocation for Energy Harvesting-Powered Wireless Body Area Network"],"prefix":"10.3390","volume":"20","author":[{"given":"Yi-Han","family":"Xu","sequence":"first","affiliation":[{"name":"College of Information Science and Technology, Nanjing Forestry University, Nanjing 210037, China"},{"name":"School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney 2052, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jing-Wei","family":"Xie","sequence":"additional","affiliation":[{"name":"College of Information Science and Technology, Nanjing Forestry University, Nanjing 210037, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yang-Gang","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Fudan University, Shanghai 200433, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Min","family":"Hua","sequence":"additional","affiliation":[{"name":"College of Information Science and Technology, Nanjing Forestry University, Nanjing 210037, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wen","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Information Science and Technology, Nanjing Forestry University, Nanjing 210037, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,12,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1007\/s11036-010-0260-8","article-title":"Body area networks: A survey","volume":"16","author":"Chen","year":"2011","journal-title":"Mob. Netw. Appl."},{"key":"ref_2","first-page":"3","article-title":"Wireless Body Area Network (WBAN): A survey on reliability, fault tolerance, and technologies coexistence","volume":"50","author":"Marwa","year":"2017","journal-title":"ACM Comput. Surv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1146\/annurev-bioeng-071516-044517","article-title":"Energy harvesting from the animal\/human body for self-powered electronics","volume":"19","author":"Dagdeviren","year":"2017","journal-title":"Annu. Rev. Biomed. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1089\/tmj.2012.0215","article-title":"A review on telemedicine-based WBAN framework for patient monitoring","volume":"19","author":"Chakraborty","year":"2013","journal-title":"Telemed. J. E-Health"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"18009","DOI":"10.3390\/s141018009","article-title":"A survey on M2M systems for mHealth: A wireless communications perspective","volume":"14","author":"Elli","year":"2014","journal-title":"Sensors"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"12635","DOI":"10.3390\/s150612635","article-title":"Cooperative energy-harvesting-adaptive MAC protocol for WBANs","volume":"15","author":"Esteves","year":"2015","journal-title":"Sensors"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"804","DOI":"10.1109\/TGCN.2018.2813060","article-title":"Reliability and energy efficiency enhancement for emergency-aware wireless body area networks (WBANs)","volume":"2","author":"Salayma","year":"2018","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_8","first-page":"5767","article-title":"Transmission-rate-adaption assisted energy-efficient resource allocation with QoS support in WBANS","volume":"17","author":"Liu","year":"2017","journal-title":"IEEE Sens. J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"8483","DOI":"10.1109\/ACCESS.2018.2889879","article-title":"Reinforcement learning-based sensor access control for WBANs","volume":"7","author":"Chen","year":"2018","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Roy, M., Chowdhury, C., and Aslam, N. (2018). Designing transmission strategies for enhancing communications in medical IoT using Markov decision process. Sensors, 18.","DOI":"10.3390\/s18124450"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1007\/s11235-014-9898-z","article-title":"HEH-BMAC: Hybrid polling MAC protocol for wireless body area networks operated by human energy harvesting","volume":"58","author":"Ibarra","year":"2015","journal-title":"Telecommun. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4307","DOI":"10.1109\/JIOT.2018.2875926","article-title":"Learning-based privacy-aware offloading for healthcare IoT with energy harvesting","volume":"6","author":"Min","year":"2019","journal-title":"IEEE Int. Things J."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1182","DOI":"10.1109\/TWC.2014.012314.121185","article-title":"Power allocation for conventional and buffer-aided link adaptive relaying systems with energy harvesting nodes","volume":"13","author":"Ahmed","year":"2014","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1007\/s11801-016-6163-6","article-title":"Energy adaptive MAC protocol for IEEE 802.15.7 with energy harvesting","volume":"12","author":"Wang","year":"2016","journal-title":"Optoelectr. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2641","DOI":"10.1016\/j.renene.2010.06.014","article-title":"Energy harvesting: State-of-the-art","volume":"36","author":"Harb","year":"2011","journal-title":"Renew. Energy"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/MPRV.2005.9","article-title":"Energy scavenging for mobile and wireless electronics","volume":"4","author":"Paradiso","year":"2005","journal-title":"IEEE Pervasive Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1384","DOI":"10.1109\/COMST.2015.2497324","article-title":"Advances in energy harvesting communications: Past, present, and future challenges","volume":"18","author":"Ku","year":"2016","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3647","DOI":"10.1109\/ACCESS.2016.2579598","article-title":"Cooperative wireless energy harvesting and spectrum sharing in 5G networks","volume":"4","author":"Gao","year":"2016","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1109\/TGCN.2019.2908086","article-title":"Modeling of hybrid energy harvesting communication systems","volume":"3","author":"Altinel","year":"2019","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1109\/LCOMM.2015.2497306","article-title":"Predictive modelling of RF energy for wireless powered communications","volume":"20","author":"Azmat","year":"2015","journal-title":"IEEE Commun. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/JBHI.2017.2733549","article-title":"Ehdc: An energy harvesting modeling and profiling platform for body sensor networks","volume":"22","author":"Fan","year":"2018","journal-title":"IEEE J. Biomed. Health Inf."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1145\/1274858.1274870","article-title":"Power management in energy harvesting sensor networks","volume":"6","author":"Kansal","year":"2007","journal-title":"ACM Trans. Embed. Comput. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6477","DOI":"10.1109\/JSEN.2018.2851187","article-title":"Energy scavenging methods for WBAN applications: A review","volume":"18","author":"Demir","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Mekikis, P., Angelos, A., Elli, K., Nikos, P., Luis, A., and Christos, V. (2017, January 21\u201325). Stochastic modeling of wireless charged wearables for reliable health monitoring in hospital environments. Proceedings of the IEEE International Conference on Communications (ICC), Paris, France.","DOI":"10.1109\/ICC.2017.7997412"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"8620","DOI":"10.1109\/ACCESS.2017.2695222","article-title":"Point-to-point wireless information and power transfer in WBAN with energy harvesting","volume":"5","author":"Ling","year":"2017","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1109\/LWC.2014.2321765","article-title":"Optimal frame length to maximize energy efficiency in IEEE 802.15.6 uwb body area networks","volume":"3","author":"Mohammadi","year":"2014","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1558","DOI":"10.1109\/TMC.2011.83","article-title":"Optimal resource allocation for pervasive health monitoring systems with body sensor networks","volume":"10","author":"He","year":"2011","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, Z., Liu, B., Chen, C., and Chen, C.W. (2015, January 6\u201310). Energy-efficient resource allocation with QoS support in wireless body area networks. Proceedings of the IEEE Global Communications Conference, San Diego, CA, USA.","DOI":"10.1109\/GLOCOM.2015.7417157"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Jung, B.H., Akbar, R.U., and Sung, D.K. (2012, January 9\u201312). Throughput, energy consumption, and energy efficiency of IEEE 802.15.6 body area network (BAN) MAC protocol. Proceedings of the IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Sydney, NSW, Australia.","DOI":"10.1109\/PIMRC.2012.6362852"},{"key":"ref_30","unstructured":"Qiu, J., Lin, B., Liu, P., Zhang, S., and Dai, G. (2011, January 5\u20139). Energy level based transmission power control scheme for energy harvesting WSNs. Proceedings of the IEEE Global Communications Conference, Houston, TX, USA."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MWC.2007.4300988","article-title":"Wireless sensor networks with energy harvesting technologies: A game-theoretic approach to optimal energy management","volume":"14","author":"Niyato","year":"2007","journal-title":"IEEE Wirel. Commun."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Leng, S., and Yener, A. (2017). Resource allocation in body area networks for energy harvesting healthcare monitoring. Handbook of Large-Scale Distributed Computing in Smart Healthcare, Springer.","DOI":"10.1007\/978-3-319-58280-1_20"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/MITP.2017.34","article-title":"Energy harvesting for self-sustainable wireless body area networks","volume":"19","author":"Akhtar","year":"2017","journal-title":"IT Prof."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4640","DOI":"10.1109\/TWC.2015.2424247","article-title":"Power scheduling for energy harvesting wireless communications with battery capacity constrain","volume":"14","author":"Wei","year":"2015","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1109\/JSEN.2015.2483064","article-title":"Qos-aware energy management in body sensor nodes powered by human energy harvesting","volume":"16","author":"Ibarra","year":"2015","journal-title":"IEEE Sens. J."},{"key":"ref_36","unstructured":"(2012). IEEE Standard for Local and Metropolitan Area Networks\u2014Part 15.6: Wireless Body Area Networks, IEEE."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1109\/MCOM.2010.5496890","article-title":"Body-posture-based dynamic link power control in wearable sensor networks","volume":"48","author":"Quwaider","year":"2010","journal-title":"IEEE Commun. Mag."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Mitran, P. (2012, January 1\u20136). On optimal online policies in energy harvesting systems for compound poisson energy arrivals. Proceedings of the IEEE International Symposium on Information Theory, Cambridge, MA, USA.","DOI":"10.1109\/ISIT.2012.6284705"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"933","DOI":"10.1109\/TITB.2009.2033054","article-title":"Characterization of on-body communication channel and energy efficient topology design for wireless body area networks","volume":"3","author":"Reusens","year":"2009","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1007\/s10776-010-0122-0","article-title":"A statistical model for on-body dynamic channels","volume":"17","author":"Ouvry","year":"2010","journal-title":"Int. J. Wirel. Inf. Netw."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1080\/00401706.1995.10484354","article-title":"Markov decision processes: Discrete stochastic dynamic programming","volume":"37","author":"Baxter","year":"1995","journal-title":"Technometrics"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","article-title":"Applications of deep reinforcement learning in communications and networking: A survey","volume":"21","author":"Luong","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1109\/TNN.2006.887555","article-title":"MCES: A novel Monte Carlo evaluative selection approach for objective feature selections","volume":"18","author":"Quah","year":"2007","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1457","DOI":"10.1109\/TNNLS.2015.2442233","article-title":"Parallel online temporal difference learning for motor control","volume":"27","author":"Caarls","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Cai, X., Zheng, J., and Zhang, Y. (2015, January 8\u201312). A Graph-coloring based resource allocation algorithm for D2D communication in cellular networks. Proceedings of the IEEE International Conference on Communications (ICC), London, UK.","DOI":"10.1109\/ICC.2015.7249187"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Fedrizzi, R., Goratti, L., Sithamparanathan, K., and Rasheed, T. (2016, January 3\u20136). A Heuristic Approach to Mobility Robustness in 4G LTE Public Safety Networks. Proceedings of the IEEE Wireless Communications and Networking Conference, Doha, Qatar.","DOI":"10.1109\/WCNC.2016.7564919"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Sandhu, M.M., Javaid, N., Akbar, M., Najeeb, F., Qasim, U., and Khan, Z.A. (2014, January 13\u201316). FEEL: forwarding data energy efficiently with load balancing in wireless body area networks. Proceedings of the IEEE International Conference on Advanced Information Networking and Applications, Victoria, BC, Canada.","DOI":"10.1109\/AINA.2014.95"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"3108","DOI":"10.1109\/TSP.2010.2046040","article-title":"On-line learning and optimization for wireless video transmission","volume":"58","author":"Zhang","year":"2010","journal-title":"IEEE Trans. Signal Process."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/44\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:43:55Z","timestamp":1760190235000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/44"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,19]]},"references-count":48,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["s20010044"],"URL":"https:\/\/doi.org\/10.3390\/s20010044","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,19]]}}}