{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T15:20:09Z","timestamp":1782400809915,"version":"3.54.5"},"reference-count":61,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2018,1,27]],"date-time":"2018-01-27T00:00:00Z","timestamp":1517011200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Current trends in interconnecting myriad smart objects to monetize on Internet of Things applications have led to high-density communications in wireless sensor networks. This aggravates the already over-congested unlicensed radio bands, calling for new mechanisms to improve spectrum management and energy efficiency, such as transmission power control. Existing protocols are based on simplistic heuristics that often approach interference problems (i.e., packet loss, delay and energy waste) by increasing power, leading to detrimental results. The scope of this work is to investigate how machine learning may be used to bring wireless nodes to the lowest possible transmission power level and, in turn, to respect the quality requirements of the overall network. Lowering transmission power has benefits in terms of both energy consumption and interference. We propose a protocol of transmission power control through a reinforcement learning process that we have set in a multi-agent system. The agents are independent learners using the same exploration strategy and reward structure, leading to an overall cooperative network. The simulation results show that the system converges to an equilibrium where each node transmits at the minimum power while respecting high packet reception ratio constraints. Consequently, the system benefits from low energy consumption and packet delay.<\/jats:p>","DOI":"10.3390\/s18020375","type":"journal-article","created":{"date-parts":[[2018,1,29]],"date-time":"2018-01-29T07:46:20Z","timestamp":1517211980000},"page":"375","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":57,"title":["Self-Learning Power Control in Wireless Sensor Networks"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5276-8703","authenticated-orcid":false,"given":"Michele","family":"Chincoli","sequence":"first","affiliation":[{"name":"Electrical Engineering Department, Eindhoven University of Technology, De Zaale (internal address: Groene Loper 19), 5612 AJ Eindhoven, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2773-4421","authenticated-orcid":false,"given":"Antonio","family":"Liotta","sequence":"additional","affiliation":[{"name":"Data Science Centre, University of Derby, Lonsdale House, Quaker Way, Derby DE1 3HD, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,1,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Freeman, R.L. (2005). Fundamentals of Telecommunications, John Wiley & Sons-IEEE Press.","DOI":"10.1002\/0471720941"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Santi, P. (2005). Topology Control. Topology Control in Wireless Ad Hoc and Sensor Networks, John Wiley & Sons.","DOI":"10.1002\/0470094559"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1016\/j.adhoc.2004.04.003","article-title":"Wireless sensor and actor networks: Research challenges","volume":"2","author":"Akyildiz","year":"2004","journal-title":"Ad Hoc Netw."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Ogundile, O.O., and Alfa, A.S. (2017). A Survey on an Energy-Efficient and Energy-Balanced Routing Protocol for Wireless Sensor Networks. Sensors, 17.","DOI":"10.3390\/s17051084"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"439","DOI":"10.4304\/jcm.6.6.439-459","article-title":"Power Saving and Energy Optimization Techniques for Wireless Sensor Neworks","volume":"6","author":"Sendra","year":"2011","journal-title":"J. Commun."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Jawad, H., Nordin, R., Gharghan, S., Jawad, A., and Ismail, M. (2017). Energy-Efficient Wireless Sensor Networks for Precision Agriculture: A Review. Sensors, 17.","DOI":"10.3390\/s17081781"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1108\/17427371211221072","article-title":"A survey on networks for smart\u2013metering systems","volume":"8","author":"Liotta","year":"2012","journal-title":"Int. J. Pervasive Comp. Commun."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/MWC.2013.6704479","article-title":"A survey on the ietf protocol suite for the internet of things: Standards, challenges, and opportunities","volume":"20","author":"Sheng","year":"2013","journal-title":"IEEE Wirel. Commun."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.future.2016.10.010","article-title":"Impact of Transmission Power Control in Multi-hop Networks","volume":"75","author":"Kotian","year":"2017","journal-title":"Futur. Gener. Comput. Syst."},{"key":"ref_10","first-page":"1","article-title":"Temporal Adaptive Link Quality Prediction with Online Learning","volume":"10","author":"Liu","year":"2014","journal-title":"ACM Trans. Sens. Netw."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liotta, A. (2012, January 16). Farewell to Deterministic Networks. Proceedings of the 2012 IEEE 19th Symposium on Communications and Vehicular Technology in the Benelux, Eindhoven, The Netherlands.","DOI":"10.1109\/SCVT.2012.6399413"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chincoli, M., and Liotta, A. (2018). Transmission Power Control in WSNs: From Deterministic to Cognitive Methods. Integration, Interconnection, and Interoperability of IoT Systems, Springer.","DOI":"10.1007\/978-3-319-61300-0_3"},{"key":"ref_13","unstructured":"Lin, S., Zhang, J., Zhou, G., Gu, L., Stankovic, J.A., and He, T. (November, January 31). ATPC: Adaptive transmission power control for wireless sensor networks. Proceedings of the 4th International Conference on Embedded Networked Sensor Systems, Boulder, CO, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Jeong, J., Culler, D., and Oh, J.H. (2007, January 6\u20138). Empirical analysis of transmission power control algorithms for wireless sensor networks. Proceedings of the Fourth International Conference on IEEE Networked Sensing Systems, Braunschweig, Germany.","DOI":"10.1109\/INSS.2007.4297383"},{"key":"ref_15","unstructured":"ElBatt, T.A., Krishnamurthy, S.V., Connors, D., and Dao, S. (2000, January 18\u201322). Power management for throughput enhancement in wireless ad-hoc networks. Proceedings of the 2000 IEEE International Conference on Communications, New Orleans, LA, USA."},{"key":"ref_16","unstructured":"Narayanaswamy, S., Kawadia, V., Sreenivas, R.S., and Kumar, P. (2002, January 6\u20138). Power control in ad-hoc networks: Theory, architecture, algorithm and implementation of the COMPOW protocol. Proceedings of the European Wireless Conference, Florence, Italy."},{"key":"ref_17","unstructured":"Fu, Y., Sha, M., Hackmann, G., and Lu, C. (November, January 30). Practical control of transmission power for wireless sensor networks. Proceedings of the 2012 20th IEEE International Conference on IEEE Network Protocols (ICNP), Austin, TX, USA."},{"key":"ref_18","unstructured":"Kubisch, M., Karl, H., Wolisz, A., Zhong, L.C., and Rabaey, J. (2003, January 16\u201320). Distributed algorithms for transmission power control in wireless sensor networks. Proceedings of the 2003 IEEE Wireless Communications and Networking (WCNC 2003), New Orleans, LA, USA."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"978","DOI":"10.1109\/TII.2014.2310594","article-title":"Adaptive Multi-Channel Transmission Power Control for Industrial Wireless Instrumentation","volume":"10","author":"Ikram","year":"2014","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1109\/TCE.2009.4814420","article-title":"Interference-aware topology control for low rate wireless personal area networks","volume":"55","author":"Kim","year":"2009","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSPEC.2013.6565557","article-title":"The cognitive NET is coming","volume":"50","author":"Liotta","year":"2013","journal-title":"IEEE Spectr."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.adhoc.2015.07.013","article-title":"Ensembles of incremental learners to detect anomalies in ad hoc sensor networks","volume":"35","author":"Bosman","year":"2015","journal-title":"Ad Hoc Netw."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.inffus.2016.04.007","article-title":"Spatial anomaly detection in sensor networks using neighborhood information","volume":"33","author":"Bosman","year":"2017","journal-title":"Inf. Fus."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1109\/SURV.2011.040310.00002","article-title":"Computational Intelligence in Wireless Sensor Networks: A Survey","volume":"13","author":"Kulkarni","year":"2011","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Galzarano, S., Liotta, A., and Fortino, G. (2013). QL-MAC: A Q-learning based MAC for wireless sensor networks. Algorithms and Architectures for Parallel Processing, Springer. ICA3PP 2013; Lecture Notes in Computer Science, vol. 8286.","DOI":"10.1007\/978-3-319-03889-6_31"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1007\/s00607-014-0438-1","article-title":"Application of reinforcement learning to wireless sensor networks: Models and algorithms","volume":"97","author":"Yau","year":"2015","journal-title":"Computing"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Galzarano, S., Savaglio, C., Liotta, A., and Fortino, G. (2013, January 13\u201316). Gossiping-Based AODV for Wireless Sensor Networks. Proceedings of the 2013 IEEE International Conference on Systems, Man, and Cybernetics, Manchester, UK.","DOI":"10.1109\/SMC.2013.12"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"9204","DOI":"10.1109\/ACCESS.2016.2645979","article-title":"Exploiting Social Internet of Things Features in Cognitive Radio","volume":"4","author":"Nitti","year":"2016","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"76","DOI":"10.5296\/npa.v8i3.10257","article-title":"Consumption of Energy and Routing Protocols in Wireless Sensor Network","volume":"8","author":"Azizi","year":"2016","journal-title":"Netw. Protoc. Algorithms"},{"key":"ref_30","first-page":"1","article-title":"Power Control in Wireless Sensor Networks with Variable Interference","volume":"2016","author":"Chincoli","year":"2016","journal-title":"Mob. Inf. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Liang, X., Balasingham, I., and Leung, V.C.M. (December, January 30). Cooperative Communications with Relay Selection for QoS Provisioning in Wireless Sensor Networks. Proceedings of the 2009 IEEE Global Telecommunications Conference, Honolulu, HI, USA.","DOI":"10.1109\/GLOCOM.2009.5425437"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1239","DOI":"10.1109\/COMST.2015.2420686","article-title":"Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey","volume":"17","author":"Alsheikh","year":"2015","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1109\/JSAC.2005.843547","article-title":"Near-optimal reinforcement learning framework for energy-aware sensor communications","volume":"23","author":"Pandana","year":"2005","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1495","DOI":"10.1109\/TAC.2014.2305951","article-title":"Optimal Power Management in Wireless Control Systems","volume":"59","author":"Gatsis","year":"2014","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1007\/s11036-006-5185-x","article-title":"An Energy-Optimal Algorithm for Neighbor Discovery in Wireless Sensor Networks","volume":"11","author":"Madan","year":"2006","journal-title":"Mob. Netw. Appl."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Stabellini, L. (2008, January 21\u201324). Energy optimal neighbor discovery for single-radio single-channel wireless sensor networks. Proceedings of the 2008 IEEE International Symposium on Wireless Communication Systems, Reykjavik, Iceland.","DOI":"10.1109\/ISWCS.2008.4726123"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lange, S., Gabel, T., and Riedmiller, M. (2012). Batch reinforcement learning. Reinforcement Learning, Springer.","DOI":"10.1007\/978-3-642-27645-3_2"},{"key":"ref_38","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Introduction to Reinforcement Learning, MIT Press. [1st ed.]."},{"key":"ref_39","unstructured":"Krishnamurthy, V., and Ngo, M.H. (2005, January 23). A game theoretical approach for transmission strategies in slotted ALOHA networks with multi-packet reception. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2005), Philadelphia, PA, USA."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Udenze, A., and McDonald-Maier, K. (2008, January 4\u20136). Partially Observable Markov Decision Process for Transmitter Power Control in Wireless Sensor Networks. Proceedings of the ECSIS Symposium on Bio-Inspired Learning and Intelligent Systems for Security (BLISS 2008), Edinburgh, UK.","DOI":"10.1109\/BLISS.2008.32"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Kobbane, A., Koulali, M.A., Tembine, H., Koutbi, M.E., and Ben-othman, J. (2012, January 10\u201315). Dynamic power control with energy constraint for Multimedia Wireless Sensor Networks. Proceedings of the 2012 IEEE International Conference on Communications (ICC), Ottawa, ON, Canada.","DOI":"10.1109\/ICC.2012.6363971"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"895","DOI":"10.1109\/JSTSP.2013.2258656","article-title":"Transmit Power Control Policies for Energy Harvesting Sensors With Retransmissions","volume":"7","author":"Aprem","year":"2013","journal-title":"IEEE J. Sel. Top. Signal Proc."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2128","DOI":"10.1109\/TAC.2014.2319011","article-title":"Optimal Energy Allocation for Kalman Filtering Over Packet Dropping Links With Imperfect Acknowledgments and Energy Harvesting Constraints","volume":"59","author":"Nourian","year":"2014","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_44","first-page":"1","article-title":"Energy Management for Energy Harvesting Wireless Sensors with Adaptive Retransmission","volume":"PP","author":"Yadav","year":"2017","journal-title":"IEEE Trans. Commun."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Liang, X., Chen, M., Leung, V.C.M., and Balasingham, I. (2010, January 25\u201327). Soft QoS Provisioning for Wireless Sensor Networks: A cooperative communications approach. Proceedings of the 2010 5th International ICST Conference on Communications and Networking in China, Beijing, China.","DOI":"10.4108\/chinacom.2010.113"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1109\/JSAC.2010.100914","article-title":"An adaptive link layer for heterogeneous multi-radio mobile sensor networks","volume":"28","author":"Gummeson","year":"2010","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1109\/TWC.2010.111910.091238","article-title":"Autonomic and Distributed Joint Routing and Power Control for Delay-Sensitive Applications in Multi-Hop Wireless Networks","volume":"10","author":"Lin","year":"2011","journal-title":"IEEE Trans.Wirel. Commun."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Udenze, A., and McDonald-Maier, K. (August, January 29). Direct Reinforcement Learning for Autonomous Power Configuration and Control in Wireless Networks. Proceedings of the NASA\/ESA Conference on Adaptive Hardware and Systems, AHS 2009, San Francisco, CA, USA.","DOI":"10.1109\/AHS.2009.50"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"233","DOI":"10.14257\/ijca.2017.10.5.22","article-title":"An Energy-Efficient Topology Control Algorithm Based on Reinforcement Learning for Wireless Sensor Networks","volume":"10","author":"Le","year":"2017","journal-title":"Int. J. Control Autom."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1007\/s11277-013-1235-4","article-title":"Q-learning Reward Propagation Method for Reducing the Transmission Power of Sensor Nodes in Wireless Sensor Networks","volume":"73","author":"Sung","year":"2013","journal-title":"Wirel. Pers. Commun."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Kazemi, R., Vesilo, R., Dutkiewicz, E., and Liu, R. (2011, January 11\u201314). Dynamic power control in Wireless Body Area Networks using reinforcement learning with approximation. Proceedings of the 2011 IEEE 22nd International Symposium on Personal, Indoor and Mobile Radio Communications, Toronto, ON, Canada.","DOI":"10.1109\/PIMRC.2011.6139908"},{"key":"ref_52","first-page":"1039","article-title":"Nash Q-learning for general-sum stochastic games","volume":"4","author":"Hu","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Wiering, M., and van Otterlo, M. (2012). Reinforcement Learning. Adaptation, Learning, and Optimization, Springer.","DOI":"10.1007\/978-3-642-27645-3"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Chincoli, M., Syed, A.A., Mocanu, D.C., and Liotta, A. (2016, January 4\u20138). Predictive Power Control in Wireless Sensor Networks. Proceedings of the 2016 IEEE First International Conference on Internet-of-Things Design and Implementation (IoTDI), Berlin, Germany.","DOI":"10.1109\/IoTDI.2015.11"},{"key":"ref_55","unstructured":"IEEE Computer Society, LAN\/MAN Standards Committee, Institute of Electrical and Electronics Engineers, and IEEE-SA Standards Board (2006). IEEE Standard for Information Technology Telecommunications and Information Exchange between Systems\u2013Local and Metropolitan area Networks\u2013Specific Requirements, Institute of Electrical and Electronics Engineers. Part 15.4."},{"key":"ref_56","unstructured":"Series, P. (2009). Propagation Data and Prediction Methods for the Planning of Indoor Radiocommunication Systems and Radio Local Area Networks in the Frequency Range 900 MHz to 100 GHz, ICU."},{"key":"ref_57","unstructured":"Hoffmann, W. (1960). The m-distribution \u2013 A general formula of intensity distribution of rapid fading. Statistical Methods in Radio Wave Propagation, Pergamon\/RAND Corporation."},{"key":"ref_58","unstructured":"(2017, November 22). Atmel 8351 MCU Wireless AT86RF233 Datasheet. Available online: http:\/\/www.atmel.com\/images\/Atmel-8351-MCU_Wireless-AT86RF233_Datasheet.pdf."},{"key":"ref_59","unstructured":"Dunkels, A. (2011). The contikimac radio duty cycling protocol, SICS. Technical report."},{"key":"ref_60","unstructured":"(2017, November 22). CC2420 Single-Chip 2.4 GHz IEEE 802.15.4 Compliant and ZigBeeTM Ready RF Transceiver. Available online: http:\/\/www.ti.com\/product\/CC2420."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Torrey, L., and Shavlik, J. (2009). Transfer learning. Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, IGI Global.","DOI":"10.4018\/978-1-60566-766-9.ch011"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/2\/375\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:52:48Z","timestamp":1760194368000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/2\/375"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,1,27]]},"references-count":61,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2018,2]]}},"alternative-id":["s18020375"],"URL":"https:\/\/doi.org\/10.3390\/s18020375","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,1,27]]}}}