{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:17:09Z","timestamp":1760217429370,"version":"build-2065373602"},"reference-count":35,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2015,3,19]],"date-time":"2015-03-19T00:00:00Z","timestamp":1426723200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The use of beamforming and power control, combined or separately, has advantages and disadvantages, depending on the application. The combined use of beamforming and power control has been shown to be highly effective in applications involving the suppression of interference signals from different sources. However, it is necessary to identify efficient methodologies for the combined operation of these two techniques. The most appropriate technique may be obtained by means of the implementation of an intelligent agent capable of making the best selection between beamforming and power control. The present paper proposes an algorithm using reinforcement learning (RL) to determine the optimal combination of beamforming and power control in sensor arrays. The RL algorithm used was Q-learning, employing an  \u03b5-greedy policy, and training was performed using the offline method. The simulations showed that RL was effective for implementation of a switching policy involving the different techniques, taking advantage of the positive characteristics of each technique in terms of signal reception.<\/jats:p>","DOI":"10.3390\/s150306668","type":"journal-article","created":{"date-parts":[[2015,3,19]],"date-time":"2015-03-19T10:38:57Z","timestamp":1426761537000},"page":"6668-6687","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Beamforming and Power Control in Sensor Arrays Using Reinforcement Learning"],"prefix":"10.3390","volume":"15","author":[{"given":"N\u00e1thalee","family":"Almeida","sequence":"first","affiliation":[{"name":"UFERSA\u2014Federal Rural University of the Semi-\u00c1rido, Pau dos Ferros 59900-000, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7536-2506","authenticated-orcid":false,"given":"Marcelo","family":"Fernandes","sequence":"additional","affiliation":[{"name":"DCA-CT-UFRN, Federal University of Rio Grande do Norte, Natal 59072-970, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adri\u00e3o","family":"Neto","sequence":"additional","affiliation":[{"name":"DCA-CT-UFRN, Federal University of Rio Grande do Norte, Natal 59072-970, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2015,3,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Rambach, K., and Yang, B. (2014, January 19\u201323). Direction of arrival estimation of two moving targets using a time division multiplexed colocated MIMO radar. Proceedings of the 2014 IEEE Radar Conference, Cincinnati, OH, USA.","DOI":"10.1109\/RADAR.2014.6875763"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1109\/JSTSP.2013.2285520","article-title":"Joint transmission and reception diversity smoothing for direction finding of coherent targets in MIMO radar","volume":"9","author":"Zhang","year":"2014","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_3","first-page":"46","article-title":"Joint DOA estimation and source tracking with kalman filetring and regularized QRD RLS algorithm","volume":"60","author":"Gu","year":"2013","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Schmidt, J.F., and Lopez-Valcarce, R. (2014, January 22\u201325). Antenna competition to boost active interference cancellation in cognitive MIMO-OFDM. Proceedings of the 8th Sensor Array and Multichannel Signal Processing Workshop (SAM), A Coruna, Spain.","DOI":"10.1109\/SAM.2014.6882392"},{"key":"ref_5","unstructured":"Balanis, C.A. (2005). Antenna Theory: Analysis and Design, Wiley-Interscience. [3rd ed.]."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1313","DOI":"10.1109\/26.725309","article-title":"Joint optimal power control and beamforming in wireless networks using antenna arrays","volume":"46","author":"Tassiulas","year":"1998","journal-title":"IEEE Trans. Commun."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1109\/49.730452","article-title":"Transmit beamforming and power control for cellular wireless systems","volume":"16","author":"Tassiulas","year":"1998","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Pitz, C.A., Vanti, M.G., Tobias, O.J., and Seara, R. (2010, January 6\u20139). Adaptive Beamforming for Antenna Arrays in Cellular Systems Based on a Duality between Uplink and Downlink Channels. Proceedings of the 7th International Telecommunications Symposium (ITS), Manaus, Brazil.","DOI":"10.14209\/sbrt.2010.23"},{"key":"ref_9","unstructured":"Visotsky, E., and Madhow, U. (1999, January 16\u201320). Optimum beamforming using transmit antenna arrays. Proceedings of the 49th IEEE Vehicular Technology Conference, Houston, TX, USA."},{"key":"ref_10","unstructured":"Yu, W., and Lan, T. (2005, January 5\u20138). Downlink beamforming with per-antenna power constraints. Proceedings of the 6th IEEE Workshop on Signal Processing Advances in Wireless Communications, New York, NY, USA."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2730","DOI":"10.1109\/TWC.2013.042313.120752","article-title":"Joint beamforming and power control in coordinated multicell: Max-min duality, effective network and large system transition","volume":"12","author":"Huang","year":"2013","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Khandaker, M.R.A., and Rong, Y. (2011, January 9\u201311). Joint power control and beamforming for peer-to-peer MIMO relay systems. Proceedings of the 2011 International Conference on Wireless Communications Signal Processing (WCSP), Nanjing, China.","DOI":"10.1109\/WCSP.2011.6096804"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lu, X., Li, W., Tolli, A., Juntti, M., Kunnari, E., and Piirainen, O. (2010, January 26\u201330). Joint power control, receiver beamforming and adaptive multi base station coordination for uplink wireless communications. Proceedings of the 21st IEEE International Symposium on Personal, Indoor and Mobile radio Communications Workshops, Instanbul, Turkey.","DOI":"10.1109\/PIMRCW.2010.5670528"},{"key":"ref_14","unstructured":"Hu, C., Wang, F., and Wang, W. (2010, January 11\u201314). Joint beamforming and power control optimization by second-order cone programming approximation. Proceedings of the 12th IEEE International Conference on Communication technology (ICCT), Nanjing, China."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"You, S., Noh, G., Lee, J., Wang, H., and Hong, D. (2010, January 18\u201321). Joint beamforming and power control algorithm for cognitive radio network with the multi-antenna base station. Proceedings of the IEEE Communications and Network Conference (WCNC), Sydney, Australia.","DOI":"10.1109\/WCNC.2010.5506457"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, Z., Yin, C., and Yue, G. (2012, January 21\u201323). A novel approach to joint beamforming and power control for the coordinated multicell multi-antenna system. Proceedings of the 8th International Conference on Wireless Communications, Networking and Mobile Computing (WiCOM), Shanghai, China.","DOI":"10.1109\/WiCOM.2012.6478320"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zeydan, E., Kivanc-Tureli, D., and Tureli, U. (2010, January 6\u201310). Iterative beamforming and power control for MIMO ad hoc networks. Proceedings of the IEEE Global Telecommunications Conference (GLOBECOM), Miami, FL, USA.","DOI":"10.1109\/GLOCOM.2010.5683754"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Gupta, N., and Reddy, A.L.N. (2007, January 19\u201320). Adaptive antenna using Fuzzy Logic Control. Proceedings of the IEEE Applied Electromagnetics Conference (AEMC), Kolkata, India.","DOI":"10.1109\/AEMC.2007.4638007"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1587\/elex.7.203","article-title":"Joint beamforming and power control in MIMO cognitive radio networks","volume":"7","author":"Noori","year":"2010","journal-title":"IEICE Elect. Express"},{"key":"ref_20","unstructured":"Yoshida, J., and Hirose, A. (2013, January 20\u201324). Beamforming for impulse-radio UWB communication systems based on complex-valued spatio-temporal neural networks. Proceedings of the International Symposium on Electromagnetic Theory, Hiroshima, Japan."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1109\/TBC.2013.2244793","article-title":"Design of a novel antenna array beamformer using neural networks trained by modified adaptive dispersion invasive weed optimization based data","volume":"59","author":"Zaharis","year":"2013","journal-title":"IEEE Trans. Broadcast."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Terabayashi, K., and Hirose, A. (2014, January 6\u201311). Ultra-short-pulse acoustic imaging using complex-valued spatio-temporal neural-network for null-steering: Experimental results. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Beijing, China.","DOI":"10.1109\/IJCNN.2014.6889695"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, Y., Zhang, P., and Hain, T. (2014, January 4\u20139). Using neural network front-ends on far field multiple microphones based speech recognition. Proceedings of the IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP), Florence, Italy.","DOI":"10.1109\/ICASSP.2014.6854663"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1016\/j.ins.2013.10.004","article-title":"An adaptive robust fuzzy beamformer for steering vector mismatch and reducing interference and noise","volume":"266","author":"Hung","year":"2014","journal-title":"Inf. Sci."},{"key":"ref_25","first-page":"641","article-title":"Neural Fuzzy Inference Based Robust Adaptive Beamforming","volume":"3","author":"Anitha","year":"2013","journal-title":"Int. J. Emerg. Technol. Adv. Eng."},{"key":"ref_26","unstructured":"Song, X., Wang, J., and Niu, X. (2008, January 1\u20133). Robust adaptive beamforming algorithm based on neural network. Proceedings of the IEEE International Conference on Automation and Logistics, Qingdao, China."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4939","DOI":"10.1016\/j.eswa.2014.01.040","article-title":"Reactive search strategies using reinforcement learning, local search algorithms and variable neighborhood search","volume":"41","author":"Santos","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Balanis, C.A., and Ioannides, P.I. (2007). Introduction to Smart Antennas, Morgan & Claypool Publishers.","DOI":"10.1007\/978-3-031-01533-5"},{"key":"ref_29","unstructured":"Monzingo, R.A., and Miller, T.W. (2004). Introduction to Adaptive Arrays, Scitech Publishing Inc."},{"key":"ref_30","unstructured":"Haykin, S. (1996). Adaptive Filter Theory, Prentice Hall. [3rd ed.]."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2143","DOI":"10.1109\/PROC.1967.6092","article-title":"Adaptive antenna systems","volume":"55","author":"Widrow","year":"1967","journal-title":"IEEE Proc."},{"key":"ref_32","unstructured":"Lima Junior, F.C., Melo, J.D., and D\u00f3ria Neto, A.D. (2008, January 1\u20138). Using the Q-learning algorithm in the constructive phase of the GRASP and reactive GRASP metaheuristics. Proceedings of the IEEE International Joint Conferecence on Neural Networks, Hong Kong, China."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sutton, R., and Barto, A. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_34","unstructured":"Peixoto, H.M., Diniz, A.A.R., and Almeida, N. (August, January 31). Modeling a system for monitoring an object using artificial neural networks and reinforcement learning. Proceedings of the International Joint Conference on Neural Networks, San Jose, CA, USA."},{"key":"ref_35","unstructured":"Watkins, C.J.C.H., and Dayan, P. (1992). Q-Learning: Machine Learning, Kluwer Academic Publishers."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/3\/6668\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:43:41Z","timestamp":1760215421000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/3\/6668"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,3,19]]},"references-count":35,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2015,3]]}},"alternative-id":["s150306668"],"URL":"https:\/\/doi.org\/10.3390\/s150306668","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2015,3,19]]}}}