{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T15:58:51Z","timestamp":1780588731530,"version":"3.54.1"},"reference-count":32,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,12,5]],"date-time":"2024-12-05T00:00:00Z","timestamp":1733356800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This paper investigates the design of active and passive beamforming in a reconfigurable intelligent surface (RIS)-aided multi-user multiple-input single-output (MU-MISO) system with the objective of maximizing the sum rate. We propose a deep evolution policy (DEP)-based algorithm to derive the optimal beamforming strategy by generating multiple agents, each utilizing distinct deep neural networks (DNNs). Additionally, a random subspace selection (RSS) strategy is incorporated to effectively balance exploitation and exploration. The proposed DEP-based algorithm operates without the need for alternating iterations, gradient descent, or backpropagation, enabling simultaneous optimization of both active and passive beamforming. Simulation results indicate that the proposed algorithm can bring significant performance enhancements.<\/jats:p>","DOI":"10.3390\/e26121056","type":"journal-article","created":{"date-parts":[[2024,12,5]],"date-time":"2024-12-05T04:13:03Z","timestamp":1733371983000},"page":"1056","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A Deep Evolution Policy-Based Approach for RIS-Enhanced Communication System"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-8050-3303","authenticated-orcid":false,"given":"Ke","family":"Zhao","sequence":"first","affiliation":[{"name":"54th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang 050081, China"},{"name":"National Key Laboratory of Advanced Communication Networks, Shijiazhuang 050081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhiqun","family":"Song","sequence":"additional","affiliation":[{"name":"54th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang 050081, China"},{"name":"National Key Laboratory of Advanced Communication Networks, Shijiazhuang 050081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yong","family":"Li","sequence":"additional","affiliation":[{"name":"54th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang 050081, China"},{"name":"National Key Laboratory of Advanced Communication Networks, Shijiazhuang 050081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xingjian","family":"Li","sequence":"additional","affiliation":[{"name":"54th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang 050081, China"},{"name":"National Key Laboratory of Advanced Communication Networks, Shijiazhuang 050081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lizhe","family":"Liu","sequence":"additional","affiliation":[{"name":"54th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang 050081, China"},{"name":"National Key Laboratory of Advanced Communication Networks, Shijiazhuang 050081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bin","family":"Wang","sequence":"additional","affiliation":[{"name":"54th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang 050081, China"},{"name":"National Key Laboratory of Advanced Communication Networks, Shijiazhuang 050081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3313","DOI":"10.1109\/TCOMM.2021.3051897","article-title":"Intelligent reflecting surface-aided wireless communications: A tutorial","volume":"69","author":"Wu","year":"2021","journal-title":"IEEE Trans. Commun."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2063","DOI":"10.1109\/LCOMM.2021.3062615","article-title":"Reconfigurable intelligent surfaces in 6G: Reflective, transmissive, or both?","volume":"25","author":"Zeng","year":"2021","journal-title":"IEEE Commun. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1109\/MCOM.002.2200047","article-title":"Reconfigurable-Intelligent-Surface-Assisted B5G\/6G Wireless Communications: Challenges, Solution, and Future Opportunities","volume":"61","author":"Chen","year":"2023","journal-title":"IEEE Commun. Mag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1109\/MWC.011.2100016","article-title":"Reconfigurable Intelligent Surfaces: Potentials, Applications, and Challenges for 6G Wireless Networks","volume":"6","author":"Basharat","year":"2021","journal-title":"IEEE Wirel. Commun."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1109\/MCOM.001.1900107","article-title":"Towards Smart and Reconfigurable Environment: Intelligent Reflecting Surface Aided Wireless Network","volume":"1","author":"Wu","year":"2020","journal-title":"IEEE Commun. Mag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1546","DOI":"10.1109\/COMST.2021.3077737","article-title":"Reconfigurable Intelligent Surfaces: Principles and Opportunities","volume":"3","author":"Liu","year":"2021","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"5394","DOI":"10.1109\/TWC.2019.2936025","article-title":"Intelligent reflecting surface enhanced wireless network via joint active and passive beamforming","volume":"18","author":"Wu","year":"2019","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_8","first-page":"1455","article-title":"Manopt, a Matlab toolbox for optimization on manifolds","volume":"15","author":"Boumal","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3064","DOI":"10.1109\/TWC.2020.2970061","article-title":"Weighted sum-rate maximization for reconfigurable intelligent surface aided wireless networks","volume":"19","author":"Guo","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TVT.2024.3452183","article-title":"Efficient Target Search and Detection in RIS-aided Integrated Sensing and Communications System","volume":"6","author":"Xiao","year":"2024","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1109\/TCCN.2021.3128605","article-title":"A robust deep learning-based beamforming design for RIS-assisted multiuser MISO communications with practical constraints","volume":"8","author":"Xu","year":"2021","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1109\/LWC.2023.3327769","article-title":"Deep Learning Based Hybrid Precoding for RIS-aided Broadband Terahertz Communication Systems in the Face of Beam Squint","volume":"13","author":"Yuan","year":"2023","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_13","first-page":"1","article-title":"Learn to Optimize RIS Aided Hybrid Beamforming With Out-of-Distribution Generalization","volume":"7","author":"He","year":"2024","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2060","DOI":"10.1109\/LWC.2024.3400279","article-title":"Deep Learning-Based CSI Feedback for RIS-Aided Massive MIMO Systems With Time Correlation","volume":"13","author":"Peng","year":"2024","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/IOTM.001.2300132","article-title":"Deep Learning for Secure UAV-Assisted RIS Communication Networks","volume":"2","author":"Mughal","year":"2024","journal-title":"IEEE Internet Things Mag."},{"key":"ref_16","first-page":"241","article-title":"Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks","volume":"23","author":"Hoefler","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_17","unstructured":"Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. (2017). Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. arXiv."},{"key":"ref_18","unstructured":"Zaheer, M., Guruganesh, G., Dubey, A., Ainslie, J., Alberti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q., and Yang, L. (2021). Big Bird: Transformers for Longer Sequences. arXiv."},{"key":"ref_19","unstructured":"Jia, J., Liu, J., Ram, P., Yao, Y., Liu, G., Liu, Y., Sharma, P., and Liu, S. (2023). Model Sparsity Can Simplify Machine Unlearning. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1839","DOI":"10.1109\/JSAC.2020.3000835","article-title":"Reconfigurable intelligent surface assisted multiuser MISO systems exploiting deep reinforcement learning","volume":"38","author":"Huang","year":"2020","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1493","DOI":"10.1109\/LWC.2022.3176666","article-title":"Deep reinforcement learning based on location-aware imitation environment for RIS-aided mmWave MIMO systems","volume":"11","author":"Xu","year":"2022","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"259","DOI":"10.23919\/JCC.fa.2022-0421.202304","article-title":"Deep reinforcement learning based power minimization for RIS-assisted MISO-OFDM systems","volume":"20","author":"Chen","year":"2023","journal-title":"China Commun."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Saglam, B., Gurgunoglu, D., and Kozat, S.S. (June, January 28). Deep reinforcement learning based joint downlink beamforming and RIS configuration in RIS-aided MU-MISO systems under hardware impairments and imperfect CSI. Proceedings of the 2023 IEEE International Conference on Communications Workshops (ICC Workshops), Rome, Italy.","DOI":"10.1109\/ICCWorkshops57953.2023.10283517"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1109\/LWC.2023.3242449","article-title":"DRL-based RIS phase shift design for OFDM communication systems","volume":"12","author":"Chen","year":"2023","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.aej.2024.01.039","article-title":"Deep reinforcement learning based rate enhancement scheme for RIS assisted mobile users underlaying UAV","volume":"91","author":"Joshi","year":"2024","journal-title":"Alex. Eng. J."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"28050","DOI":"10.1109\/JIOT.2024.3416334","article-title":"Deep Reinforcement Learning Based Uplink Security Enhancement for STAR-RIS-Assisted NOMA Systems With Dual Eavesdroppers","volume":"11","author":"Qin","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"8091","DOI":"10.1007\/s11042-020-10139-6","article-title":"A review on genetic algorithm: Past, present, and future","volume":"80","author":"Katoch","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1109\/COMST.2023.3340099","article-title":"A Survey on Model-Based, Heuristic, and Machine Learning Optimization Approaches in RIS-Aided Wireless Networks","volume":"2","author":"Zhou","year":"2024","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1109\/MWC.010.2300321","article-title":"Heuristic Algorithms for RIS-Assisted Wireless Networks: Exploring Heuristic-Aided Machine Learning","volume":"31","author":"Zhou","year":"2024","journal-title":"IEEE Wirel. Commun."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"110680","DOI":"10.1016\/j.asoc.2023.110680","article-title":"NeuroCrossover: An intelligent genetic locus selection scheme for genetic algorithm using reinforcement learning","volume":"146","author":"Liu","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3558","DOI":"10.1109\/TCOMM.2022.3162580","article-title":"Power Scaling Law Analysis and Phase Shift Optimization of RIS-Aided Massive MIMO Systems With Statistical CSI","volume":"5","author":"Zhi","year":"2022","journal-title":"IEEE Trans. Commun."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1128","DOI":"10.1109\/LWC.2021.3059938","article-title":"Statistical CSI-Based Design for Reconfigurable Intelligent Surface-Aided Massive MIMO Systems With Direct Links","volume":"5","author":"Zhi","year":"2021","journal-title":"IEEE Wirel. Commun. Lett."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/12\/1056\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:47:26Z","timestamp":1760114846000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/12\/1056"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,5]]},"references-count":32,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["e26121056"],"URL":"https:\/\/doi.org\/10.3390\/e26121056","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,5]]}}}