{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T00:44:38Z","timestamp":1760229878401,"version":"build-2065373602"},"reference-count":18,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2022,6,29]],"date-time":"2022-06-29T00:00:00Z","timestamp":1656460800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2020R1I1A3061195"],"award-info":[{"award-number":["2020R1I1A3061195"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We investigate a power control problem for overlay device-to-device (D2D) communication networks relying on a deep deterministic policy gradient (DDPG), which is a model-free off-policy algorithm for learning continuous actions such as transmitting power levels. We propose a DDPG-based self-regulating power control scheme whereby each D2D transmitter can autonomously determine its transmission power level with only local channel gains that can be measured from the sounding symbols transmitted by D2D receivers. The performance of the proposed scheme is analyzed in terms of average sum-rate and energy efficiency and compared to several conventional schemes. Our numerical results show that the proposed scheme increases the average sum-rate compared to the conventional schemes, even with severe interference caused by increasing the number of D2D pairs or high transmission power, and the proposed scheme has the highest energy efficiency.<\/jats:p>","DOI":"10.3390\/s22134894","type":"journal-article","created":{"date-parts":[[2022,6,29]],"date-time":"2022-06-29T22:43:28Z","timestamp":1656542608000},"page":"4894","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Self-Regulating Power-Control Scheme Using Reinforcement Learning for D2D Communication Networks"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4832-5827","authenticated-orcid":false,"given":"Tae-Won","family":"Ban","sequence":"first","affiliation":[{"name":"Department of Intelligent Communication Engineering, Gyeongsang National University, Marine Science Bldg 807, Tongyeong-si 53064, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"136684","DOI":"10.1109\/ACCESS.2020.3010786","article-title":"An Economic Incentive for D2D Assisted Offloading Using Stackelberg Game","volume":"8","author":"Qiao","year":"2020","journal-title":"IEEE Access"},{"key":"ref_2","first-page":"56","article-title":"D2D communications in LTE advanced release 12","volume":"17","author":"Yasukawa","year":"2015","journal-title":"NTT Docomo Tech. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"198254","DOI":"10.1109\/ACCESS.2020.3030629","article-title":"Improve Energy Consumption and Signal Transmission Quality of Routings in Wireless Sensor Networks","volume":"8","author":"Yeh","year":"2020","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"111858","DOI":"10.1109\/ACCESS.2019.2934479","article-title":"Power Control for Full-Duplex D2D Communications Underlaying Cellular Networks","volume":"7","author":"Han","year":"2019","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4331","DOI":"10.1109\/TSP.2011.2147784","article-title":"An IterativelyWeighted MMSE Approach to Distributed Sum-UtilityMaximization for a MIMO Interfering Broadcast Channel","volume":"59","author":"Shi","year":"2011","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1215","DOI":"10.1109\/TNET.2013.2264633","article-title":"FlashLinQ: A Synchronous Distributed Scheduler for Peer-to-Peer Ad Hoc Networks","volume":"21","author":"Wu","year":"2013","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shen, K., and Yu, W. (2017, January 25\u201330). FPLinQ: A cooperative spectrum sharing strategy for device-to-device communications. Proceedings of the 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany.","DOI":"10.1109\/ISIT.2017.8006944"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"151968","DOI":"10.1109\/ACCESS.2020.3017799","article-title":"Power Control for Two-Way AF Relay Assisted D2D Communications Underlaying Cellular Networks","volume":"8","author":"Han","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3202","DOI":"10.1109\/TVT.2020.2964286","article-title":"Efficient Resource Allocation and Power Control for LTE-A D2D Communication with Pure D2D Model","volume":"69","author":"Lai","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1333","DOI":"10.1109\/LWC.2019.2916352","article-title":"Adaptive Power Control for D2D Communications in Downlink SWIPT Networks with Partial CSI","volume":"8","author":"Lim","year":"2019","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3143","DOI":"10.1109\/JIOT.2020.3014926","article-title":"Deep-Reinforcement-Learning-Based Proportional Fair Scheduling Control Scheme for Underlay D2D Communication","volume":"8","author":"Kumar","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"15880","DOI":"10.1109\/TVT.2020.3037060","article-title":"Deep Learning Based Power Allocation for Workload Driven Full-Duplex D2D-Aided Underlaying Networks","volume":"69","author":"Du","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Bi, Z., and Zhou, W. (2020, January 25\u201328). Deep Reinforcement Learning Based Power Allocation for D2D Network. Proceedings of the 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring), Antwerp, Belgium.","DOI":"10.1109\/VTC2020-Spring48590.2020.9129537"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1363","DOI":"10.1109\/TWC.2020.3032991","article-title":"Deep Reinforcement Learning for Joint Channel Selection and Power Control in D2D Networks","volume":"20","author":"Tan","year":"2021","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"122125","DOI":"10.1109\/ACCESS.2021.3109948","article-title":"Deep Learning-Based Power Control Scheme with Partial Channel Information in Overlay Device-to-Device Communication Systems","volume":"9","author":"Kim","year":"2021","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5702","DOI":"10.1109\/TWC.2021.3069638","article-title":"Distributed Deep Learning for Power Control in D2D Networks With Outdated Information","volume":"20","author":"Shi","year":"2021","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_17","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"9404","DOI":"10.1109\/TVT.2016.2519461","article-title":"On the Link Scheduling for Cellular-Aided Device-to-Device Networks","volume":"65","author":"Ban","year":"2016","journal-title":"IEEE Trans. Veh. Technol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/4894\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:40:06Z","timestamp":1760139606000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/4894"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,29]]},"references-count":18,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22134894"],"URL":"https:\/\/doi.org\/10.3390\/s22134894","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,6,29]]}}}