{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T12:10:44Z","timestamp":1775218244402,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2022,7,3]],"date-time":"2022-07-03T00:00:00Z","timestamp":1656806400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ontario Centers of Excellence (OCE) 5G ENCQOR program and Ciena"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Open radio access network (O-RAN) is one of the promising candidates for fulfilling flexible and cost-effective goals by considering openness and intelligence in its architecture. In the O-RAN architecture, a central unit (O-CU) and a distributed unit (O-DU) are virtualized and executed on processing pools of general-purpose processors that can be placed at different locations. Therefore, it is challenging to choose a proper location for executing network functions (NFs) over these entities by considering propagation delay and computational capacity. In this paper, we propose a Soft Actor\u2013Critic Energy-Aware Dynamic DU Selection algorithm (SA2C-EADDUS) by integrating two nested actor\u2013critic agents in the O-RAN architecture. In addition, we formulate an optimization model that minimizes delay and energy consumption. Then, we solve that problem with an MILP solver and use that solution as a lower bound comparison for our SA2C-EADDUS algorithm. Moreover, we compare that algorithm with recent works, including RL- and DRL-based resource allocation algorithms and a heuristic method. We show that by collaborating A2C agents in different layers and by dynamic relocation of NFs, based on service requirements, our schemes improve the energy efficiency by 50% with respect to other schemes. Moreover, we reduce the mean delay by a significant amount with our novel SA2C-EADDUS approach.<\/jats:p>","DOI":"10.3390\/s22135029","type":"journal-article","created":{"date-parts":[[2022,7,4]],"date-time":"2022-07-04T23:38:55Z","timestamp":1656977935000},"page":"5029","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Energy-Aware Dynamic DU Selection and NF Relocation in O-RAN Using Actor\u2013Critic Learning"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4869-1934","authenticated-orcid":false,"given":"Shahram","family":"Mollahasani","sequence":"first","affiliation":[{"name":"School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6N5, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1444-8263","authenticated-orcid":false,"given":"Turgay","family":"Pamuklu","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6N5, Canada"}]},{"given":"Rodney","family":"Wilson","sequence":"additional","affiliation":[{"name":"Ciena, Ottawa, ON K2K 0L1, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6787-8457","authenticated-orcid":false,"given":"Melike","family":"Erol-Kantarci","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6N5, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Klinkowski, M. (2020). Latency-Aware DU\/CU Placement in Convergent Packet-Based 5G Fronthaul Transport Networks. Appl. Sci., 10.","DOI":"10.3390\/app10217429"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Semov, P., Koleva, P., Tonchev, K., Poulkov, V., and Cooklev, T. (2020, January 7\u20139). Evolution of mobile networks and C-RAN on the road beyond 5G. Proceedings of the 2020 43rd International Conference on Telecommunications and Signal Processing (TSP), Milan, Italy.","DOI":"10.1109\/TSP49548.2020.9163513"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dryjanski, M., Ku\u0142acz, \u0141., and Kliks, A. (2021). Toward Modular and Flexible Open RAN Implementations in 6G Networks: Traffic Steering Use Case and O-RAN xApps. Sensors, 21.","DOI":"10.3390\/s21248173"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/j.comnet.2018.01.021","article-title":"A comprehensive survey of network function virtualization","volume":"133","author":"Yi","year":"2018","journal-title":"Comput. Netw."},{"key":"ref_5","unstructured":"Gilson, M., Mackenzie, R., Sutton, A., and Huang, J. (2018). NGMN Overview on 5G RAN Functional Decomposition, NGMN Alliance."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/TGCN.2020.3042121","article-title":"GROVE: A Cost-Efficient Green Radio Over Ethernet Architecture for Next Generation Radio Access Networks","volume":"5","author":"Pamuklu","year":"2021","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Mollahasani, S., Erol-Kantarci, M., and Wilson, R. (2021, January 7\u201311). Dynamic CU-DU Selection for Resource Allocation in O-RAN Using actor\u2013critic Learning. Proceedings of the IEEE Global Communications Conference (GLOBECOM), Madrid, Spain.","DOI":"10.1109\/GLOBECOM46510.2021.9685837"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1109\/COMST.2015.2403395","article-title":"Energy-efficient base-stations sleep-mode techniques in green cellular networks: A survey","volume":"17","author":"Wu","year":"2015","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2126","DOI":"10.1109\/TWC.2013.032013.120494","article-title":"Dynamic base station switching-on\/off strategies for green cellular networks","volume":"12","author":"Oh","year":"2013","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/MWC.2011.6056689","article-title":"TANGO: Traffic-aware network planning and green operation","volume":"18","author":"Niu","year":"2011","journal-title":"IEEE Wirel. Commun."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"65852","DOI":"10.1109\/ACCESS.2019.2917722","article-title":"Density-aware, energy-and spectrum-efficient small cell scheduling","volume":"7","author":"Mollahasani","year":"2019","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1109\/LWC.2015.2393355","article-title":"Baseband processing units virtualization for cloud radio access networks","volume":"4","author":"Qian","year":"2015","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.1109\/JSAC.2016.2520247","article-title":"Energy-efficient virtual base station formation in optical-access-enabled cloud-RAN","volume":"34","author":"Wang","year":"2016","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1637","DOI":"10.1109\/LCOMM.2017.2695545","article-title":"Energy-efficient BBU allocation for green C-RAN","volume":"21","author":"Sahu","year":"2017","journal-title":"IEEE Commun. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1010","DOI":"10.1109\/JSAC.2016.2549438","article-title":"Traffic-aware cloud RAN: A key for green 5G networks","volume":"34","author":"Saxena","year":"2016","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1756","DOI":"10.1109\/TNET.2019.2931038","article-title":"An Optimization-Enhanced MANO for Energy-Efficient 5G Networks","volume":"27","author":"Malandrino","year":"2019","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1109\/COMST.2018.2868805","article-title":"A survey of the functional splits proposed for 5G mobile crosshaul networks","volume":"21","author":"Larsen","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"982","DOI":"10.1109\/TGCN.2018.2869294","article-title":"Multiplexing gain and processing savings of 5G radio-access-network functional splits","volume":"2","author":"Shehata","year":"2018","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1109\/TGCN.2018.2802419","article-title":"Optimal processing allocation to minimize energy and bandwidth consumption in hybrid CRAN","volume":"2","author":"Alabbasi","year":"2018","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Akoush, S., Sohan, R., Rice, A., Moore, A.W., and Hopper, A. (2010, January 17\u201319). Predicting the performance of virtual machine migration. Proceedings of the 2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, Miami Beach, FL, USA.","DOI":"10.1109\/MASCOTS.2010.13"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2788397","article-title":"Cloud computing resource scheduling and a survey of its evolutionary approaches","volume":"47","author":"Zhan","year":"2015","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/MVT.2019.2919236","article-title":"AI-enabled future wireless networks: Challenges, opportunities, and open issues","volume":"14","author":"Elsayed","year":"2019","journal-title":"IEEE Veh. Technol. Mag."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"\u015eahin, T., Khalili, R., Boban, M., and Wolisz, A. (2018, January 5\u20137). Reinforcement learning scheduler for vehicle-to-vehicle communications outside coverage. Proceedings of the 2018 IEEE Vehicular Networking Conference (VNC), Taipei, Taiwan.","DOI":"10.1109\/VNC.2018.8628366"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Pamuklu, T., Erol-Kantarci, M., and Ersoy, C. (2021, January 14\u201323). Reinforcement Learning Based Dynamic Function Splitting in Disaggregated Green Open RANs. Proceedings of the IEEE International Conference on Communications, Montreal, QC, Canada.","DOI":"10.1109\/ICC42927.2021.9500721"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2838","DOI":"10.1109\/TWC.2020.3044597","article-title":"Transfer Reinforcement Learning for 5G-NR mm-Wave Networks","volume":"20","author":"Elsayed","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, T., Shen, S., Mao, S., and Chang, G.K. (2020, January 7\u201311). Delay-aware Cellular Traffic Scheduling with Deep Reinforcement Learning. Proceedings of the GLOBECOM 2020\u20142020 IEEE Global Communications Conference, Taipei, Taiwan.","DOI":"10.1109\/GLOBECOM42002.2020.9322560"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, G., Zhang, X., Shen, F., and Zeng, Q. (2022). Two Tier Slicing Resource Allocation Algorithm Based on Deep Reinforcement Learning and Joint Bidding in Wireless Access Networks. Sensors, 22.","DOI":"10.3390\/s22093495"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/TNSE.2021.3070476","article-title":"actor\u2013critic Learning Based QoS-Aware Scheduler for Reconfigurable Wireless Networks","volume":"9","author":"Mollahasani","year":"2021","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Pamuklu, T., Mollahasani, S., and Erol-Kantarci, M. (2021, January 13\u201315). Energy-Efficient and Delay-Guaranteed Joint Resource Allocation and DU Selection in O-RAN. Proceedings of the 5G World Forum (5GWF), Montreal, QC, Canada.","DOI":"10.1109\/5GWF52925.2021.00025"},{"key":"ref_30","unstructured":"O-RAN Alliance (2021). O-RAN-WG1-O-RAN Architecture Description\u2014v04.00.00, O-RAN Alliance. Technical Specification."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Yu, Y.J., Pang, A.C., Hsiu, P.C., and Fang, Y. (2013, January 9\u201313). Energy-efficient downlink resource allocation for mobile devices in wireless systems. Proceedings of the 2013 IEEE Global Communications Conference (GLOBECOM), Atlanta, GA, USA.","DOI":"10.1109\/GLOCOMW.2013.6855692"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/MCOM.101.2001120","article-title":"Intelligence and Learning in O-RAN for Data-Driven NextG Cellular Networks","volume":"59","author":"Bonati","year":"2021","journal-title":"IEEE Commun. Mag."},{"key":"ref_33","unstructured":"ITU (2018). ITU-T Recommendation G Suppl. 66. 5G Wireless Fronthaul Requirements in a Passive Optical Network Context, International Telecommunications Union. Technical Report."},{"key":"ref_34","unstructured":"3GPP (2020). Table 6.1.7-A: Standardized QCI Characteristics from 3GPP TS 23.203 V16.1.0, 3GPP. Technical Report."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gaw\u0142owicz, P., and Zubow, A. (2019, January 25\u201329). NS-3 meets openai gym: The playground for machine learning in networking research. Proceedings of the 22nd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, Miami Beach, FL, USA.","DOI":"10.1145\/3345768.3355908"},{"key":"ref_36","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/5029\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:42:21Z","timestamp":1760139741000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/5029"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,3]]},"references-count":36,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22135029"],"URL":"https:\/\/doi.org\/10.3390\/s22135029","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,3]]}}}