{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:41:20Z","timestamp":1760139680239,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,4,24]],"date-time":"2022-04-24T00:00:00Z","timestamp":1650758400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61902351, 61902353, 61872322"],"award-info":[{"award-number":["61902351, 61902353, 61872322"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004731","name":"Zhejiang Provincial Natural Science Foundation","doi-asserted-by":"publisher","award":["LY21F020022, LY21F020023, LR20F020003"],"award-info":[{"award-number":["LY21F020022, LY21F020023, LR20F020003"]}],"id":[{"id":"10.13039\/501100004731","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The combination of ambient backscatter (AB) communications (ABCs) and RF-powered cognitive radio networks (CRNs) deals with challenges of both energy supply and spectrum shortage, and improves the network performances. With the expansion of wireless networks, many applications raise requirements for both high-throughput and timely data. Driven by these facts, we study the long-term throughput optimization of the secondary network in the AB-assisted overlay CRN (ABO-CRN), ABCs, and CRNs with the age of information (AoI) constraint, which is a novel metric for measuring the freshness of data received by receivers. Due to the dynamic environment, complete knowledge of the environment could not be obtained. Then, the deep deterministic policy gradient (DDPG), a deep reinforcement learning (DRL) method that addresses decision issues in both continuous and discrete spaces, is deployed to address the throughput optimization. We consider the impacts of time and energy allocation on the reward when the AoI constraint can not be satisfied, and develop the corresponding reward functions. Furthermore, we analyze the impacts of the minimum throughput requirement and maximum allowable AoI on the throughput and AoI of the secondary networks in the ABO-CRN, ABCs, and CRNs. We compare the throughput optimization scheme under the AoI constraint with two baseline schemes (i.e., throughput-optimal (T-O) and AoI-optimal (A-O) baseline schemes), and the simulation results show that the throughput of the ABO-CRN is close to the optimal throughput of the T-O baseline scheme, and the AoI of the ABO-CRN is close to the optimal AoI of the A-O baseline scheme.<\/jats:p>","DOI":"10.3390\/s22093262","type":"journal-article","created":{"date-parts":[[2022,4,24]],"date-time":"2022-04-24T22:22:41Z","timestamp":1650838961000},"page":"3262","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["DDPG-Based Throughput Optimization with AoI Constraint in Ambient Backscatter-Assisted Overlay CRN"],"prefix":"10.3390","volume":"22","author":[{"given":"Xueli","family":"Jia","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3886-4288","authenticated-orcid":false,"given":"Kechen","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kaikai","family":"Chi","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4592-0631","authenticated-orcid":false,"given":"Xiaoying","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"7663","DOI":"10.1109\/TWC.2020.3015260","article-title":"Cooperative spectrum sensing optimization in energy-harvesting cognitive radio networks","volume":"19","author":"Liu","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1109\/TWC.2019.2946813","article-title":"Total throughput maximization of cooperative cognitive radio networks with energy harvesting","volume":"19","author":"Zheng","year":"2019","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2386","DOI":"10.1109\/JIOT.2017.2755620","article-title":"The future Internet of Things: Secure, efficient, and model-based","volume":"5","author":"Siegel","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1171","DOI":"10.1109\/JIOT.2021.3079455","article-title":"Energy management for secure transmission in wireless powered communication networks","volume":"9","author":"Zhang","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MWC.2016.7462479","article-title":"Wireless powered communication networks: Architectures, protocol designs, and standardization","volume":"23","author":"Niyato","year":"2016","journal-title":"IEEE Wirel. Commun."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1671","DOI":"10.1109\/JIOT.2017.2689777","article-title":"Minimization of transmission completion time in wireless powered communication networks","volume":"4","author":"Chi","year":"2017","journal-title":"IEEE Internet Things J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1109\/JERM.2018.2817364","article-title":"Compact multiband wireless energy harvesting based battery-free body area networks sensor for mobile healthcare","volume":"2","author":"Yang","year":"2018","journal-title":"IEEE J. Electromagn. Microw. Med. Biol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"6401","DOI":"10.1109\/TCOMM.2019.2921584","article-title":"Energy provision minimization in wireless powered communication networks with network throughput demand: TDMA or NOMA","volume":"67","author":"Chi","year":"2019","journal-title":"IEEE Trans. Commun."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2593","DOI":"10.1109\/TCOMM.2019.2962436","article-title":"Security and energy harvesting for MIMO-OFDM networks","volume":"68","author":"Hoang","year":"2019","journal-title":"IEEE Trans. Commun."},{"key":"ref_10","first-page":"1000","article-title":"Energy efficient resource allocation in wireless energy harvesting sensor networks","volume":"9","author":"Azarhava","year":"2020","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1109\/LWC.2021.3058170","article-title":"Learning to optimize energy efficiency in energy harvesting wireless sensor networks","volume":"10","author":"Ghosh","year":"2021","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/TWC.2019.2942296","article-title":"Efficient rendezvous for heterogeneous interference in cognitive radio networks","volume":"19","author":"Gu","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1818","DOI":"10.1109\/TNET.2020.2994134","article-title":"A practical spectrum sharing scheme for cognitive radio networks: Design and experiments","volume":"28","author":"Sangdeh","year":"2020","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4669","DOI":"10.1109\/TCOMM.2019.2912605","article-title":"Hybrid overlay-underlay cognitive radio networks with energy harvesting","volume":"67","author":"Zheng","year":"2019","journal-title":"IEEE Trans. Commun."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4195","DOI":"10.1109\/TWC.2020.2980821","article-title":"Network coding techniques for primary-secondary user cooperation in cognitive radio networks","volume":"19","author":"Papadopoulos","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"959","DOI":"10.1109\/TCCN.2020.2971703","article-title":"A secure spectrum handoff mechanism in cognitive radio networks","volume":"6","author":"Rathee","year":"2020","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hoang, D.T., Niyato, D., Kim, D.I., Van Huynh, N., and Gong, S. (2020). Ambient Backscatter Communication Networks, Cambridge University Press. [1st ed.].","DOI":"10.1017\/9781108691383"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1145\/2534169.2486015","article-title":"Ambient backscatter: Wireless communication out of thin air","volume":"43","author":"Liu","year":"2013","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"7265","DOI":"10.1109\/JIOT.2020.2984449","article-title":"On the outage performance of ambient backscatter communications","volume":"7","author":"Ye","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6884","DOI":"10.1109\/TWC.2021.3078051","article-title":"Enhancing ambient backscatter communication utilizing coherent and non-coherent space-time codes","volume":"20","author":"Liu","year":"2022","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Madavani, F.K., Soleimanpour-Moghadam, M., Talebi, S., Chatzinotas, S., and Ottersten, B. (2022). Joint resource allocation for full-duplex ambient backscatter communication: A difference convex algorithm. IEEE Trans. Wirel. Commun., in press.","DOI":"10.1109\/TWC.2022.3163718"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3659","DOI":"10.1109\/TCOMM.2017.2710338","article-title":"Ambient backscatter: A new approach to improve network performance for RF-powered cognitive radio networks","volume":"65","author":"Hoang","year":"2017","journal-title":"IEEE Trans. Commun."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"15216","DOI":"10.1109\/TVT.2020.3037152","article-title":"Optimal resource allocation for RF-powered underlay cognitive radio networks with ambient backscatter communication","volume":"69","author":"Zhuang","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1657","DOI":"10.1109\/TGCN.2021.3102660","article-title":"Distributed resource allocation in RF-powered cognitive ambient backscatter networks","volume":"5","author":"Zhu","year":"2021","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kaul, S., Yates, R., and Gruteser, M. (2012, January 25\u201330). Real-time status: How often should one update. Proceedings of the IEEE INFOCOM, Orlando, FL, USA.","DOI":"10.1109\/INFCOM.2012.6195689"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1109\/TCCN.2019.2916097","article-title":"Age of information minimization for an energy harvesting cognitive radio","volume":"5","author":"Leng","year":"2019","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"10273","DOI":"10.1109\/JIOT.2019.2937334","article-title":"Minimizing age of information in cognitive radio-based IoT systems: Underlay or overlay?","volume":"6","author":"Gu","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"8054","DOI":"10.1109\/TWC.2020.3019056","article-title":"Minimizing the age of information of cognitive radio-based IoT systems under a collision constraint","volume":"19","author":"Wang","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_29","unstructured":"Abbas, Q., Zeb, S., and Hassan, S.A. (2021). Wireless-Powered Backscatter Communications for Internet of Things, Springer International Publishing. [1st ed.]."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2488","DOI":"10.1109\/COMST.2019.2897800","article-title":"Enabling technologies for ultra-reliable and low latency communications: From PHY and MAC layer perspectives","volume":"21","author":"Sutton","year":"2019","journal-title":"IEEE Commun. Surv. Tuts."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1325","DOI":"10.1109\/JSAC.2021.3065061","article-title":"Not just age but age and quality of information","volume":"39","author":"Rajaraman","year":"2021","journal-title":"IEEE J. Sel. Area Commun."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, Q., Zeng, H., and Chen, M. (2019, January 2). Minimizing age-of-information with throughput requirements in multi-path network communication. Proceedings of the Twentieth ACM International Symposium on Mobile Ad Hoc Networking and Computing, New York, NY, USA.","DOI":"10.1145\/3323679.3326502"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1359","DOI":"10.1109\/TNET.2019.2918736","article-title":"Scheduling algorithms for optimizing age of information in wireless networks with throughput constraints","volume":"27","author":"Kadota","year":"2019","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_34","first-page":"481","article-title":"Throughput maximization with an average age of information constraint in fading channels","volume":"20","author":"Bhat","year":"2021","journal-title":"IEEE Trans. Commun."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","article-title":"Applications of deep reinforcement learning in communications and networking: A survey","volume":"21","author":"Luong","year":"2019","journal-title":"IEEE Commun. Surv. Tuts."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhu, B., Chi, K., Liu, J., Yu, K., and Mumtaz, S. (2022). Efficient offloading for minimizing task computation delay of NOMA-based multi-access edge computing. IEEE Trans. Commun., in press.","DOI":"10.1109\/TCOMM.2022.3162263"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1109\/TWC.2017.2769644","article-title":"User scheduling and resource allocation in hetNets with hybrid energy supply: An actor-critic reinforcement learning approach","volume":"17","author":"Wei","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_38","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016, January 2\u20134). Continuous control with deep reinforcement learning. Proceedings of the ICLR, San Juan, Puerto Rico."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2691","DOI":"10.1109\/JIOT.2018.2800716","article-title":"Outage performance analysis of wireless energy harvesting relay-assisted random underlay cognitive networks","volume":"5","author":"Yan","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Huynh, N.V., Hoang, D.T., Nguyen, D.N., Dutkiewicz, E., Niyato, D., and Wang, P. (2018, January 9\u201313). Reinforcement learning approach for RF-powered cognitive radio network with ambient backscatter. Proceedings of the IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/GLOCOM.2018.8647862"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1109\/LWC.2015.2456066","article-title":"On the convergence of multipath fading channel gains to the rayleigh distribution","volume":"4","author":"Taricco","year":"2015","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_42","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 7\u20139). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the PMLR, Lille, France."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3262\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:59:42Z","timestamp":1760137182000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3262"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,24]]},"references-count":42,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["s22093262"],"URL":"https:\/\/doi.org\/10.3390\/s22093262","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,4,24]]}}}