{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T15:53:34Z","timestamp":1783439614770,"version":"3.54.6"},"reference-count":26,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2016,10,12]],"date-time":"2016-10-12T00:00:00Z","timestamp":1476230400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Key Development Program of Basic Research of China","award":["JCKY2013604B001"],"award-info":[{"award-number":["JCKY2013604B001"]}]},{"name":"Nation Nature Science Foundation of China","award":["61301095"],"award-info":[{"award-number":["61301095"]}]},{"name":"Nature Science Foundation of Heilongjiang Province of China","award":["F201408"],"award-info":[{"award-number":["F201408"]}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["No. HEUCF100814 and HEUCF100816"],"award-info":[{"award-number":["No. HEUCF100814 and HEUCF100816"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Cognitive radio sensor networks are one of the kinds of application where cognitive techniques can be adopted and have many potential applications, challenges and future research trends. According to the research surveys, dynamic spectrum access is an important and necessary technology for future cognitive sensor networks. Traditional methods of dynamic spectrum access are based on spectrum holes and they have some drawbacks, such as low accessibility and high interruptibility, which negatively affect the transmission performance of the sensor networks. To address this problem, in this paper a new initialization mechanism is proposed to establish a communication link and set up a sensor network without adopting spectrum holes to convey control information. Specifically, firstly a transmission channel model for analyzing the maximum accessible capacity for three different polices in a fading environment is discussed. Secondly, a hybrid spectrum access algorithm based on a reinforcement learning model is proposed for the power allocation problem of both the transmission channel and the control channel. Finally, extensive simulations have been conducted and simulation results show that this new algorithm provides a significant improvement in terms of the tradeoff between the control channel reliability and the efficiency of the transmission channel.<\/jats:p>","DOI":"10.3390\/s16101675","type":"journal-article","created":{"date-parts":[[2016,10,12]],"date-time":"2016-10-12T10:18:49Z","timestamp":1476267529000},"page":"1675","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":82,"title":["A Novel Dynamic Spectrum Access Framework Based on Reinforcement Learning for Cognitive Radio Sensor Networks"],"prefix":"10.3390","volume":"16","author":[{"given":"Yun","family":"Lin","sequence":"first","affiliation":[{"name":"College of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chao","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiaxing","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing Huawei Digital Technologies Co., Ltd., Beijing 100032, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zheng","family":"Dou","sequence":"additional","affiliation":[{"name":"College of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2016,10,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1109\/MSP.2007.361604","article-title":"A Survey of Dynamic Spectrum Access: Signal Processing, Networking, and Regulatory Policy","volume":"24","author":"Zhao","year":"2007","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"11197","DOI":"10.3390\/s130911196","article-title":"Cognitive Radio Wireless Sensor Networks: Applications, Challenges and Research Trends","volume":"13","author":"Joshi","year":"2013","journal-title":"Sensors"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"27760","DOI":"10.3390\/s151127760","article-title":"A Novel Wireless Power Transfer-Based Weighed Clustering Cooperative Spectrum Sensing Method for Cognitive Sensor Networks","volume":"15","author":"Liu","year":"2015","journal-title":"Sensors"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kondareddy, Y.R., Agrawal, P., and Sivalingam, K. (2008, January 16\u201319). Cognitive radio network setup without a common control channel. Proceedings of the 2008 IEEE Military Communications Conference (MILCOM 2008), San Diego, CA, USA.","DOI":"10.1109\/MILCOM.2008.4753398"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2575","DOI":"10.1109\/TWC.2010.070610.091149","article-title":"Dynamic spectrum access using a network coded cognitive control channel","volume":"9","author":"Baldo","year":"2010","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"19783","DOI":"10.3390\/s150819783","article-title":"An Energy-Efficient Spectrum-Aware Reinforcement Learning-Based Clustering Algorithm for Cognitive Radio Sensor Networks","volume":"15","author":"Cormio","year":"2015","journal-title":"Sensors"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1109\/TMC.2010.57","article-title":"Access strategies for spectrum sharing in fading environment: Overlay, underlay and mixed","volume":"9","author":"Khoshkholgh","year":"2010","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1109\/TIT.2006.889016","article-title":"On Capacity under Receive and Spatial Spectrum Sharing Constraints","volume":"53","author":"Gastpar","year":"2007","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_9","unstructured":"Viterbi, A.J. (1995). CDMA: Principles of Spread Spectrum Communication, Addison-Wesley."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3794","DOI":"10.1109\/TCOMM.2009.12.080400","article-title":"Novel Overlay\/Underlay Cognitive Radio Waveforms Using SD-SMSE Framework to Enhance Spectrum Efficiency\u2014Part I: Theoretical Framework and Analysis in AWGN Channel","volume":"57","author":"Chakravarthy","year":"2009","journal-title":"IEEE Trans. Commun."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1868","DOI":"10.1109\/TCOMM.2010.06.090176","article-title":"Novel Overlay\/Underlay Cognitive Radio Waveforms Using SD-SMSE Framework to Enhance Spectrum Efficiency\u2014Part II: Analysis in Fading Channel","volume":"58","author":"Chakravarthy","year":"2010","journal-title":"IEEE Trans. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2038","DOI":"10.1109\/TVT.2015.2417195","article-title":"Hybrid Overlay\/Underlay Cognitive Radio Network with MC-CDMA","volume":"65","author":"Jasbi","year":"2016","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1109\/JSAC.2008.080111","article-title":"Cross-Layer Based Opportunistic MAC Protocols for QOS Provisioning over Cognitive Radio Wireless Networks","volume":"26","author":"Su","year":"2008","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1109\/18.825799","article-title":"The Capacity of Wireless Networks","volume":"46","author":"Gupta","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Tse, D., and Viswanath, P. (2004). Fundamentals of Wireless Communication, Cambridge University Press.","DOI":"10.1017\/CBO9780511807213"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1109\/JSAC.2007.070403","article-title":"Capacity Limits of Cognitive Radio with Distributed and Dynamic Spectral Activity","volume":"25","author":"Jafar","year":"2007","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1109\/TWC.2007.05447","article-title":"Fundamental Limits of Spectrum Sharing in Fading Environments","volume":"6","author":"Ghasemi","year":"2007","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_18","unstructured":"Ross, S.M. (2012). A First Course in Probability, University of Southern California Press."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1109\/TMC.2010.15","article-title":"Achievable Capacity in Hybrid DS-CDMA\/OFDM Spectrum-Sharing","volume":"9","author":"Khoshkholgh","year":"2010","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Boyd, S., and Vandenberghe, L. (2004). Convex Optimization, Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"ref_21","unstructured":"Papoulis, A., and Pillai, S.U. (2002). Probability, Random Variables, and Stochastic Processes, McGraw-Hill. [4th ed.]."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement Learning: A Survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_23","unstructured":"Brodersen, R.W., Wolisz, A., Cabric, D., Mishra, S.M., and Willkomm, D. (2004). CORVUS: A Cognitive Radio Approach for Usage of Virtual Unlicensed Spectrum, University of Berkeley. White Paper."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bush, R.R., and Mosteller, F. (1955). Stochastic Models for Learning, John Wiley & Sons.","DOI":"10.1037\/14496-000"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1016\/S0899-8256(05)80020-X","article-title":"Learning in Extensive Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Run","volume":"8","author":"Erev","year":"1995","journal-title":"Games Econ. Behav."},{"key":"ref_26","first-page":"143","article-title":"Review of Learning Model and Experiment Based on Learning Theory","volume":"6","author":"Li","year":"2013","journal-title":"Sci. Technol. Manag. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/10\/1675\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:32:45Z","timestamp":1760211165000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/10\/1675"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,10,12]]},"references-count":26,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2016,10]]}},"alternative-id":["s16101675"],"URL":"https:\/\/doi.org\/10.3390\/s16101675","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,10,12]]}}}