{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:46:54Z","timestamp":1760240814077,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2019,9,21]],"date-time":"2019-09-21T00:00:00Z","timestamp":1569024000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004735","name":"Natural Science Foundation of\u00a0Hunan Province","doi-asserted-by":"publisher","award":["2018JJ3607","2019JJ50121"],"award-info":[{"award-number":["2018JJ3607","2019JJ50121"]}],"id":[{"id":"10.13039\/501100004735","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["51575517"],"award-info":[{"award-number":["51575517"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Technology Foundation Project","award":["181GF22006"],"award-info":[{"award-number":["181GF22006"]}]},{"name":"Frontier Science and Technology Innovation Project in the National Key Research and Development Program","award":["2016QY11W2003"],"award-info":[{"award-number":["2016QY11W2003"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the development of wireless communication technology, cognitive radio needs to solve the spectrum sensing problem of wideband wireless signals. Due to performance limitation of electronic components, it is difficult to complete spectrum sensing of wideband wireless signals at once. Therefore, it is required that the wideband wireless signal has to be split into a set of sub-bands before the further signal processing. However, the sequence of sub-band perception has become one of the important factors, which deeply-impact wideband spectrum sensing performance. In this paper, we develop a novel approach for sub-band selection through the non-stationary multi-arm bandit (NS-MAB) model. This approach is based on a well-known order optimal policy for NS-MAB mode called discounted upper confidence bound (D-UCB) policy. In this paper, according to different application requirements, various discount functions and exploration bonuses of D-UCB are designed, which are taken as the parameters of the policy proposed in this paper. Our simulation result demonstrates that the proposed policy can provide lower cumulative regret than other existing state-of-the-art policies for sub-band selection of wideband spectrum sensing.<\/jats:p>","DOI":"10.3390\/s19194090","type":"journal-article","created":{"date-parts":[[2019,9,23]],"date-time":"2019-09-23T03:26:32Z","timestamp":1569209192000},"page":"4090","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["A Policy for Optimizing Sub-Band Selection Sequences in Wideband Spectrum Sensing"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5607-7096","authenticated-orcid":false,"given":"Yangyi","family":"Chen","sequence":"first","affiliation":[{"name":"College of Intelligent Science and Technology, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shaojing","family":"Su","sequence":"additional","affiliation":[{"name":"College of Intelligent Science and Technology, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junyu","family":"Wei","sequence":"additional","affiliation":[{"name":"College of Intelligent Science and Technology, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,9,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5962","DOI":"10.1109\/JIOT.2018.2847731","article-title":"A Novel Multichannel Internet of Things Based on Dynamic Spectrum Sharing in 5G Communication","volume":"6","author":"Liu","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"97887","DOI":"10.1109\/ACCESS.2019.2929915","article-title":"Technical Issues on Cognitive Radio-Based Internet of Things Systems: A Survey","volume":"7","author":"Awin","year":"2019","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1056","DOI":"10.1109\/COMST.2018.2794358","article-title":"Wireless Multimedia Cognitive Radio Networks: A Comprehensive Survey","volume":"20","author":"Amjad","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, C., Wang, H., Zhang, J., and He, Z. (2018). Wideband Spectrum Sensing Based on Single-Channel Sub-Nyquist Sampling for Cognitive Radio. Sensors, 18.","DOI":"10.3390\/s18072222"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Arjoune, Y., and Kaabouch, N. (2019). A Comprehensive Survey on Spectrum Sensing in Cognitive Radio Networks: Recent Advances, New Challenges, and Future Research Directions. Sensors, 19.","DOI":"10.3390\/s19010126"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1109\/MCOM.2018.1700719","article-title":"Compressed Wideband Spectrum Sensing: Concept, Challenges, and Enablers","volume":"56","author":"Hamdaoui","year":"2018","journal-title":"IEEE Commun. Mag."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Jayaweera, S.K. (2014). Signal Processing for Cognitive Radios, John Wiley & Sons.","DOI":"10.1002\/9781118824818"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1109\/ACCESS.2017.2761910","article-title":"Multi-Modal Cooperative Spectrum Sensing Based on Dempster-Shafer Fusion in 5G-Based Cognitive Radio","volume":"6","author":"Liu","year":"2018","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"238","DOI":"10.1109\/COMST.2018.2863681","article-title":"Blind Spectrum Sensing Approaches for Interweaved Cognitive Radio System: A Tutorial and Short Course","volume":"21","author":"Awin","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Chen, Y., Su, S., Yin, H., Guo, X., Zuo, Z., Wei, J., and Zhang, L. (2019). Optimized Non-Cooperative Spectrum Sensing Algorithm in Cognitive Wireless Sensor Networks. Sensors, 19.","DOI":"10.3390\/s19092174"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3801","DOI":"10.1109\/ACCESS.2017.2677976","article-title":"Optimal Resource Allocation in Simultaneous Cooperative Spectrum Sensing and Energy Harvesting for Multichannel Cognitive Radio","volume":"5","author":"Liu","year":"2017","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1214","DOI":"10.1109\/TSP.2015.2391072","article-title":"An Order Optimal Policy for Exploiting Idle Spectrum in Cognitive Radio Networks","volume":"63","author":"Oksanen","year":"2015","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_13","unstructured":"Oksanen, J. (2016). Machine Learning Methods for Spectrum Exploration and Exploitation, Aalto University."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1093\/biomet\/25.3-4.285","article-title":"On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two Samples","volume":"25","author":"Thompson","year":"1933","journal-title":"Biometrika"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1090\/S0002-9904-1952-09620-8","article-title":"Some aspects of the sequential design of experiments","volume":"58","author":"Robbins","year":"1952","journal-title":"Bull. Am. Math. Soc."},{"key":"ref_16","unstructured":"Auer, P., Cesa-Bianchi, N., Freund, Y., and Schapire, R.E. (1995, January 23\u201325). Gambling in a Rigged Casino: The Adversarial Multi-Armed Bandit Problem. Proceedings of the IEEE 36th Annual Foundations of Computer Science, Milwaukee, WI, USA."},{"key":"ref_17","unstructured":"Watkins, C.J. (1989). Learning from Delayed Rewards. [Ph.D. Thesis, Psychology Department, University of Cambridge]."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Tran-Thanh, L., Chapman, A., de Cote, E.M., Rogers, A., and Jennings, N.R. (2010, January 11\u201315). Epsilon\u2013First Policies for Budget\u2013Limited Multi-Armed Bandits. Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, Atlanta, GE, USA.","DOI":"10.1609\/aaai.v24i1.7758"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tokic, M., and Palm, G. (2011). Value-Difference Based Exploration: Adaptive Control between Epsilon-Greedy and Softmax, Springer.","DOI":"10.1007\/978-3-642-24455-1_33"},{"key":"ref_20","first-page":"100","article-title":"Finite-Time Regret Bounds for the Multiarmed Bandit Problem","volume":"Volume 1998","author":"Fischer","year":"1998","journal-title":"Proceedings of the ICML"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., and Torgo, L. (2005). Multi-armed Bandit Algorithms and Empirical Evaluation. Proceedings of the Machine Learning: ECML 2005, Springer.","DOI":"10.1007\/11564096"},{"key":"ref_22","unstructured":"Luce, R.D. (1959). Individual Choice Behavior, John Wiley."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1137\/S0097539701398375","article-title":"The Nonstochastic Multiarmed Bandit Problem","volume":"32","author":"Auer","year":"2002","journal-title":"SIAM J. Comput."},{"key":"ref_24","unstructured":"Bubeck, S., and Slivkins, A. (2012, January 25\u201327). The Best of Both Worlds: Stochastic and Adversarial Bandits. Proceedings of the Conference on Learning Theory, Edinburgh, Scotland."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1023\/A:1013689704352","article-title":"Finite-time Analysis of the Multiarmed Bandit Problem","volume":"47","author":"Auer","year":"2002","journal-title":"Mach. Lear."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.2307\/1427934","article-title":"Sample mean based index policies by O(log n) regret for the multi-armed bandit problem","volume":"27","author":"Agrawal","year":"1995","journal-title":"Adv. Appl. Probab."},{"key":"ref_27","unstructured":"Bubeck, S. (2010). Bandits Games and Clustering Foundations. [Ph.D. Thesis, Lille 1 University of Science and Technology]."},{"key":"ref_28","unstructured":"Garivier, A., and Capp\u00e9, O. (2011, January 9\u201311). The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond. Proceedings of the 24th Annual Conference on Learning Theory, Budapest, Hungary."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Li, L., Chu, W., Langford, J., and Schapire, R.E. (2010, January 26\u201330). A Contextual-bandit Approach to Personalized News Article Recommendation. Proceedings of the 19th International Conference on World Wide Web, Raleigh, CA, USA.","DOI":"10.1145\/1772690.1772758"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"5588","DOI":"10.1109\/TIT.2012.2198613","article-title":"Online Learning of Rested and Restless Bandits","volume":"58","author":"Tekin","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1902","DOI":"10.1109\/TIT.2012.2230215","article-title":"Learning in a Changing World: Restless Multiarmed Bandit With Unknown Dynamics","volume":"59","author":"Liu","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Bshouty, N.H., Stoltz, G., Vayatis, N., and Zeugmann, T. (2012). Regret Bounds for Restless Markov Bandits. Proceedings of the Algorithmic Learning Theory, Springer.","DOI":"10.1007\/978-3-642-34106-9"},{"key":"ref_33","unstructured":"Bubeck, S., Cohen, M., and Li, Y. (2018, January 7\u20139). Sparsity, Variance and Curvature in Multi-Armed Bandits. Proceedings of the Algorithmic Learning Theory, Lanzarote, Spain."},{"key":"ref_34","unstructured":"Jun, K., Jamieson, K., Nowak, R., and Zhu, X. (2016, January 9\u201311). Top Arm Identification in Multi-Armed Bandits with Batch Arm Pulls. Proceedings of the Artificial Intelligence and Statistics, Cadiz, Spain."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Liu, F., Lee, J., and Shroff, N. (2018, January 2\u20137). A Change-Detection Based Framework for Piecewise-Stationary Multi-Armed Bandit Problem. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11746"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Pang, K., Dong, M., Wu, Y., and Hospedales, T. (2018, January 23\u201326). Dynamic Ensemble Active Learning: A Non-Stationary Bandit with Expert Advice. Proceedings of the 24th International Conference on Pattern Recognition (ICPR), Guangzhou, China.","DOI":"10.1109\/ICPR.2018.8545422"},{"key":"ref_37","unstructured":"Kocsis, L., and Szepesv\u00e1ri, C. (2006, January 26). Discounted Ucb. Proceedings of the 2nd PASCAL Challenges Workshop, Venice, Italy."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Saito, K., Notsu, A., and Honda, K. (2014, January 3\u20136). Discounted UCB1-tuned for Q-learning. Proceedings of the Joint 7th International Conference on Soft Computing and Intelligent Systems (SCIS) and 15th International Symposium on Advanced Intelligent Systems (ISIS), Kitakyushu, Japan.","DOI":"10.1109\/SCIS-ISIS.2014.7044672"},{"key":"ref_39","unstructured":"Kivinen, J., Szepesv\u00e1ri, C., Ukkonen, E., and Zeugmann, T. On Upper-Confidence Bound Policies for Switching Bandit Problems. Proceedings of the Algorithmic Learning Theory."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Oksanen, J., and Koivunen, V. (2017, January 5\u20139). Learning Spectrum Opportunities in Non-Stationary Radio Environments. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7952596"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Alaya-Feki, A.B., Moulines, E., and LeCornec, A. (2008, January 6\u20139). Dynamic spectrum access with non-stationary Multi-Armed Bandit. Proceedings of the IEEE 9th Workshop on Signal Processing Advances in Wireless Communications, Recife, Brazil.","DOI":"10.1109\/SPAWC.2008.4641641"},{"key":"ref_42","unstructured":"Shedden, K. (2013). Binomial Confidence Intervals, Department of Statistics, University of Michigan."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/19\/4090\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:22:50Z","timestamp":1760188970000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/19\/4090"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,21]]},"references-count":42,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2019,10]]}},"alternative-id":["s19194090"],"URL":"https:\/\/doi.org\/10.3390\/s19194090","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,9,21]]}}}