{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T06:30:07Z","timestamp":1775716207478,"version":"3.50.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,5,24]],"date-time":"2019-05-24T00:00:00Z","timestamp":1558656000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,5,24]],"date-time":"2019-05-24T00:00:00Z","timestamp":1558656000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005073","name":"Agency for Defense Development","doi-asserted-by":"publisher","award":["UC160007ED"],"award-info":[{"award-number":["UC160007ED"]}],"id":[{"id":"10.13039\/501100005073","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Wireless Com Network"],"published-print":{"date-parts":[[2019,12]]},"DOI":"10.1186\/s13638-019-1433-1","type":"journal-article","created":{"date-parts":[[2019,5,24]],"date-time":"2019-05-24T12:03:22Z","timestamp":1558699402000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Reinforcement learning-based dynamic band and channel selection in cognitive radio ad-hoc networks"],"prefix":"10.1186","volume":"2019","author":[{"given":"Sung-Jeen","family":"Jang","sequence":"first","affiliation":[]},{"given":"Chul-Hee","family":"Han","sequence":"additional","affiliation":[]},{"given":"Kwang-Eog","family":"Lee","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1533-0814","authenticated-orcid":false,"given":"Sang-Jo","family":"Yoo","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,5,24]]},"reference":[{"key":"1433_CR1","volume-title":"Effective Spectrum Pricing: Supporting Better Quality and more Affordable Mobile Services","author":"R Marsden","year":"2017","unstructured":"R. Marsden, B. Soria, H.M. Ihle, Effective Spectrum Pricing: Supporting Better Quality and more Affordable Mobile Services (GSMA, London, UK, 2017) Tech. Rep"},{"key":"1433_CR2","unstructured":"FCC, ET Docket No 03-222, Notice of Proposed Rule Making and Order. (2003)"},{"key":"1433_CR3","unstructured":"FCC, ET Docket No 03-237, Notice of Proposed Rule Making and Order. (2003)"},{"key":"1433_CR4","volume-title":"Cognitive radio: An Integrated Agent Architecture for Software Defined radio","author":"J Miltola","year":"2000","unstructured":"J. Miltola, Cognitive radio: An Integrated Agent Architecture for Software Defined radio (Doctor of Technology, Royal Inst. Technol. (KTH), Stockholm, 2000)"},{"issue":"13","key":"1433_CR5","doi-asserted-by":"publisher","first-page":"2127","DOI":"10.1016\/j.comnet.2006.05.001","volume":"50","author":"IF Akyildiz","year":"2006","unstructured":"I.F. Akyildiz et al., Next generation\/dynamic spectrum access\/cognitive radio wireless networks: a survey. Comp. Networks J. 50(13), 2127\u20132159 (2006)","journal-title":"Comp. Networks J."},{"key":"1433_CR6","unstructured":"Unlicensed operation in the TV broadcast bands and additional spectrum for unlicensed devices below 900 MHz and in the 3 GHz band, Third Memorandum Opinion and Order, (FCC 2012), \n                    http:\/\/transition.fcc.gov\/Daily_Releases\/Daily_Business\/2012\/db0405\/FCC-12-36A1.pdf\n                    \n                  . Accessed 25 Jun 2018"},{"key":"1433_CR7","unstructured":"Report ITU-R M.2330\u20130 (11\/2014) ITU-R M.2330\u20130 Cognitive radio Systems in the Land Mobile Service"},{"key":"1433_CR8","unstructured":"DARPA Developing Cognitive radio IC Technology for Next-Gen Communications, Radar, and Electronic Warfare, (IDST 2018) \n                    www.darpa.mil\/program\/computational-leverage-against-surveillance-systems\n                    \n                  . Accessed 25 Jun 2018"},{"key":"1433_CR9","volume-title":"DARPA-BAA-11-61, Computational Leverage against Surveillance Systems (CLASS)","author":"DARPA","year":"2011","unstructured":"DARPA, DARPA-BAA-11-61, Computational Leverage against Surveillance Systems (CLASS), (2011)"},{"key":"1433_CR10","unstructured":"The Spectrum Collaboration Challenge. (DARPA, 2018), \n                    https:\/\/spectrumcollaborationchallenge.com\n                    \n                  . Accessed 25 Jun 2018"},{"issue":"5","key":"1433_CR11","doi-asserted-by":"publisher","first-page":"545","DOI":"10.1109\/LWC.2015.2455035","volume":"4","author":"M Vishram","year":"2015","unstructured":"M. Vishram, L.C. Tong, C. Syin, A channel allocation based self-coexistence scheme for homogeneous ad-hoc networks. IEEE Wireless Commun. Lett. 4(5), 545\u2013548 (2015)","journal-title":"IEEE Wireless Commun. Lett."},{"issue":"4","key":"1433_CR12","doi-asserted-by":"publisher","first-page":"2481","DOI":"10.1109\/TVT.2015.2423691","volume":"65","author":"S Maghsudi","year":"2016","unstructured":"S. Maghsudi, S. Sta\u0144czak, Hybrid centralized\u2013distributed resource allocation for device-to-device communication underlaying cellular networks. IEEE Trans. Veh. Technol. 65(4), 2481\u20132495 (2016)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"2","key":"1433_CR13","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1109\/TWC.2016.2629484","volume":"16","author":"Y Han","year":"2017","unstructured":"Y. Han, E. Ekici, H. Kremo, O. Altintas, Throughput-efficient channel allocation algorithms in multi-channel cognitive vehicular networks. IEEE Trans. Wirel. Commun. 16(2), 757\u2013770 (2017)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"9","key":"1433_CR14","doi-asserted-by":"publisher","first-page":"7925","DOI":"10.1109\/TVT.2017.2676462","volume":"66","author":"M Li","year":"2017","unstructured":"M. Li, L. Zhao, H. Liang, An SMDP-based prioritized channel allocation scheme in cognitive enabled vehicular ad hoc networks. IEEE Trans. Veh. Technol. 66(9), 7925\u20137933 (2017)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"7","key":"1433_CR15","doi-asserted-by":"publisher","first-page":"4802","DOI":"10.1109\/TWC.2017.2703109","volume":"16","author":"O Semiari","year":"2017","unstructured":"O. Semiari, W. Saad, M. Bennis, Joint millimeter wave and microwave resources allocation in cellular networks with dual-mode base stations. IEEE Trans. Wirel. Commun. 16(7), 4802\u20134816 (2017)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"4","key":"1433_CR16","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1109\/LWC.2017.2702747","volume":"6","author":"L Liang","year":"2017","unstructured":"L. Liang, J. Kim, S.C. Jha, K. Sivanesan, G.Y. Li, Spectrum and power allocation for vehicular communications with delayed CSI feedback. IEEE Wireless Commun. Lett. 6(4), 458\u2013461 (2017)","journal-title":"IEEE Wireless Commun. Lett."},{"key":"1433_CR17","doi-asserted-by":"crossref","unstructured":"P. Simon, Too Big to Ignore: The Business Case for Big Data (Wiley, (2013)","DOI":"10.1002\/9781119204039"},{"issue":"7553","key":"1433_CR18","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1038\/nature14540","volume":"521","author":"ML Littman","year":"2015","unstructured":"M.L. Littman, Reinforcement learning improves behavior from evaluative feedback. Nature 521(7553), 445\u2013451 (2015)","journal-title":"Nature"},{"issue":"9","key":"1433_CR19","doi-asserted-by":"publisher","first-page":"3996","DOI":"10.1109\/TCOMM.2016.2593468","volume":"64","author":"A Asheralieva","year":"2016","unstructured":"A. Asheralieva, Y. Miyanaga, An autonomous learning-based algorithm for Joint Channel and power level selection by D2D pairs in heterogeneous cellular networks. IEEE Trans. Commun. 64(9), 3996\u20134012 (2016)","journal-title":"IEEE Trans. Commun."},{"issue":"11","key":"1433_CR20","doi-asserted-by":"publisher","first-page":"2887","DOI":"10.1109\/JSAC.2016.2614952","volume":"34","author":"M Srinivasan","year":"2016","unstructured":"M. Srinivasan, V.J. Kotagi, C.S.R. Murthy, A Q-learning framework for user QoE enhanced self-organizing spectrally efficient network using a novel inter-operator proximal spectrum sharing. IEEE J. Sel. Areas Commun. 34(11), 2887\u20132901 (2016)","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"1","key":"1433_CR21","first-page":"2161","volume":"29","author":"K Salma","year":"2017","unstructured":"K. Salma, S. Reza, G.S. Ali, Learning-based resource allocation in D2D communications with QoS and fairness considerations. Eur. Trans. Telecommun. 29(1), 2161\u20133915 (2017)","journal-title":"Eur. Trans. Telecommun."},{"issue":"15","key":"1433_CR22","doi-asserted-by":"publisher","first-page":"2380","DOI":"10.1049\/iet-com.2017.0213","volume":"11","author":"E Fakhfakh","year":"2017","unstructured":"E. Fakhfakh, S. Hamouda, Optimised Q-learning for WiFi offloading in dense cellular networks. IET Commun. 11(15), 2380\u20132385 (2017)","journal-title":"IET Commun."},{"issue":"5","key":"1433_CR23","doi-asserted-by":"publisher","first-page":"4539","DOI":"10.1109\/TVT.2018.2793186","volume":"67","author":"M Yan","year":"2018","unstructured":"M. Yan, G. Feng, J. Zhou, S. Qin, Smart multi-RAT access based on multiagent reinforcement learning. IEEE Trans. Veh. Technol. 67(5), 4539\u20134551 (2018)","journal-title":"IEEE Trans. Veh. Technol."},{"key":"1433_CR24","doi-asserted-by":"publisher","first-page":"27278","DOI":"10.1109\/ACCESS.2018.2829492","volume":"6","author":"V Maglogiannis","year":"2018","unstructured":"V. Maglogiannis, D. Naudts, A. Shahid, I. Moerman, A Q-learning scheme for fair coexistence between LTE and Wi-Fi in unlicensed spectrum. IEEE Access 6, 27278\u201327293 (2018)","journal-title":"IEEE Access"},{"issue":"4","key":"1433_CR25","doi-asserted-by":"publisher","first-page":"856","DOI":"10.1049\/cje.2017.06.015","volume":"26","author":"N Xu","year":"2017","unstructured":"N. Xu, H. Zhang, F. Xu, Z. Wang, Q-learning based interference-aware channel handoff for partially observable cognitive radio ad hoc networks. Chin. J. Electron. 26(4), 856\u2013863 (2017)","journal-title":"Chin. J. Electron."},{"key":"1433_CR26","doi-asserted-by":"crossref","unstructured":"S.J. Jang, S.J. Yoo, Reinforcement learning for dynamic sensing parameter control in cognitive radio systems (IEEE ICTC, 2017), pp. 471\u2013474","DOI":"10.1109\/ICTC.2017.8191022"},{"key":"1433_CR27","doi-asserted-by":"crossref","unstructured":"Y.Y. Liu, S.J. Yoo, Dynamic resource allocation using reinforcement learning for LTE-U and WiFi in the unlicensed spectrum (IEEE ICUFN, 2017), pp. 471\u2013475","DOI":"10.1109\/ICUFN.2017.7993829"},{"issue":"4","key":"1433_CR28","doi-asserted-by":"publisher","first-page":"883","DOI":"10.1007\/s11277-016-3935-z","volume":"93","author":"L Shi","year":"2017","unstructured":"L. Shi, S.J. Yoo, Distributed fair resource allocation for cognitive femtocell networks. Wireless Personal Commun. 93(4), 883\u2013902 (2017)","journal-title":"Wireless Personal Commun."},{"key":"1433_CR29","unstructured":"Fairness measure, (Wikipedia 2018) \n                    https:\/\/en.wikipedia.org\/wiki\/Fairness_measure\n                    \n                  . Accessed 25 Jun 2018"},{"key":"1433_CR30","first-page":"1","volume-title":"User Pairing and Channel Allocation for Full-Duplex Self-Organizing Small Cell Networks","author":"Q Zhang","year":"2017","unstructured":"Q. Zhang, Q. Du, K. Zhu, User Pairing and Channel Allocation for Full-Duplex Self-Organizing Small Cell Networks (WCSP, Nanjing, 2017), pp. 1\u20136"},{"key":"1433_CR31","doi-asserted-by":"crossref","unstructured":"M. Yan, G. Feng and S. Qin, Multi-RAT access based on multi-agent reinforcement learning. IEEE GLOBECOM, 1\u20136 (2017)","DOI":"10.1109\/GLOCOM.2017.8254980"},{"key":"1433_CR32","doi-asserted-by":"crossref","unstructured":"F. Zeng, H. Liu and J. Xu, Sequential channel selection for decentralized cognitive radio sensor network based on modified Q-learning algorithm. ICNC-FSKD, 657\u2013662 (2016)","DOI":"10.1109\/FSKD.2016.7603252"}],"container-title":["EURASIP Journal on Wireless Communications and Networking"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13638-019-1433-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s13638-019-1433-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13638-019-1433-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,5,22]],"date-time":"2020-05-22T23:09:04Z","timestamp":1590188944000},"score":1,"resource":{"primary":{"URL":"https:\/\/jwcn-eurasipjournals.springeropen.com\/articles\/10.1186\/s13638-019-1433-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,24]]},"references-count":32,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["1433"],"URL":"https:\/\/doi.org\/10.1186\/s13638-019-1433-1","relation":{},"ISSN":["1687-1499"],"issn-type":[{"value":"1687-1499","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,24]]},"assertion":[{"value":"18 April 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 April 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 May 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"- Prof. Sang-Jo Yoo, PhD (Corresponding author): Sang-Jo Yoo received the B.S. degree in electronic communication engineering from Hanyang University, Seoul, South Korea, in 1988, and the M.S. and Ph.D. degrees in electrical engineering from the Korea Advanced Institute of Science and Technology, in 1990 and 2000, respectively. From 1990 to 2001, he was a Member of the Technical Staff in the Korea Telecom Research and Development Group, where he was involved in communication protocol conformance testing and network design fields. From 1994 to 1995 and from 2007 to 2008, he was a Guest Researcher with the National Institute Standards and Technology, USA. Since 2001, he has been with Inha University, where he is currently a Professor with the Information and Communication Engineering Department. His current research interests include cognitive radio network protocols, ad-hoc wireless network, MAC and routing protocol design, wireless network QoS, and wireless sensor networks.- Mr. Sung-Jeen Jang: Sung-Jeen Jang received a B.S degree in electrical engineering from Inha University Incheon, Korea, 2007. He received his M.S. degree in Graduate School of Information Technology and Telecommunication, Inha University, Incheon Korea, 2009. Since March 2009, he has been pursuing a Ph.D. degree at the Graduate School of Information Technology and Telecommunication, Inha University, Incheon Korea. His current research interests include cognitive radio network protocols and machine learning applied wireless communications.- Dr. Chul-Hee Han: Chulhee Han received the B.S. degree in Electronic Engineering from Chung-ang University, Korea, in 1997, and M.S. and Ph.D. degrees in Electrical and Electronic Engineering from Yonsei University, Korea, in 1999 and 2007, respectively. Currently, he is working at Hanwha Systems, Korea, as a chief engineer. He was involved in various projects including tactical mobile WiMAX system and tactical LOS PMP radio. His research interests include Tactical Broadband Communications, Combat Network Radio, and Cognitive Radio for Military Applications.- Dr. Kwang-Eog Lee: Kwang-Eog Lee received the B.S. and M.S. degrees in electronic engineering from Kyungpook National University, Daegu, South Korea, in 1988 and 1990, respectively. He has been working in Agency for Defense Development since 1990. From 2007 to 2008, he was an exchange scientist in CERDEC (Communications-Electronics Research, Development and Engineering Center) U.S. Army. Currently, he is a principal researcher and his research interests include cognitive radio and terrestrial and satellite tactical communication.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Authors\u2019 information"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Publisher\u2019s Note"}}],"article-number":"131"}}