{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,28]],"date-time":"2025-11-28T11:56:24Z","timestamp":1764330984918,"version":"build-2065373602"},"reference-count":26,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2019,5,2]],"date-time":"2019-05-02T00:00:00Z","timestamp":1556755200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Unmanned aerial vehicle (UAV) networks have a wide range of applications, such as in the Internet of Things (IoT), 5G communications, and so forth. However, the communications between UAVs and UAVs to ground control stations mainly use radio channels, and therefore these communications are vulnerable to cyberattacks. With the advent of software-defined radio (SDR), smart attacks that can flexibly select attack strategies according to the defender\u2019s state information are gradually attracting the attention of researchers and potential attackers of UAV networks. The smart attack can even induce the defender to take a specific defense strategy, causing even greater damage. Inspired by symmetrical thinking, a solution using a software-defined network (SDN) to combat software-defined radio was proposed. We propose a network architecture which uses dual controllers, including a UAV flight controller and SDN controller, to achieve collaborative decision-making. Built on the top of the SDN, the state information of the whole network converges quickly and is fitted to an environment model used to develop an improved Dyna-Q-based reinforcement learning algorithm. The improved algorithm integrates the power allocation and track planning of UAVs into a unified action space. The simulation data showed that the proposed communication solution can effectively avoid smart jamming attacks and has faster learning efficiency and higher convergence performance than the compared algorithms.<\/jats:p>","DOI":"10.3390\/sym11050617","type":"journal-article","created":{"date-parts":[[2019,5,7]],"date-time":"2019-05-07T03:15:46Z","timestamp":1557198946000},"page":"617","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["A Dyna-Q-Based Solution for UAV Networks Against Smart Jamming Attacks"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3378-3457","authenticated-orcid":false,"given":"Zhiwei","family":"Li","sequence":"first","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, Hebei, China"},{"name":"Aircraft Maintenance Center, 93413 Units, Yongji 044500, Shanxi, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Lu","sequence":"additional","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, Hebei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yun","family":"Shi","sequence":"additional","affiliation":[{"name":"Meteorological Investigation Center, 66199 Units, Beijing 100000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zengguang","family":"Wang","sequence":"additional","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, Hebei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenxin","family":"Qiao","sequence":"additional","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, Hebei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yicen","family":"Liu","sequence":"additional","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, Hebei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,5,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zhang, X., Wang, H.J., and Zhao, H.T. (2018, January 15\u201319). An SDN Framework for UAV Backbone Network towards Knowledge Centric Networking. Proceedings of the IEEE INFOCOM 2018-IEEE Conference on Computer Communications Workshops, Honolulu, HI, USA.","DOI":"10.1109\/INFCOMW.2018.8406959"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"White, K.J.S., Denney, E., Knudson, M.D., Mamerides, A.K., and Pezaros, D.P. (2017, January 8\u201311). A Programmable SDN Plus NFV-Based Architecture for UAV Telemetry Monitoring. Proceedings of the 2017 14th IEEE Annual Consumer Communications & Networking Conference, Las Vegas, NV, USA.","DOI":"10.1109\/CCNC.2017.7983162"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.adhoc.2018.08.023","article-title":"Software-defined unmanned aerial vehicles networking for video dissemination services","volume":"83","author":"Zhao","year":"2019","journal-title":"Ad Hoc Netw."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2255","DOI":"10.1007\/s11277-018-5815-1","article-title":"Efficient Data Processing in Software-Defined UAV-Assisted Vehicular Networks: A Sequential Game Approach","volume":"101","author":"Alioua","year":"2018","journal-title":"Wirel. Pers. Commun."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Barritt, B., Kichkaylo, T., Mandke, K., Zalcman, A., and Lin, V. (2017, January 4\u201311). Operating a UAV Mesh & Internet Backhaul Network Using Temporospatial SDN. Proceedings of the 2017 IEEE Aerospace Conference, Big Sky, MT, USA.","DOI":"10.1109\/AERO.2017.7943701"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kirichek, R., Vladyko, A., and Paramonov, A. (2017, January 19\u201322). Software-Defined Architecture for Flying Ubiquitous Sensor Networking. Proceedings of the 2017 19th International Conference on Advanced Communication Technology (ICACT), Bongpyeong, Korea.","DOI":"10.23919\/ICACT.2017.7890076"},{"key":"ref_7","unstructured":"Rahman, S.U., Kim, G.-H., Cho, Y.-Z., and Khan, A. (2017, January 18\u201320). Deployment of an SDN-based UAV Network: Controller Placement and Tradeoff Between Control Overhead and Delay. Proceedings of the 2017 International Conference on Information and Communication Technology Convergence, Jeju, Korea."},{"key":"ref_8","unstructured":"Zhou, Y., and Kunz, T. (2017). Intelligent Wireless Ad Hoc Routing Protocol and Controller for UAV Networks. Ad Hoc Networks, Proceedings of the 8th International Conference, ADHOCNETS 2016, Ottawa, ON, Canada, 26\u201327 September 2016, Springer."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Toufga, S., Abdellatif, S., Owezarski, P., and Villemur, T. (2018, January 5\u20137). OpenFlow Based Topology Discovery Service in Software Defined Vehicular Networks: Limitations and Future Approaches. Proceedings of the 2018 IEEE Vehicular Networking Conference (VNC), 2018; IEEE: Taipei, Taiwan, China.","DOI":"10.1109\/VNC.2018.8628349"},{"key":"ref_10","first-page":"31","article-title":"UAV positioning for throughput maximization","volume":"1","author":"Cho","year":"2018","journal-title":"EURASIP J. Wirel. Commun. Netw."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kim, A., Wampler, B., and Goppert, J. (2013, January 19\u201420). Cyber Attack Vulnerabilities Analysis for Unmanned Aerial Vehicles. Proceedings of the Infotech@ Aerospace, Hilton London Olympia, London, UK.","DOI":"10.2514\/6.2012-2438"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1066","DOI":"10.1109\/COMST.2014.2365951","article-title":"On the Security of the Automatic Dependent Surveillance-Broadcast Protocol","volume":"17","author":"Strohmeier","year":"2015","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wesson, K.D., Evans, B.L., and Humphreys, T.E. (2013, January 3\u20135). A combined symmetric difference and power monitoring GNSS anti-spoofing technique. Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Austin, TX, USA.","DOI":"10.1109\/GlobalSIP.2013.6736854"},{"key":"ref_14","unstructured":"Shepard, D.P., Bhatti, J.A., Humphreys, T.E., and Fansler, A.A. (2012, January 19\u201321). Evaluation of Smart Grid and Civilian UAV Vulnerability to GPS Spoofing Attacks. Proceedings of the Radionavigation Laboratory Conference Proceedings, Nashville, TN, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.ijcip.2017.10.002","article-title":"Analysis of vulnerabilities, attacks, countermeasures and overall risk of the Automatic Dependent Surveillance-Broadcast (ADS-B) system","volume":"19","author":"Manesh","year":"2017","journal-title":"Int. J. Crit. Infrastruct. Prot."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Brust, M.R., Danoy, G., Bouvry, P., Gashi, D., Pathak, H., and Gon\u00e7alves, M.P. (2017, January 9). Defending Against Intrusion of Malicious Uavs with Networked Uav Defense Swarms. Proceedings of the 2017 IEEE 42nd Conference on Local Computer Networks Workshops (LCN Workshops), Singapore.","DOI":"10.1109\/LCN.Workshops.2017.71"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2015","DOI":"10.1109\/JSAC.2018.2864376","article-title":"Deployment algorithms for uav airborne networks toward on-demand coverage","volume":"36","author":"Zhao","year":"2018","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/JSAC.2012.120102","article-title":"Anti-jamming games in multi-channel cognitive radio networks","volume":"30","author":"Wu","year":"2012","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3237","DOI":"10.1007\/s11227-015-1420-1","article-title":"Power control with reinforcement learning in cooperative cognitive radio networks against jamming","volume":"71","author":"Xiao","year":"2015","journal-title":"J. Supercomput."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tang, X., Ren, P., Wang, Y., Du, Q.H., and Sun, L. (2015, January 6\u201310). Securing Wireless Transmission Against Reactive Jamming: A Stackelberg Game Framework. Proceedings of the 2015 IEEE Global Communications Conference (GLOBECOM), San Diego, CA, USA.","DOI":"10.1109\/GLOCOM.2015.7417368"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3420","DOI":"10.1109\/TVT.2017.2785414","article-title":"User-Centric View of Unmanned Aerial Vehicle Transmission Against Smart Attacks","volume":"67","author":"Xiao","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"El-Bardan, R., Sharma, V., and Varshney, P.K. (2016, January 7\u20139). Learning Equilibria for Power Allocation Games in Cognitive Radio Networks with a Jammer. Proceedings of the 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Washington, DC, USA.","DOI":"10.1109\/GlobalSIP.2016.7906013"},{"key":"ref_23","unstructured":"Wang, H., Zhao, H., Zhang, J., Ma, D., Li, J., and Wei, J. (2018). Survey on Unmanned Aerial Vehicle Networks: A Cyber Physical System Perspective. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Garnaev, A., and Trappe, W. (2013, January 9\u201313). The Eavesdropping and Jamming Dilemma in Multi-Channel Communications. Proceedings of the 2013 IEEE International Conference on Communications (ICC), Budapest, Hungary.","DOI":"10.1109\/ICC.2013.6654847"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"9499","DOI":"10.1109\/TVT.2018.2856854","article-title":"Two-dimensional antijamming mobile communication based on reinforcement learning","volume":"67","author":"Xiao","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_26","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/11\/5\/617\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:48:50Z","timestamp":1760186930000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/11\/5\/617"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,2]]},"references-count":26,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2019,5]]}},"alternative-id":["sym11050617"],"URL":"https:\/\/doi.org\/10.3390\/sym11050617","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2019,5,2]]}}}