{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T22:07:07Z","timestamp":1778969227158,"version":"3.51.4"},"reference-count":37,"publisher":"National Library of Serbia","issue":"1","license":[{"start":{"date-parts":[[2026,1,1]],"date-time":"2026-01-01T00:00:00Z","timestamp":1767225600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ComSIS","COMPUT SCI INF SYST","COMPUT SCI INFORM SY","COMPUTER SCI INFORM","COMSIS J"],"published-print":{"date-parts":[[2026]]},"abstract":"<jats:p>With the rapid development of modern power systems, traditional scheduling and reinforcement learning methods often fail to meet stringent Quality of Service (QoS) demands for low latency, high reliability, and stable bandwidth under large-scale bursty traffic. To address this problem, we propose a QoS-driven routing optimization approach based on Adversarial Reinforcement Learning, referred to as Adversarial Critic-Cooperative Actor (ACCA). By introducing adversarial agents that model worst-case perturbations, ACCA establishes a multi-agent game framework that enhances policy robustness and adaptability in dynamic network environments. Furthermore, a multi-dimensional state representation and a QoS-aware cost function are designed to capture metrics such as delay, bandwidth utilization, queue length, and packet loss. Experiments demonstrate that ACCA outperforms traditional routing protocols and standard reinforcement learning algorithms in terms of end-to-end delay, load balancing, and throughput, thereby providing an effective solution for QoS assurance in intelligent power communication networks.<\/jats:p>","DOI":"10.2298\/csis251015009h","type":"journal-article","created":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T10:44:04Z","timestamp":1768905844000},"page":"299-320","source":"Crossref","is-referenced-by-count":0,"title":["Robust QoS-aware network scheduling for smart substations via multi-agent adversarial reinforcement learning"],"prefix":"10.2298","volume":"23","author":[{"given":"Ping","family":"He","sequence":"first","affiliation":[{"name":"State Grid Suzhou Power Supply Company Suzhou Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongsheng","family":"Jing","sequence":"additional","affiliation":[{"name":"State Grid Suzhou Power Supply Company Suzhou Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Baozhen","family":"Qi","sequence":"additional","affiliation":[{"name":"State Grid Suzhou Power Supply Company Suzhou Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Yang","sequence":"additional","affiliation":[{"name":"State Grid Suzhou Power Supply Company Suzhou Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingsong","family":"Xue","sequence":"additional","affiliation":[{"name":"State Grid Suzhou Power Supply Company Suzhou Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1078","reference":[{"key":"ref1","doi-asserted-by":"crossref","unstructured":"Juan C Lozano, Keerthi Koneru, Neil Ortiz, and Alvaro A Cardenas. Digital substations and iec 61850: A primer. IEEE Communications Magazine, 61(6):28-34, 2023.","DOI":"10.1109\/MCOM.001.2200568"},{"key":"ref2","doi-asserted-by":"crossref","unstructured":"Jos\u00e9 Gaspar, Tiago Cruz, Chan-Tong Lam, and Paulo Sim\u00f5es. Smart substation communications and cybersecurity: A comprehensive survey. IEEE communications surveys & tutorials, 25(4):2456-2493, 2023.","DOI":"10.1109\/COMST.2023.3305468"},{"key":"ref3","doi-asserted-by":"crossref","unstructured":"Tze-Yang Tung and Deniz G\u00fcnd\u00fcz. Deepwive: Deep-learning-aided wireless video transmission. IEEE Journal on Selected Areas in Communications, 40(9):2570-2583, 2022.","DOI":"10.1109\/JSAC.2022.3191354"},{"key":"ref4","doi-asserted-by":"crossref","unstructured":"S.C. Jayasinghe, M. Mahmoodian, A. Sidiq, T.M. Nanayakkara, A. Alavi, Sam Mazaheri, F. Shahrivar, Z. Sun, and S. Setunge. Innovative digital twin with artificial neural networks for real-time monitoring of structural response: A port structure case study. Ocean Engineering, 312:119187, 2024.","DOI":"10.1016\/j.oceaneng.2024.119187"},{"key":"ref5","doi-asserted-by":"crossref","unstructured":"Wenqing Zheng, Hao Frank Yang, Jiarui Cai, Peihao Wang, Xuan Jiang, Simon Shaolei Du, YinhaiWang, and ZhangyangWang. Integrating the traffic science with representation learning for city-wide network congestion prediction. Information Fusion, 99:101837, 2023.","DOI":"10.1016\/j.inffus.2023.101837"},{"key":"ref6","doi-asserted-by":"crossref","unstructured":"Xiang Cheng, Dongliang Duan, Shijian Gao, and Liuqing Yang. Integrated sensing and communications (isac) for vehicular communication networks (vcn). IEEE Internet of Things Journal, 9(23):23441-23451, 2022.","DOI":"10.1109\/JIOT.2022.3191386"},{"key":"ref7","doi-asserted-by":"crossref","unstructured":"Lipeng Zhu, Wenyan Ma, and Rui Zhang. Movable antennas for wireless communication: Opportunities and challenges. IEEE Communications Magazine, 62(6):114-120, 2023.","DOI":"10.1109\/MCOM.001.2300212"},{"key":"ref8","doi-asserted-by":"crossref","unstructured":"G. BalaMurugan, N. Palanivel, and A. John. Application reliable traffic control method for efficient data management in wireless-aided computer applications. In 2023 International Conference on System, Computation, Automation and Networking (ICSCAN), pages 1-5, 2023.","DOI":"10.1109\/ICSCAN58655.2023.10394734"},{"key":"ref9","doi-asserted-by":"crossref","unstructured":"Quan Chen, Lei Yang, Yong Zhao, YiWang, Haibo Zhou, and Xiaoqian Chen. Shortest path in leo satellite constellation networks: An explicit analytic approach. IEEE Journal on Selected Areas in Communications, 42(5):1175-1187, 2024.","DOI":"10.1109\/JSAC.2024.3365873"},{"key":"ref10","doi-asserted-by":"crossref","unstructured":"Wenbin Tian, Chaojie Gu, Miao Guo, Shibo He, Jiawen Kang, Dusit Niyato, and Jiming Chen. Large-scale deterministic networks: Architecture, enabling technologies, case study, and future directions. IEEE Network, 38(4):284-291, 2024.","DOI":"10.1109\/MNET.2024.3355116"},{"key":"ref11","doi-asserted-by":"crossref","unstructured":"Sijin Yang, Lei Zhuang, Jianhui Zhang, Julong Lan, and Bingkui Li. A multipolicy deep reinforcement learning approach for multiobjective joint routing and scheduling in deterministic networks. IEEE Internet of Things Journal, 11(10):17402-17418, 2024.","DOI":"10.1109\/JIOT.2024.3358403"},{"key":"ref12","doi-asserted-by":"crossref","unstructured":"Ying Ma, Haijie Lou, Ming Yan, Fanghui Sun, and Guoqi Li. Spatio-temporal fusion graph convolutional network for traffic flow forecasting. Information Fusion, 104:102196, 2024.","DOI":"10.1016\/j.inffus.2023.102196"},{"key":"ref13","doi-asserted-by":"crossref","unstructured":"Shreyas Chaudhari, Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan, Ameet Deshpande, and Bruno Castro da Silva. Rlhf deciphered: A critical analysis of reinforcement learning from human feedback for llms. ACM Comput. Surv., 58(2):53:1-53:37, September 2025.","DOI":"10.1145\/3743127"},{"key":"ref14","doi-asserted-by":"crossref","unstructured":"Maxwell Standen, Junae Kim, and Claudia Szabo. Adversarial machine learning attacks and defences in multi-agent reinforcement learning. ACM Computing Surveys, 57(5):1-35, 2025.","DOI":"10.1145\/3708320"},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"Rana M. Sohaib, Oluwakayode Onireti, Yusuf Sambo, Rafiq Swash, Shuja Ansari, and Muhammad A. Imran. Intelligent resource management for embb and urllc in 5g and beyond wireless networks. IEEE Access, 11:65205-65221, 2023.","DOI":"10.1109\/ACCESS.2023.3288698"},{"key":"ref16","unstructured":"Chen Wu, Shikai Guo, Hui Li, Chenchen Li, and Rong Chen. Estimating uncertainty in line-level defect prediction via perceptual borderline oversampling. ACM Trans. Softw. Eng. Methodol., June 2025. Just Accepted."},{"key":"ref17","doi-asserted-by":"crossref","unstructured":"Shuhua Deng, Zhangping Yin, and Xieping Gao. Investigating vulnerabilities in openflow discovery protocol: Novel attacks and their defense. IEEE Transactions on Dependable and Secure Computing, 22(6):6376-6388, 2025.","DOI":"10.1109\/TDSC.2025.3584851"},{"key":"ref18","doi-asserted-by":"crossref","unstructured":"M.Sahaya Sheela, R. Suganthi, S. Gopalakrishnan, T. Karthikeyan, K. Jyothi, and K. Ramamoorthy. Secure routing and reliable packets transmission in manet using fast recursive transfer algorithm. Babylonian Journal of Networking, 2024:78-87, 06 2024.","DOI":"10.58496\/BJN\/2024\/009"},{"key":"ref19","doi-asserted-by":"crossref","unstructured":"Muhammad Numan, Akif Zia Khan, Mansoor Asif, Sarmad Majeed Malik, and Kashif Imran. Exploiting the inherent flexibility in transmission network for optimal scheduling, wind power utilization, and network congestion management. IEEE Access, 9:88746-88758, 2021.","DOI":"10.1109\/ACCESS.2021.3090089"},{"key":"ref20","doi-asserted-by":"crossref","unstructured":"Christian Blum. Ant colony optimization: A bibliometric review. Physics of Life Reviews, 51:87-95, 2024.","DOI":"10.1016\/j.plrev.2024.09.014"},{"key":"ref21","doi-asserted-by":"crossref","unstructured":"Yusuf Alper Kaplan. Forecasting of global solar radiation: A statistical approach using simulated annealing algorithm. Engineering Applications of Artificial Intelligence, 136:109034, 2024.","DOI":"10.1016\/j.engappai.2024.109034"},{"key":"ref22","doi-asserted-by":"crossref","unstructured":"Bushra Alhijawi and Arafat Awajan. Genetic algorithms: Theory, genetic operators, solutions, and applications. Evolutionary Intelligence, 17(3):1245-1256, 2024.","DOI":"10.1007\/s12065-023-00822-6"},{"key":"ref23","doi-asserted-by":"crossref","unstructured":"Ayodeji Olalekan Salau and Melesew Mossie Beyene. Software defined networking based network traffic classification using machine learning techniques. Scientific Reports, 14(1):20060, 2024.","DOI":"10.1038\/s41598-024-70983-6"},{"key":"ref24","doi-asserted-by":"crossref","unstructured":"Daniela M Casas-Velasco, Oscar Mauricio Caicedo Rendon, and Nelson LS da Fonseca. Intelligent routing based on reinforcement learning for software-defined networking. IEEE Transactions on Network and Service Management, 18(1):870-881, 2020.","DOI":"10.1109\/TNSM.2020.3036911"},{"key":"ref25","doi-asserted-by":"crossref","unstructured":"Sandra Sendra, Albert Rego, Jaime Lloret, Jose Miguel Jimenez, and Oscar Romero. Including artificial intelligence in a routing protocol using software defined networks. In 2017 IEEE International Conference on Communications Workshops (ICC Workshops), pages 670-674, 2017.","DOI":"10.1109\/ICCW.2017.7962735"},{"key":"ref26","doi-asserted-by":"crossref","unstructured":"Xinqiang Chen, Ruiyang Hu, Kai Luo, Huafeng Wu, Salvatore Antonio Biancardo, Yiwen Zheng, and Jiangfeng Xian. Intelligent ship route planning via an a* search model enhanced double-deep q-network. Ocean Engineering, 327:120956, 2025.","DOI":"10.1016\/j.oceaneng.2025.120956"},{"key":"ref27","doi-asserted-by":"crossref","unstructured":"Juan Moreno-Malo, Juan-Luis Posadas-Yag\u00fce, Juan Carlos Cano, Carlos T Calafate, J Alberto Conejero, and Jose-Luis Poza-Lujan. Improving traffic light systems using deep q-networks. Expert Systems with Applications, 252:124178, 2024.","DOI":"10.1016\/j.eswa.2024.124178"},{"key":"ref28","doi-asserted-by":"crossref","unstructured":"Yi-Ren Chen, Amir Rezapour, Wen-Guey Tzeng, and Shi-Chun Tsai. Rl-routing: An sdn routing algorithm based on deep reinforcement learning. IEEE Transactions on Network Science and Engineering, 7(4):3185-3199, 2020.","DOI":"10.1109\/TNSE.2020.3017751"},{"key":"ref29","doi-asserted-by":"crossref","unstructured":"Hao Zhang, Linfeng Tang, Xinyu Xiang, Xuhui Zuo, and Jiayi Ma. Dispel darkness for better fusion: A controllable visual enhancer based on cross-modal conditional adversarial learning. In 2024 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26477-26486, 2024.","DOI":"10.1109\/CVPR52733.2024.02501"},{"key":"ref30","doi-asserted-by":"crossref","unstructured":"Qiankun Zuo, Huisi Wu, C. L. Philip Chen, et al. Prior-guided adversarial learning with hypergraph for predicting abnormal connections in alzheimer\u2019s disease. IEEE Transactions on Cybernetics, 54(6):3652-3665, 2024.","DOI":"10.1109\/TCYB.2023.3344641"},{"key":"ref31","doi-asserted-by":"crossref","unstructured":"Xuhui Liu, Bohan Zeng, Sicheng Gao, Shanglin Li, Yutang Feng, Hong Li, Boyu Liu, Jianzhuang Liu, and Baochang Zhang. Ladiffgan: Training gans with diffusion supervision in latent spaces. In 2024 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1115-1125, 2024.","DOI":"10.1109\/CVPRW63382.2024.00118"},{"key":"ref32","doi-asserted-by":"crossref","unstructured":"Willone Lim, Kelvin Sheng Chek Yong, Bee Theng Lau, and Colin Choon Lin Tan. Future of generative adversarial networks (gan) for anomaly detection in network security: A review. Computers & Security, 139:103733, 2024.","DOI":"10.1016\/j.cose.2024.103733"},{"key":"ref33","doi-asserted-by":"crossref","unstructured":"Chuyao Wang and Nabil Aouf. Explainable deep adversarial reinforcement learning approach for robust autonomous driving. IEEE Transactions on Intelligent Vehicles, 10(4):2551-2563, 2025.","DOI":"10.1109\/TIV.2024.3379367"},{"key":"ref34","doi-asserted-by":"crossref","unstructured":"Ao Qu, Yihong Tang, and Wei Ma. Adversarial attacks on deep reinforcement learningbased traffic signal control systems with colluding vehicles. ACM Trans. Intell. Syst. Technol., 14(6):113:1-113:22, November 2023.","DOI":"10.1145\/3625236"},{"key":"ref35","doi-asserted-by":"crossref","unstructured":"Juan Chafla Altamirano, Mariem Guitouni, Hassan Hassan, and Khalil Drira. Routing optimization based on drl and generative adversarial networks for sdn environments. In NOMS 2024-2024 IEEE Network Operations and Management Symposium, pages 1-5, 2024.","DOI":"10.1109\/NOMS59830.2024.10575453"},{"key":"ref36","doi-asserted-by":"crossref","unstructured":"Jingbo Li, Xingjun Zhang, Jia Wei, Zeyu Ji, and Zheng Wei. Garlsched: Generative adversarial deep reinforcement learning task scheduling optimization for large-scale high performance computing systems. Future Generation Computer Systems, 135:259-269, 2022.","DOI":"10.1016\/j.future.2022.04.032"},{"key":"ref37","doi-asserted-by":"crossref","unstructured":"Mansour Selseleh Jonban, Luis Romeral, Mousa Marzband, and Abdullah Abusorrah. A reinforcement learning approach using markov decision processes for battery energy storage control within a smart contract framework. Journal of Energy Storage, 86:111342, 2024.","DOI":"10.1016\/j.est.2024.111342"}],"container-title":["Computer Science and Information Systems"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T21:34:33Z","timestamp":1778967273000},"score":1,"resource":{"primary":{"URL":"https:\/\/doiserbia.nb.rs\/Article.aspx?ID=1820-02142600009H"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026]]}},"URL":"https:\/\/doi.org\/10.2298\/csis251015009h","relation":{},"ISSN":["1820-0214","2406-1018"],"issn-type":[{"value":"1820-0214","type":"print"},{"value":"2406-1018","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026]]}}}