{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T06:05:14Z","timestamp":1771308314561,"version":"3.50.1"},"reference-count":43,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T00:00:00Z","timestamp":1771027200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100007782","name":"Tshwane University of Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007782","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Fog computing extends cloud services to the network edge, enabling low-latency processing for Internet of Things (IoT) applications. However, this distributed approach is vulnerable to a wide range of attacks, necessitating advanced intrusion detection systems (IDSs) that operate under resource constraints. This study proposes integrating self-awareness (online learning and concept drift adaptation) into a lightweight RL (reinforcement learning)-based IDS for fog networks and quantitatively comparing it with non-RL static thresholds and bandit-based approaches in real time. Novel self-aware reinforcement learning (RL) models, the Hierarchical Adaptive Thompson Sampling\u2013Reinforcement Learning (HATS-RL) model, and the Federated Hierarchical Adaptive Thompson Sampling\u2013Reinforcement Learning (F-HATS-RL), were proposed for real-time intrusion detection in a fog network. These self-aware RL policies integrated online uncertainty estimation and concept-drift detection to adapt to evolving attacks. The RL models were benchmarked against the static threshold (ST) model and a widely adopted linear bandit (Linear Upper Confidence Bound\/LinUCB). A realistic fog network simulator with heterogeneous nodes and streaming traffic, including multi-type attack bursts and gradual concept drift, was established. The models\u2019 detection performance was compared using metrics including latency, energy consumption, detection accuracy, and the area under the precision\u2013recall curve (AUPR) and the area under the receiver operating characteristic curve (AUROC). Notably, the federated self-aware agent (F-HATS-RL) achieved the best AUROC (0.933) and AUPR (0.857), with a latency of 0.27 ms and the lowest energy consumption of 0.0137 mJ, indicating its ability to detect intrusions in fog networks with minimal energy. The findings suggest that self-aware RL agents can detect traffic\u2013dynamic attack methods and adapt accordingly, resulting in more stable long-term performance. By contrast, a static model\u2019s accuracy degrades under drift.<\/jats:p>","DOI":"10.3390\/fi18020100","type":"journal-article","created":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T08:38:39Z","timestamp":1771231119000},"page":"100","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Comparative Analysis of Self-Aware Reinforcement Learning Models for Real-Time Intrusion Detection in Fog Networks"],"prefix":"10.3390","volume":"18","author":[{"given":"Nyashadzashe","family":"Tamuka","sequence":"first","affiliation":[{"name":"Department of Information Technology, Tshwane University of Technology, Soshanguve, Pretoria 0152, South Africa"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0439-7186","authenticated-orcid":false,"given":"Topside Ehleketani","family":"Mathonsi","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Tshwane University of Technology, Soshanguve, Pretoria 0152, South Africa"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3013-474X","authenticated-orcid":false,"given":"Thomas Otieno","family":"Olwal","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Tshwane University of Technology, Soshanguve, Pretoria 0152, South Africa"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Solly","family":"Maswikaneng","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Tshwane University of Technology, Soshanguve, Pretoria 0152, South Africa"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tonderai","family":"Muchenje","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Tshwane University of Technology, Soshanguve, Pretoria 0152, South Africa"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4354-5222","authenticated-orcid":false,"given":"Tshimangadzo Mavin","family":"Tshilongamulenzhe","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Tshwane University of Technology, Soshanguve, Pretoria 0152, South Africa"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1186\/s13677-023-00420-y","article-title":"Enhancement of an IoT hybrid intrusion detection system based on fog-to-cloud computing","volume":"12","author":"Mohamed","year":"2023","journal-title":"J. Cloud Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1186\/s13638-024-02348-6","article-title":"An adversarial-environment reinforcement-learning-driven intrusion-detection algorithm for the Internet of Things","volume":"2024","author":"Mahjoub","year":"2024","journal-title":"EURASIP J. Wirel. Commun. Netw."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Saeed, I.A., Selamat, A., Rohani, F., and Krejcar, O. (2025). Performance Evaluation of Reinforcement Learning-Based Intrusion Detection Systems. IEEE Access.","DOI":"10.1109\/ACCESS.2025.3628626"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rehman, T., Tariq, N., Khan, F.A., and Rehman, S.U. (2024). FFL-IDS: A FOG-Enabled Federated Learning-Based Intrusion Detection System to counter jamming and spoofing attacks for the industrial Internet of Things. Sensors, 25.","DOI":"10.3390\/s25010010"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.cose.2009.06.008","article-title":"A survey of coordinated attacks and collaborative intrusion detection","volume":"29","author":"Zhou","year":"2010","journal-title":"Comput. Secur."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Tawfik, M. (2024). Optimized intrusion detection in IoT and fog computing using ensemble learning and advanced feature selection. PLoS ONE, 19.","DOI":"10.1371\/journal.pone.0304082"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"13340","DOI":"10.1109\/JIOT.2023.3337755","article-title":"Heterogeneous data-aware federated learning for intrusion detection systems via meta-sampling in artificial intelligence of things","volume":"11","author":"Han","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Fedorchenko, E., Novikova, E., and Shulepov, A. (2022). Comparative review of the intrusion detection systems based on federated learning: Advantages and open challenges. Algorithms, 15.","DOI":"10.3390\/a15070247"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Chang, V., Golightly, L., Modesti, P., Xu, Q.A., Doan, L.M.T., Hall, K., Boddu, S., and Kobusi\u0144ska, A. (2022). A survey on intrusion detection systems for fog and cloud computing. Future Internet, 14.","DOI":"10.3390\/fi14030089"},{"key":"ref_10","first-page":"182","article-title":"A survey on data-driven network intrusion detection","volume":"54","author":"Chou","year":"2021","journal-title":"ACM Comput. Surv. CSUR"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s10922-021-09589-6","article-title":"As-ids: Anomaly and signature-based IDs for the Internet of Things","volume":"29","author":"Otoum","year":"2021","journal-title":"J. Netw. Syst. Manag."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Banaamah, A.M., and Ahmad, I. (2022). Intrusion detection in IoT using deep learning. Sensors, 22.","DOI":"10.3390\/s22218417"},{"key":"ref_13","first-page":"1313","article-title":"Adaptive user-managed service placement for mobile edge computing via contextual multi-armed bandit learning","volume":"22","author":"Ouyang","year":"2021","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1832","DOI":"10.1109\/TNSM.2020.3037486","article-title":"Adaptive and reinforcement learning approaches for online network monitoring and analysis","volume":"18","author":"Wassermann","year":"2020","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1007\/s10586-024-04711-0","article-title":"Securing fog-assisted IoT smart homes: A federated learning-based intrusion detection approach","volume":"28","author":"Bensaid","year":"2025","journal-title":"Clust. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Alghayadh, F., and Debnath, D. (August, January 31). A hybrid intrusion detection system for smart home security. Proceedings of the 2020 IEEE International Conference on Electro Information Technology (EIT), Chicago, IL, USA.","DOI":"10.1109\/EIT48999.2020.9208296"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"167059","DOI":"10.1109\/ACCESS.2020.3022855","article-title":"Intrusion detection based on an autoencoder and an isolation forest in fog computing","volume":"8","author":"Sadaf","year":"2020","journal-title":"IEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Beshah, Y.K., Abebe, S.L., and Melaku, H.M. (2024). Drift adaptive online DDoS attack detection framework for IoT system. Electronics, 13.","DOI":"10.3390\/electronics13061004"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"8491","DOI":"10.1109\/JIOT.2022.3196942","article-title":"An efficient hybrid-DNN for DDoS detection and classification in software-defined IoT networks","volume":"10","author":"Zainudin","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1186\/s13677-025-00762-9","article-title":"Energy-efficient deep learning-based intrusion detection system for edge computing: A novel DNN-KDQ model","volume":"14","author":"Umar","year":"2025","journal-title":"J. Cloud Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1186\/s13677-024-00609-9","article-title":"A fog-edge-enabled intrusion detection system for smart grids","volume":"13","author":"Tariq","year":"2024","journal-title":"J. Cloud Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yao, W., Hu, L., Hou, Y., and Li, X. (2023). A lightweight intelligent network intrusion detection system using one-class autoencoder and ensemble learning for IoT. Sensors, 23.","DOI":"10.3390\/s23084141"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"111563","DOI":"10.1016\/j.engappai.2025.111563","article-title":"An adaptive collaborative intrusion detection system for vehicular fog computing networks","volume":"158","author":"Frimpong","year":"2025","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Sudqi Khater, B., Abdul Wahab, A.W.B., Idris, M.Y.I.B., Abdulla Hussain, M., and Ahmed Ibrahim, A. (2019). A lightweight perceptron-based intrusion detection system for fog computing. Appl. Sci., 9.","DOI":"10.3390\/app9010178"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Patil, S., Varadarajan, V., Mazhar, S.M., Sahibzada, A., Ahmed, N., Sinha, O., Kumar, S., Shaw, K., and Kotecha, K. (2022). Explainable artificial intelligence for an intrusion detection system. Electronics, 11.","DOI":"10.3390\/electronics11193079"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Shi, Q., Xiao, F., Pickard, D., Chen, I., and Chen, L. (May, January 30). Deep neural network with linucb: A contextual bandit approach for personalized recommendation. Proceedings of the Companion Proceedings of the ACM Web Conference 2023, Austin, TX, USA.","DOI":"10.1145\/3543873.3587684"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Albulayhi, K., and Sheldon, F.T. (2021, January 10\u201313). An adaptive deep-ensemble anomaly-based intrusion detection system for the Internet of Things. Proceedings of the 2021 IEEE World AI IoT Congress (AIIoT), Online.","DOI":"10.1109\/AIIoT52608.2021.9454168"},{"key":"ref_28","unstructured":"Li, C., and Wang, H. (2022, January 28\u201330). Asynchronous upper confidence bound algorithms for federated linear bandits. Proceedings of the International Conference on Artificial Intelligence and Statistics, Online."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"9523","DOI":"10.1016\/j.jksuci.2021.11.006","article-title":"Concept drift detection in data stream mining: A literature review","volume":"34","author":"Agrahari","year":"2022","journal-title":"J. King Saud Univ. Comput. Inf. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Palli, A.S., Jaafar, J., Gomes, H.M., Hashmani, M.A., and Gilal, A.R. (2022). An experimental analysis of drift detection methods on multi-class imbalanced data streams. Appl. Sci., 12.","DOI":"10.3390\/app122211688"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"128933","DOI":"10.1016\/j.neucom.2024.128933","article-title":"Novelty-aware concept drift detection for neural networks","volume":"617","author":"Shang","year":"2025","journal-title":"Neurocomputing"},{"key":"ref_32","unstructured":"Fangwei, K.F. (2026, January 30). Contextual Bandit Analysis of the LinUCB Disjoint Algorithm with a Dataset. Contextual Bandits Analysis of LinUCB Disjoint Algorithm with Dataset. Available online: https:\/\/kfoofw.github.io\/contextual-bandits-linear-ucb-disjoint\/#:~:text=In%20matrix%20formulation%2C%20the%20above,return%20can%20be%20formulated%20as."},{"key":"ref_33","first-page":"23630","article-title":"Noise-adaptive Thompson sampling for linear contextual bandits","volume":"36","author":"Xu","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Qi, H., Guo, F., and Zhu, L. (2025). Thompson Sampling for Non-Stationary Bandit Problems. Entropy, 27.","DOI":"10.3390\/e27010051"},{"key":"ref_35","first-page":"9687","article-title":"Federated Bayesian optimization via Thompson sampling","volume":"33","author":"Dai","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2529","DOI":"10.1093\/comjnl\/bxae023","article-title":"Concept drift\u2013based intrusion detection for evolving data stream classification in IDS: Approaches and comparative study","volume":"67","author":"Seth","year":"2024","journal-title":"Comput. J."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"34617","DOI":"10.1109\/JIOT.2024.3407584","article-title":"Decentralized federated learning: A survey and perspective","volume":"11","author":"Yuan","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1145\/3785657","article-title":"Decentralized Federated Learning with Non-IID Data: Challenges, Trends, and Future Opportunities","volume":"58","author":"Chung","year":"2026","journal-title":"ACM Comput. Surv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"12796","DOI":"10.1109\/TNNLS.2023.3264740","article-title":"Clustered federated learning in heterogeneous environment","volume":"35","author":"Yan","year":"2023","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"7632","DOI":"10.1109\/TDSC.2025.3599321","article-title":"Self-Supervised Adaptation Method to Concept Drift for Network Intrusion Detection","volume":"22","author":"Yang","year":"2025","journal-title":"IEEE Trans. Dependable Secur. Comput."},{"key":"ref_41","unstructured":"McMahan, B., Moore, E., Ramage, D., Hampson, S., and Arcas, B.A. (2017, January 20\u201322). Communication-efficient learning of deep networks from decentralized data. Proceedings of the Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"B\u00e9n\u00e9zit, F., Blondel, V., Thiran, P., Tsitsiklis, J., and Vetterli, M. (2010, January 13\u201318). Weighted gossip: Distributed averaging using non-doubly stochastic matrices. Proceedings of the 2010 IEEE International Symposium on Information Theory, Austin, TX, USA.","DOI":"10.1109\/ISIT.2010.5513273"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Fahimullah, M., Philippe, G., Ahvar, S., and Trocan, M. (2023). Simulation tools for fog computing: A comparative analysis. Sensors, 23.","DOI":"10.3390\/s23073492"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/18\/2\/100\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T05:13:43Z","timestamp":1771305223000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/18\/2\/100"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,14]]},"references-count":43,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["fi18020100"],"URL":"https:\/\/doi.org\/10.3390\/fi18020100","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,14]]}}}