{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T02:06:42Z","timestamp":1771466802562,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,2,14]],"date-time":"2024-02-14T00:00:00Z","timestamp":1707868800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the rapid advancement of the Internet of Things (IoT), there is a global surge in network traffic. Software-Defined Networks (SDNs) provide a holistic network perspective, facilitating software-based traffic analysis, and are more suitable to handle dynamic loads than a traditional network. The standard SDN architecture control plane has been designed for a single controller or multiple distributed controllers; however, a logically centralized single controller faces severe bottleneck issues. Most proposed solutions in the literature are based on the static deployment of multiple controllers without the consideration of flow fluctuations and traffic bursts, which ultimately leads to a lack of load balancing among controllers in real time, resulting in increased network latency. Moreover, some methods addressing dynamic controller mapping in multi-controller SDNs consider load fluctuation and latency but face controller placement problems. Earlier, we proposed priority scheduling and congestion control algorithm (eSDN) and dynamic mapping of controllers for dynamic SDN (dSDN) to address this issue. However, the future growth of IoT is unpredictable and potentially exponential; to accommodate this futuristic trend, we need an intelligent solution to handle the complexity of growing heterogeneous devices and minimize network latency. Therefore, this paper continues our previous research and proposes temporal deep Q learning in the dSDN controller. A Temporal Deep Q learning Network (tDQN) serves as a self-learning reinforcement-based model. The agent in the tDQN learns to improve decision-making for switch-controller mapping through a reward\u2013punish scheme, maximizing the goal of reducing network latency during the iterative learning process. Our approach\u2014tDQN\u2014effectively addresses dynamic flow mapping and latency optimization without increasing the number of optimally placed controllers. A multi-objective optimization problem for flow fluctuation is formulated to divert the traffic to the best-suited controller dynamically. Extensive simulation results with varied network scenarios and traffic show that the tDQN outperforms traditional networks, eSDNs, and dSDNs in terms of throughput, delay, jitter, packet delivery ratio, and packet loss.<\/jats:p>","DOI":"10.3390\/s24041216","type":"journal-article","created":{"date-parts":[[2024,2,14]],"date-time":"2024-02-14T09:30:18Z","timestamp":1707903018000},"page":"1216","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["A Temporal Deep Q Learning for Optimal Load Balancing in Software-Defined Networks"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9556-0638","authenticated-orcid":false,"given":"Aakanksha","family":"Sharma","sequence":"first","affiliation":[{"name":"Melbourne Institute of Technology (MIT), Melbourne, VIC 3000, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6686-4424","authenticated-orcid":false,"given":"Venki","family":"Balasubramanian","sequence":"additional","affiliation":[{"name":"Institute of Innovation, Science and Sustainability, Federation University Australia, Ballarat, VIC 3350, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3748-0277","authenticated-orcid":false,"given":"Joarder","family":"Kamruzzaman","sequence":"additional","affiliation":[{"name":"Institute of Innovation, Science and Sustainability, Federation University Australia, Ballarat, VIC 3350, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1109\/TNSM.2019.2946428","article-title":"A QoS-aware data collection protocol for LLNs in fog-enabled Internet of Things","volume":"17","author":"Hosen","year":"2019","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4188","DOI":"10.1109\/JIOT.2018.2875246","article-title":"Joint computation and communication cooperation for energy-efficient mobile edge computing","volume":"6","author":"Cao","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3337065","article-title":"Big data analytics for large-scale wireless networks: Challenges and opportunities","volume":"52","author":"Dai","year":"2019","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_4","unstructured":"Fisher, W., Suchara, M., and Rexford, J. (2010). First ACM SIGCOMM Workshop on Green Networking, in Green Networking \u201910, Association for Computing Machinery."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Mahadevan, P., Sharma, P., Banerjee, S., and Ranganathan, P. (2009, January 11\u201315). A Power Benchmarking Framework For Network Devices. Proceedings of the NETWORKING 2009: 8th International IFIP-TC 6 Networking Conference, Aachen, Germany.","DOI":"10.1007\/978-3-642-01399-7_62"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.csi.2017.04.001","article-title":"Energy saving in carrier-grade networks: A survey","volume":"55","author":"Maaloul","year":"2018","journal-title":"Comput. Stand. Interfaces"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1109\/SURV.2011.071410.00073","article-title":"Energy efficiency in the future internet: A survey of existing approaches and trends in energy-aware fixed network infrastructures","volume":"13","author":"Bolla","year":"2010","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/2534169.2486019","article-title":"B4: Experience with a globally-deployed software defined WAN","volume":"43","author":"Jain","year":"2013","journal-title":"Acm Sigcomm Comput. Commun. Rev."},{"key":"ref_9","first-page":"1","article-title":"Are we ready to drive software-defined networks? A comprehensive survey on management tools and techniques","volume":"51","author":"Rojas","year":"2018","journal-title":"ACM Comput. Surv.s (CSUR)"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1109\/COMST.2018.2866942","article-title":"A survey of machine learning techniques applied to software defined networking (SDN): Research issues and challenges","volume":"21","author":"Xie","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/MNET.2017.1600182","article-title":"The controller placement problem in software defined networking: A survey","volume":"31","author":"Wang","year":"2017","journal-title":"IEEE Netw."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.comnet.2017.04.029","article-title":"LCMSC: A lightweight collaborative mechanism for SDN controllers","volume":"121","author":"Chen","year":"2017","journal-title":"Comput. Netw."},{"key":"ref_13","unstructured":"Koponen, T., Casado, M., Gude, N., Stribling, J., Poutievski, L., Zhu, M., Ramanathan, R., Iwata, Y., Inoue, H., and Hama, T. (2010, January 4\u20136). Onix: A Distributed Control Platform For Large-Scale Production networks. Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI 10), Vancouver, BC, Canada."},{"key":"ref_14","unstructured":"Tootoonchian, A., and Ganjali, Y. (2010, January 27). Hyperflow: A Distributed Control Plane For Openflow. Proceedings of the 2010 Internet Network Management Conference On Research On Enterprise Networking, San Jose, CA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Curtis, A.R., Mogul, J.C., Tourrilhes, J., Yalagandula, P., Sharma, P., and Banerjee, S. (2011, January 15\u201319). DevoFlow: Scaling Flow Management for High-Performance Networks. Proceedings of the ACM SIGCOMM 2011 Conference, Toronto, ON, Canada.","DOI":"10.1145\/2018436.2018466"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Muthanna, A., Ateya, A.A., Makolkina, M., Vybornova, A., Markova, E., Gogol, A., and Koucheryavy, A. (2018, January 26\u201327). SDN Multi-Controller Networks with Load Balanced. Proceedings of the 2nd International Conference on Future Networks and Distributed Systems, New York, NY, USA.","DOI":"10.1145\/3231053.3231124"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cheng, T.Y., Wang, M., and Jia, X. (2015, January 9\u201312). QoS-Guaranteed Controller Placement in SDN. Proceedings of the 2015 IEEE Global Communications Conference (GLOBECOM), Fort Lauderdale, FL, USA.","DOI":"10.1109\/GLOCOM.2015.7416960"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1145\/2534169.2491193","article-title":"Towards an elastic distributed SDN controller","volume":"43","author":"Dixit","year":"2013","journal-title":"ACM Sigcomm Comput. Commun. Rev."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sharma, A., Balasubramanian, V., and Kamruzzaman, J. (2023). A Novel Dynamic Software-Defined Networking Approach to Neutralize Traffic Burst. Computers, 12.","DOI":"10.3390\/computers12070131"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1145\/1355734.1355746","article-title":"OpenFlow: Enabling innovation in campus networks","volume":"38","author":"McKeown","year":"2008","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1109\/CC.2015.7112035","article-title":"HiQoS: An SDN-based multipath QoS solution","volume":"12","author":"Yan","year":"2015","journal-title":"China Commun."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1145\/1384609.1384625","article-title":"NOX: Towards an operating system for networks","volume":"38","author":"Gude","year":"2008","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gupta, R., and Gupta, R. (2016, January 18\u201319). ABC of Internet of Things: Advancements, Benefits, Challenges, Enablers And Facilities of IoT. Proceedings of the 2016 Symposium on Colossal Data Analysis and Networking (CDAN), Indore, India.","DOI":"10.1109\/CDAN.2016.7570875"},{"key":"ref_24","unstructured":"Kaur, S., Kumar, K., Singh, J., and Ghumman, N.S. (2015, January 11\u201313). Round-robin Based Load Balancing in Software Defined Networking. Proceedings of the 2015 2nd International Conference On Computing For Sustainable Global Development (INDIACom), New Delhi, India."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Hong, C.Y., Kandula, S., Mahajan, R., Zhang, M., Gill, V., Nanduri, M., and Wattenhofer, R. (2013, January 12\u201316). Achieving High Utilization with Software-Driven WAN. Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM, Hong Kong, China.","DOI":"10.1145\/2486001.2486012"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1339","DOI":"10.1109\/LCOMM.2014.2332341","article-title":"On the capacitated controller placement problem in software defined networks","volume":"18","author":"Yao","year":"2014","journal-title":"IEEE Commun. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"01014","DOI":"10.1051\/matecconf\/201714001014","article-title":"Controller placement algorithms in software defined network-a review of trends and challenges","volume":"Volume 140","author":"Yoon","year":"2017","journal-title":"Proceedings of the MATEC Web of Conferences"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1109\/MCOM.2013.6461195","article-title":"Improving network management with software defined networking","volume":"51","author":"Kim","year":"2013","journal-title":"IEEE Commun. Mag."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1645","DOI":"10.1016\/j.future.2013.01.010","article-title":"Internet of Things (IoT): A vision, architectural elements, and future directions","volume":"29","author":"Gubbi","year":"2013","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Oteafy, S.M., and Hassanein, H.S. (February, January 30). Towards a Global IoT: Resource Re-utilization in WSNs. Proceedings of the 2012 International Conference On Computing, Networking And Communications (ICNC), Maui, HI, USA.","DOI":"10.1109\/ICCNC.2012.6167496"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"14159","DOI":"10.1109\/ACCESS.2018.2805842","article-title":"Load balancing mechanisms in the software defined networks: A systematic and comprehensive review of the literature","volume":"6","author":"Neghabi","year":"2018","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Li, X., Djukic, P., and Zhang, H. (2014, January 5\u20139). Zoning for Hierarchical Network Optimization in Software Defined Networks. Proceedings of the 2014 IEEE Network Operations and Management Symposium (NOMS), Krakow, Poland.","DOI":"10.1109\/NOMS.2014.6838414"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"He, M., Basta, A., Blenk, A., and Kellerer, W. (2017, January 21\u201325). Modeling Flow Setup Time for Controller Placement in Sdn: Evaluation for Dynamic Flows. Proceedings of the 2017 IEEE International Conference on Communications (ICC), Paris, France.","DOI":"10.1109\/ICC.2017.7996654"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Yao, L., Hong, P., Zhang, W., Li, J., and Ni, D. (2015, January 8\u201312). Controller Placement and Flow Based Dynamic Management Problem towards SDN. Proceedings of the 2015 IEEE International Conference on Communication Workshop (ICCW), London, UK.","DOI":"10.1109\/ICCW.2015.7247206"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/LCOMM.2014.2371014","article-title":"Optimal model for the controller placement problem in software defined networks","volume":"19","author":"Sallahi","year":"2014","journal-title":"IEEE Commun. Lett."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Liang, C., Kawashima, R., and Matsuo, H. (2014, January 10\u201312). Scalable and Crash-Tolerant Load Balancing Based on Switch Migration for Multiple Open Flow Controllers. Proceedings of the 2014 Second International Symposium on Computing and Networking, Shizuoka, Japan.","DOI":"10.1109\/CANDAR.2014.108"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Li, J., Hu, X., and Zhang, M. (2018, January 12\u201314). Research on Dynamic Switch Migration Strategy Based on Fmopso. Proceedings of the 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China.","DOI":"10.1109\/IAEAC.2018.8577774"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhou, X., Gao, J., and Qin, Y. (2018, January 23\u201325). SDN Controller Load Balancing Based on Reinforcement Learning. Proceedings of the 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China.","DOI":"10.1109\/ICSESS.2018.8663757"},{"key":"ref_40","unstructured":"McKinney, W. (2012). Python for Data Analysis: Data Wrangling with Pandas, NumPy, and IPython, O\u2019Reilly Media, Inc."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1216\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:59:42Z","timestamp":1760104782000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1216"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,14]]},"references-count":40,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,2]]}},"alternative-id":["s24041216"],"URL":"https:\/\/doi.org\/10.3390\/s24041216","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,14]]}}}