{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T20:14:31Z","timestamp":1774556071018,"version":"3.50.1"},"reference-count":49,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T00:00:00Z","timestamp":1710979200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T00:00:00Z","timestamp":1710979200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"NSF","award":["#2113945"],"award-info":[{"award-number":["#2113945"]}]},{"name":"North Carolina A&T University"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Netw Syst Manage"],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>OpenFlow-compliant commodity switches face challenges in efficiently managing flow rules due to the limited capacity of expensive high-speed memories used to store them. The accumulation of inactive flows can disrupt ongoing communication, necessitating an optimized approach to flow rule timeouts. This paper proposes Delayed Dynamic Timeout (DDT), a Reinforcement Learning-based approach to dynamically adjust flow rule timeouts and enhance the utilization of a switch\u2019s flow table(s) for improved efficiency. Despite the dynamic nature of network traffic, our DDT algorithm leverages advancements in Reinforcement Learning algorithms to adapt and achieve flow-specific optimization objectives. The evaluation results demonstrate that DDT outperforms static timeout values in terms of both flow rule match rate and flow rule activity. By continuously adapting to changing network conditions, DDT showcases the potential of Reinforcement Learning algorithms to effectively optimize flow rule management. This research contributes to the advancement of flow rule optimization techniques and highlights the feasibility of applying Reinforcement Learning in the context of SDN.<\/jats:p>","DOI":"10.1007\/s10922-024-09815-x","type":"journal-article","created":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T15:02:28Z","timestamp":1711033348000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["DDT: A Reinforcement Learning Approach to Dynamic Flow Timeout Assignment in Software Defined Networks"],"prefix":"10.1007","volume":"32","author":[{"suffix":"Jr.","given":"Nathan","family":"Harris","sequence":"first","affiliation":[]},{"given":"Sajad","family":"Khorsandroo","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,3,21]]},"reference":[{"issue":"2","key":"9815_CR1","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1145\/1355734.1355746","volume":"38","author":"N McKeown","year":"2008","unstructured":"McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., Shenker, S., Turner, J.: Openflow: enabling innovation in campus networks. ACM SIGCOMM Comput. Commun. rev. 38(2), 69\u201374 (2008)","journal-title":"ACM SIGCOMM Comput. Commun. rev."},{"key":"9815_CR2","unstructured":"Open Networking Foundation. https:\/\/opennetworking.org\/"},{"key":"9815_CR3","doi-asserted-by":"crossref","unstructured":"Katta, N., Alipourfard, O., Rexford, J., Walker, D.: Infinite cacheflow in software-defined networks. In: Proceedings of the Third Workshop on Hot Topics in Software-defined Networking, pp. 175\u2013180. ACM (2014)","DOI":"10.1145\/2620728.2620734"},{"key":"9815_CR4","doi-asserted-by":"crossref","unstructured":"Khorsandroo, S., Tosun, A.S.: Time inference attacks on software defined networks: Challenges and countermeasures. In: 2018 IEEE 11th International Conference on Cloud Computing (CLOUD), pp. 342\u2013349. IEEE (2018)","DOI":"10.1109\/CLOUD.2018.00050"},{"key":"9815_CR5","doi-asserted-by":"crossref","unstructured":"Khorsandroo, S., Tosun, A.S.: White box analysis at the service of low rate saturation attacks on virtual SDN data plane. In: 2019 IEEE 44th LCN Symposium on Emerging Topics in Networking (LCN Symposium), pp. 100\u2013107. IEEE (2019)","DOI":"10.1109\/LCNSymposium47956.2019.9000660"},{"key":"9815_CR6","doi-asserted-by":"crossref","unstructured":"Qiao, S., Hu, C., Guan, X., Zou, J.: Taming the flow table overflow in openflow switch. In: Proceedings of the 2016 ACM SIGCOMM Conference, pp. 591\u2013592 (2016)","DOI":"10.1145\/2934872.2959063"},{"key":"9815_CR7","doi-asserted-by":"crossref","unstructured":"Wang, L., Song, C., Xu, Z., et al.: Proactive mitigation to table-overflow in software-defined networking. In: 2018 IEEE Symposium on Computers and Communications (ISCC), pp. 00719\u201300725. IEEE (2018)","DOI":"10.1109\/ISCC.2018.8538670"},{"issue":"4","key":"9815_CR8","doi-asserted-by":"publisher","first-page":"254","DOI":"10.1145\/2043164.2018466","volume":"41","author":"AR Curtis","year":"2011","unstructured":"Curtis, A.R., Mogul, J.C., Tourrilhes, J., Yalagandula, P., Sharma, P., Banerjee, S.: Devoflow: scaling flow management for high-performance networks. ACM SIGCOMM Comput. Commun. Rev. 41(4), 254\u2013265 (2011)","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"issue":"1","key":"9815_CR9","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1109\/LCOMM.2015.2496955","volume":"20","author":"M Aslan","year":"2015","unstructured":"Aslan, M., Matrawy, A.: On the impact of network state collection on the performance of SDN applications. IEEE Commun. Lett. 20(1), 5\u20138 (2015)","journal-title":"IEEE Commun. Lett."},{"key":"9815_CR10","volume-title":"Openflow Timeouts Demystified","author":"A Zarek","year":"2012","unstructured":"Zarek, A., Ganjali, Y., Lie, D.: Openflow Timeouts Demystified. Univ. of Toronto, Toronto (2012)"},{"key":"9815_CR11","unstructured":"Ryu, B., Cheney, D., Braun, H.-W.: Internet flow characterization: adaptive timeout strategy and statistical modeling. In: Workshop on Passive and Active Measurement (PAM), vol. 105 (2001)"},{"key":"9815_CR12","doi-asserted-by":"publisher","first-page":"14952","DOI":"10.1109\/ACCESS.2017.2726114","volume":"5","author":"T Li","year":"2017","unstructured":"Li, T., Zhou, H., Luo, H., You, I., Xu, Q.: Sat-flow: multi-strategy flow table management for software defined satellite networks. IEEE Access 5, 14952\u201314965 (2017)","journal-title":"IEEE Access"},{"key":"9815_CR13","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym. arXiv preprint arXiv:1606.01540 (2016)"},{"key":"9815_CR14","doi-asserted-by":"crossref","unstructured":"Zhu, H., Fan, H., Luo, X., Jin, Y.: Intelligent timeout master: dynamic timeout for SDN-based data centers. In: 2015 IFIP\/IEEE International Symposium on Integrated Network Management (IM), pp. 734\u2013737. IEEE (2015)","DOI":"10.1109\/INM.2015.7140363"},{"key":"9815_CR15","doi-asserted-by":"crossref","unstructured":"Zhang, L., Wang, S., Xu, S., Lin, R., Yu, H.: Timeoutx: an adaptive flow table management method in software defined networks. In: 2015 IEEE Global Communications Conference (GLOBECOM), pp. 1\u20136. IEEE (2015)","DOI":"10.1109\/GLOCOM.2015.7417563"},{"key":"9815_CR16","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/j.comnet.2017.04.046","volume":"125","author":"Z Guo","year":"2017","unstructured":"Guo, Z., Liu, R., Xu, Y., Gushchin, A., Walid, A., Chao, H.J.: Star: preventing flow-table overflow in software-defined networks. Comput. Netw. 125, 15\u201325 (2017)","journal-title":"Comput. Netw."},{"issue":"5","key":"9815_CR17","doi-asserted-by":"publisher","first-page":"331","DOI":"10.7763\/IJFCC.2014.V3.321","volume":"3","author":"T Kim","year":"2014","unstructured":"Kim, T., Lee, K., Lee, J., Park, S., Kim, Y.-H., Lee, B.: A dynamic timeout control algorithm in software defined networks. Int. J. Future Comput. Commun. 3(5), 331 (2014)","journal-title":"Int. J. Future Comput. Commun."},{"key":"9815_CR18","doi-asserted-by":"crossref","unstructured":"Lu, M., Deng, W., Shi, Y.: Tf-idletimeout: improving efficiency of TCAM in SDN by dynamically adjusting flow entry lifecycle. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 002681\u2013002686. IEEE (2016)","DOI":"10.1109\/SMC.2016.7844645"},{"key":"9815_CR19","doi-asserted-by":"crossref","unstructured":"Liu, Y., Tang, B., Yuan, D., Ran, J., Hu, H.: A dynamic adaptive timeout approach for SDN switch. In: 2016 2nd IEEE International Conference on Computer and Communications (ICCC), pp. 2577\u20132582. IEEE (2016)","DOI":"10.1109\/CompComm.2016.7925164"},{"key":"9815_CR20","doi-asserted-by":"crossref","unstructured":"Wang, D., Li, Q., Wang, L., Sinnott, R.O., Jiang, Y.: A hybrid-timeout mechanism to handle rule dependencies in software defined networks. In: 2017 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 241\u2013246. IEEE (2017)","DOI":"10.1109\/INFCOMW.2017.8116383"},{"key":"9815_CR21","doi-asserted-by":"crossref","unstructured":"Panda, A., Samal, S.S., Turuk, A.K., Panda, A., Venkatesh, V.C.: Dynamic hard timeout based flow table management in openflow enabled SDN. In: 2019 International Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN), pp. 1\u20136. IEEE (2019)","DOI":"10.1109\/ViTECoN.2019.8899359"},{"issue":"1","key":"9815_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13174-018-0087-2","volume":"9","author":"R Boutaba","year":"2018","unstructured":"Boutaba, R., Salahuddin, M.A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., Caicedo, O.M.: A comprehensive survey on machine learning for networking: evolution, applications and research opportunities. J. Internet Serv. Appl. 9(1), 1\u201399 (2018)","journal-title":"J. Internet Serv. Appl."},{"key":"9815_CR23","doi-asserted-by":"crossref","unstructured":"Haq, F., Naaz, A., Bantupalli, T.P.K., Kataoka, K.: Drl-fto: dynamic flow rule timeout optimization in SDN using deep reinforcement learning. In: Asian Internet Engineering Conference, pp. 41\u201348 (2021)","DOI":"10.1145\/3497777.3498549"},{"issue":"1","key":"9815_CR24","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1109\/TNSM.2018.2890754","volume":"16","author":"Q Li","year":"2019","unstructured":"Li, Q., Huang, N., Wang, D., Li, X., Jiang, Y., Song, Z.: Hqtimer: a hybrid $$q$$-learning-based timeout mechanism in software-defined networks. IEEE Trans. Netw. Serv. Manag. 16(1), 153\u2013166 (2019)","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"9815_CR25","unstructured":"Fujimoto, S., Hoof, H., Meger, D.: Addressing function approximation error in actor-critic methods. In: International Conference on Machine Learning, pp. 1587\u20131596. PMLR (2018)"},{"key":"9815_CR26","doi-asserted-by":"publisher","first-page":"65579","DOI":"10.1109\/ACCESS.2019.2916648","volume":"7","author":"M Usama","year":"2019","unstructured":"Usama, M., Qadir, J., Raza, A., Arif, H., Yau, K.-L.A., Elkhatib, Y., Hussain, A., Al-Fuqaha, A.: Unsupervised machine learning for networking: techniques, applications and research challenges. IEEE Access 7, 65579\u201365615 (2019)","journal-title":"IEEE Access"},{"key":"9815_CR27","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)"},{"key":"9815_CR28","unstructured":"RYU. https:\/\/ryu-sdn.org"},{"key":"9815_CR29","unstructured":"Specification, O.S.: Open networking foundation. Version ONF TS-015 1(3), pp. 1\u2013164 (2013)"},{"key":"9815_CR30","unstructured":"Yahyaoui, H., Zhani, M.F., Bouachir, O., Aloqaily, M.: On minimizing flow monitoring costs in large-scale SDN networks"},{"key":"9815_CR31","unstructured":"Foundation, O.N.: OpenFlow Switch Specification. https:\/\/opennetworking.org\/wp-content\/uploads\/2014\/10\/openflow-spec-v1.3.0.pdf"},{"key":"9815_CR32","unstructured":"OpenFlow v1.3 Messages and Structures: Ryu 4.34 documentation. https:\/\/ryu.readthedocs.io\/en\/latest\/ofproto_v1_3_ref.html"},{"key":"9815_CR33","doi-asserted-by":"crossref","unstructured":"Vidyadhar, V., Nagaraj, R., Ashoka, D.: Netai-gym: customized environment for network to evaluate agent algorithm using reinforcement learning in open-AI gym platform. Int. J. Adv. Comput. Sci. Appl. 12(4) (2021)","DOI":"10.14569\/IJACSA.2021.0120423"},{"issue":"2","key":"9815_CR34","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1109\/MNET.2017.1500057NM","volume":"31","author":"Y Zhao","year":"2017","unstructured":"Zhao, Y., Zhang, B., Li, C., Chen, C.: On\/off traffic shaping in the internet: motivation, challenges, and solutions. IEEE Netw. 31(2), 48\u201357 (2017)","journal-title":"IEEE Netw."},{"key":"9815_CR35","unstructured":"Chandrasekaran, B.: Survey of Network Traffic Models. Waschington University in St. Louis CSE 567 (2009)"},{"issue":"7","key":"9815_CR36","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/35.601746","volume":"35","author":"A Adas","year":"1997","unstructured":"Adas, A.: Traffic models in broadband networks. IEEE Commun. Mag. 35(7), 82\u201389 (1997)","journal-title":"IEEE Commun. Mag."},{"issue":"1","key":"9815_CR37","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1145\/1672308.1672325","volume":"40","author":"T Benson","year":"2010","unstructured":"Benson, T., Anand, A., Akella, A., Zhang, M.: Understanding data center traffic characteristics. ACM SIGCOMM Comput. Commun. Rev. 40(1), 92\u201399 (2010)","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"key":"9815_CR38","doi-asserted-by":"crossref","unstructured":"Kandula, S., Sengupta, S., Greenberg, A., Patel, P., Chaiken, R.: The nature of data center traffic: measurements & analysis. In: Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement, pp. 202\u2013208 (2009)","DOI":"10.1145\/1644893.1644918"},{"key":"9815_CR39","unstructured":"cachetools 5.3.1. https:\/\/pypi.org\/project\/cachetools\/"},{"key":"9815_CR40","unstructured":"Laboratory, U.N.R.: Multi-Generator (MGEN) traffic generation tool (2021). https:\/\/github.com\/USNavalResearchLaboratory\/mgen"},{"key":"9815_CR41","unstructured":"Kaur, K., Singh, J., Ghumman, N.S.: Mininet as software defined networking testing platform. In: International Conference on Communication, Computing & Systems (ICCCS), pp. 139\u201342 (2014)"},{"key":"9815_CR42","unstructured":"Pfaff, B., Pettit, J., Koponen, T., Jackson, E., Zhou, A., Rajahalme, J., Gross, J., Wang, A., Stringer, J., Shelar, P., et al.: The design and implementation of open vswitch. In: 12th $$\\{$$USENIX$$\\}$$ Symposium on Networked Systems Design and Implementation ($$\\{$$NSDI$$\\}$$ 15), pp. 117\u2013130 (2015)"},{"key":"9815_CR43","unstructured":"OpenDaylight. https:\/\/opendaylight.org\/"},{"issue":"3","key":"9815_CR44","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1145\/1384609.1384625","volume":"38","author":"N Gude","year":"2008","unstructured":"Gude, N., Koponen, T., Pettit, J., Pfaff, B., Casado, M., McKeown, N., Shenker, S.: Nox: towards an operating system for networks. ACM SIGCOMM Comput. Commun. Rev. 38(3), 105\u2013110 (2008)","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"issue":"3","key":"9815_CR45","doi-asserted-by":"publisher","first-page":"2846","DOI":"10.1109\/TNSM.2020.3030102","volume":"18","author":"H Harkous","year":"2020","unstructured":"Harkous, H., Jarschel, M., He, M., Pries, R., Kellerer, W.: P8: P4 with predictable packet processing performance. IEEE Trans. Netw. Serv. Manag. 18(3), 2846\u20132859 (2020)","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"9815_CR46","unstructured":"P4.org: Clarifying the differences between P4 and OpenFlow. Open Networking Foundation (2021). https:\/\/opennetworking.org\/news-and-events\/blog\/clarifying-the-differences-between-p4-and-openflow\/"},{"key":"9815_CR47","doi-asserted-by":"crossref","unstructured":"Xie, L., Zhao, Z., Zhou, Y., Wang, G., Ying, Q., Zhang, H.: An adaptive scheme for data forwarding in software defined network. In: 2014 Sixth International Conference on Wireless Communications and Signal Processing (WCSP), pp. 1\u20135. IEEE (2014)","DOI":"10.1109\/WCSP.2014.6992181"},{"key":"9815_CR48","doi-asserted-by":"crossref","unstructured":"Jim\u00e9nez-L\u00e1zaro, M., Berrocal, J., Gal\u00e1n-Jim\u00e9nez, J.: Flow-based service time optimization in software-defined networks using deep reinforcement learning. Comput. Commun. (2024)","DOI":"10.2139\/ssrn.4576661"},{"key":"9815_CR49","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1016\/j.comcom.2021.10.006","volume":"181","author":"TG Nguyen","year":"2022","unstructured":"Nguyen, T.G., Phan, T.V., Hoang, D.T., Nguyen, H.H., Le, D.T.: Deepplace: deep reinforcement learning for adaptive flow rule placement in software-defined IoT networks. Comput. Commun. 181, 156\u2013163 (2022)","journal-title":"Comput. Commun."}],"container-title":["Journal of Network and Systems Management"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10922-024-09815-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10922-024-09815-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10922-024-09815-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T16:03:34Z","timestamp":1731600214000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10922-024-09815-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,21]]},"references-count":49,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["9815"],"URL":"https:\/\/doi.org\/10.1007\/s10922-024-09815-x","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-3200987\/v1","asserted-by":"object"}]},"ISSN":["1064-7570","1573-7705"],"issn-type":[{"value":"1064-7570","type":"print"},{"value":"1573-7705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,21]]},"assertion":[{"value":"25 July 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 February 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 February 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 March 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}}],"article-number":"35"}}