{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,14]],"date-time":"2025-10-14T00:47:44Z","timestamp":1760402864451,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,1,13]],"date-time":"2022-01-13T00:00:00Z","timestamp":1642032000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Kaken Pharmaceutical (Japan)","award":["20K19791"],"award-info":[{"award-number":["20K19791"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Many studies focusing on improving Transmission Control Protocol (TCP) flow control realize a more effective use of bandwidth in data center networks. They are excellent ways to more effectively use the bandwidth between clients and back-end servers. However, these schemes cannot achieve the total optimization of bandwidth use for data center networks as they do not take into account the path design of TCP flows against a hierarchical complex structure of data center networks. To address this issue, this paper proposes a TCP flow management scheme specified a hierarchical complex data center network for effective bandwidth use. The proposed scheme dynamically controls the paths of TCP flows by reinforcement learning based on a hierarchical feedback model, which obtains an optimal TCP flow establishment policy even if both the network topology and link states are more complicated. In evaluation, the proposed scheme achieved more effective bandwidth use and reduced the probability of TCP incast up to 30% than the conventional TCP flow management schemes: Variant Load Balancing (VLB), Equal Cost Multi Path (ECMP), and Intelligent Forwarding Strategy Based on Reinforcement Learning (IFS-RL) in the complex data center network.<\/jats:p>","DOI":"10.3390\/s22020611","type":"journal-article","created":{"date-parts":[[2022,1,14]],"date-time":"2022-01-14T03:14:56Z","timestamp":1642130096000},"page":"611","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Effective TCP Flow Management Based on Hierarchical Feedback Learning in Complex Data Center Network"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2020-6578","authenticated-orcid":false,"given":"Kimihiro","family":"Mizutani","sequence":"first","affiliation":[{"name":"Department of Informatics, Kindai University, 3-4-1, Kowakae, Higashi-Osaka 577-8502, Osaka, Japan"},{"name":"Cyber Informatics Research Institute, Kindai University, 3-4-1, Kowakae, Higashi-Osaka 577-8502, Osaka, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,13]]},"reference":[{"key":"ref_1","first-page":"95","article-title":"VL2: A scalable and flexible data center network","volume":"54","author":"Greenberg","year":"2011","journal-title":"ACM Spec. Interest Group Data Commun. (SIGCOMM) Comput. Commun. Rev."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1109\/TPDS.2018.2874659","article-title":"HSDC: A highly scalable data center network architecture for greater incremental scalability","volume":"30","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Jain, N., Bhatele, A., Howell, L.H., B\u00f6hme, D., Karlin, I., Le\u00f3n, E.A., Mubarak, M., Wolfe, N., Gamblin, T., and Leininger, M.L. (2017, January 12\u201317). Predicting the performance impact of different fat-tree configurations. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Denver, CO, USA.","DOI":"10.1145\/3126908.3126967"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"102421","DOI":"10.1016\/j.jnca.2019.102421","article-title":"TCP incast solutions in data center networks: A classification and survey","volume":"146","author":"Alipio","year":"2019","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Han, Z., and Yu, L. (2018, January 3\u20135). A survey of the bcube data center network topology. Proceedings of the International Conference on Big Data Security on Cloud (BigDataSecurity), Omaha, NE, USA.","DOI":"10.1109\/BDS\/HPSC\/IDS18.2018.00056"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Guo, C., Wu, H., Tan, K., Shi, L., Zhang, Y., and Lu, S. (2008, January 17\u201322). Dcell: A scalable and fault-tolerant network structure for data centers. Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM) Annual Conference, Seattle, WA, USA.","DOI":"10.1145\/1402958.1402968"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hong, E.T.B., and Wey, C.Y. (2017, January 11\u201313). An optimized flow management mechanism in OpenFlow network. Proceedings of the International Conference on Information Networking (ICOIN), Da Nang, Vietnam.","DOI":"10.1109\/ICOIN.2017.7899493"},{"key":"ref_8","unstructured":"Hollot, C.V., Misra, V., Towsley, D., and Gong, W.B. (2001, January 22\u201326). On designing improved controllers for AQM routers supporting TCP flows. Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), Anchorage, AK, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Bai, B., Xu, K., and Lei, K. (2018, January 20\u201325). IFS-RL: An Intelligent Forwarding Strategy Based on Reinforcement Learning in Named-Data Networking. Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM) Workshop on Network Meets AI and ML (NetAI\u201918), Budapest, Hungary.","DOI":"10.1145\/3229543.3229547"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"38427","DOI":"10.1109\/ACCESS.2021.3064008","article-title":"Survey on Traffic Management in Data Center Network: From Link Layer to Application Layer","volume":"9","author":"Li","year":"2021","journal-title":"IEEE Access"},{"key":"ref_11","unstructured":"Alizadeh, M., Greenberg, A., Maltz, D., Padhye, J., Patel, P., Prabhakar, B., Sengupta, S., and Sridharan, M. (September, January 30). Data center TCP (DCTCP). Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM) Annual Conference, Delhi, India."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chen, L., Lingys, J., Chen, K., and Liu, F. (2018, January 20\u201325). AuTO: Scaling Deep Reinforcement Learning for Datacenter-Scale Automatic Traffic Optimization. Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM) Annual Conference, Budapest, Hungary.","DOI":"10.1145\/3230543.3230551"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1145\/2043164.2018443","article-title":"Better never than late: Meeting deadlines in datacenter networks","volume":"41","author":"Wilson","year":"2011","journal-title":"ACM Spec. Interest Group Data Commun. (SIGCOMM), Comput. Commun. Rev."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Wischik, D., Raiciu, C., Greenhalgh, A., and Handley, M. (2011, January 25\u201327). Design, Implementation and Evaluation of Congestion Control for Multipath TCP. Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), San Jose, CA, USA.","DOI":"10.17487\/rfc6356"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Van Der Pol, R., Boele, S., Dijkstra, F., Barczyk, A., van Malenstein, G., Chen, J.H., and Mambretti, J. (2012, January 10\u201316). Multipathing with MPTCP and OpenFlow. Proceedings of the Super Computing (SC), High Performance Computing, Networking Storage and Analysis, Salt Lake City, UT, USA.","DOI":"10.1109\/SC.Companion.2012.339"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Narayan, A., Cangialosi, F., Raghavan, D., Goyal, P., Narayana, S., Mittal, R., Alizadeh, M., and Balakrishnan, H. (2018, January 20\u201325). Restructuring endpoint congestion control. Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM), Budapest, Hungary.","DOI":"10.1145\/3230543.3230553"},{"key":"ref_17","unstructured":"Tokui, S., Oono, K., Hido, S., and Clayton, J. (2015, January 2\u20138). Chainer: A next-generation open source framework for deep learning. Proceedings of the Workshop on Machine Learning Systems (LEARNINGSYS) in the Twenty-Ninth Annual Conference on Neural Information Processing Systems (NIPS), Montreal, QC, Canada."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Jose, L., Yan, L., Alizadeh, M., Varghese, G., McKeown, N., and Katti, S. (2015, January 16\u201317). High speed networks need proactive congestion control. Proceedings of the ACM Workshop on Hot Topics in Networks, Philadelphia, PA, USA.","DOI":"10.1145\/2834050.2834096"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.jnca.2015.03.010","article-title":"DC-Vegas: A delay-based TCP congestion control algorithm for datacenter applications","volume":"53","author":"Wang","year":"2015","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MNET.2005.1383434","article-title":"FAST TCP: From theory to experiments","volume":"19","author":"Jin","year":"2005","journal-title":"IEEE Netw."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Xia, Y., Wu, J., Xia, J., Wang, T., and Mao, S. (2021, January 25\u201328). Multipath-aware TCP for Data Center Traffic Load-balancing. Proceedings of the IEEE\/ACM 29th International Symposium on Quality of Service (IWQOS), Tokyo, Japan.","DOI":"10.1109\/IWQOS52092.2021.9521276"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1325","DOI":"10.1109\/JSAC.2019.2904358","article-title":"Experience-Driven Congestion Control: When Multi-Path TCP Meets Deep Reinforcement Learning","volume":"37","author":"Xu","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2155","DOI":"10.1109\/TNSE.2020.2991106","article-title":"A software defined network based fuzzy normalized neural adaptive multipath congestion control for the Internet of Things","volume":"7","author":"Naeem","year":"2020","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cao, Y., Dai, B., Mo, Y., and Xu, Y. (2020, January 16\u201319). IQoR: An Intelligent QoS-aware Routing Mechanism with Deep Reinforcement Learning. Proceedings of the IEEE Conference on Local Computer Networks (LCN), Sydney, Australia.","DOI":"10.1109\/LCN48667.2020.9314768"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Dhurandher, S.K., Singh, J., Obaidat, M.S., Woungang, I., Srivastava, S., and Rodrigues, J.J. (2020, January 7\u201311). Reinforcement learning-based routing protocol for opportunistic networks. Proceedings of the IEEE International Conference on Communications (ICC), Dublin, Ireland.","DOI":"10.1109\/ICC40277.2020.9149039"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Mizutani, K., Akashi, O., Terauchi, A., Kugimoto, T., and Maruyama, M. (2012, January 16\u201320). A dynamic flow control mechanism based on a hierarchical feedback model for data center networks. Proceedings of the IEEE Network Operations and Management Symposium, Maui, HI, USA.","DOI":"10.1109\/NOMS.2012.6211965"},{"key":"ref_27","unstructured":"Abu-Libdeh, H., Costa, P., Rowstron, A., O\u2019Shea, G., and Donnelly, A. (September, January 30). Symbiotic routing in future data centers. Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM) Annual Conference, Delhi, India."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Touihri, R., Alwan, S., Dandoush, A., Aitsaadi, N., and Veillon, C. (2019, January 22\u201324). CRP: Optimized SDN routing protocol in server-only CamCube data-center networks. Proceedings of the IEEE International Conference on Communications (ICC), Shanghai, China.","DOI":"10.1109\/ICC.2019.8761884"},{"key":"ref_29","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1162\/neco.1996.8.2.340","article-title":"Annealed competition of experts for a segmentation and classification of switching dynamics","volume":"8","author":"Pawelzik","year":"1996","journal-title":"Neural Comput."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1145\/263932.264023","article-title":"The macroscopic behavior of the TCP congestion avoidance algorithm","volume":"27","author":"Mathis","year":"1997","journal-title":"ACM Spec. Interest Group Data Commun. (SIGCOMM) Comput. Commun. Rev."},{"key":"ref_33","first-page":"1","article-title":"Chainerrl: A deep reinforcement learning library","volume":"22","author":"Fujita","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1145\/2534169.2486031","article-title":"pfabric: Minimal near-optimal datacenter transport","volume":"43","author":"Alizadeh","year":"2013","journal-title":"ACM Spec. Interest Group Data Commun. (SIGCOMM) Comput. Commun. Rev."},{"key":"ref_35","unstructured":"Liu, J., Huang, J., Lv, W., and Wang, J. (2020). APS: Adaptive packet spraying to isolate mix-flows in data center network. IEEE Trans. Cloud Comput."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Cho, I., Jang, K., and Han, D. (2017, January 21\u201325). Credit-scheduled delay-bounded congestion control for datacenters. Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM), Los Angeles, CA, USA.","DOI":"10.1145\/3098822.3098840"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kodialam, M., Lakshman, T.V., Orlin, J.B., and Sengupta, S. (2006, January 23\u201329). A Versatile Scheme for Routing Highly Variable Traffic in Service Overlays and IP Backbones. Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), Barcelona, Spain.","DOI":"10.1109\/INFOCOM.2006.181"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1617","DOI":"10.1109\/COMST.2016.2532458","article-title":"Next generation 5G wireless networks: A comprehensive survey","volume":"18","author":"Agiwal","year":"2016","journal-title":"IEEE Commun. Surv. Tutor."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/2\/611\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:02:17Z","timestamp":1760364137000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/2\/611"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,13]]},"references-count":38,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["s22020611"],"URL":"https:\/\/doi.org\/10.3390\/s22020611","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,1,13]]}}}