{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T07:38:59Z","timestamp":1767339539265,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,1,23]],"date-time":"2024-01-23T00:00:00Z","timestamp":1705968000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"COMCAST Innovation Fund"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>We introduce a novel multipath data transport approach at the transport layer referred to as \u2018Deep Deterministic Policy Gradient for Multipath Performance-oriented Congestion Control\u2019 (DDPG-MPCC), which leverages deep reinforcement learning to enhance congestion management in multipath networks. Our method combines DDPG with online convex optimization to optimize fairness and performance in simultaneously challenging multipath internet congestion control scenarios. Through experiments by developing kernel implementation, we show how DDPG-MPCC performs compared to the state-of-the-art solutions.<\/jats:p>","DOI":"10.3390\/fi16020037","type":"journal-article","created":{"date-parts":[[2024,1,24]],"date-time":"2024-01-24T04:54:01Z","timestamp":1706072041000},"page":"37","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["DDPG-MPCC: An Experience Driven Multipath Performance Oriented Congestion Control"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5819-765X","authenticated-orcid":false,"given":"Shiva Raj","family":"Pokhrel","sequence":"first","affiliation":[{"name":"IoT Research Lab, Deakin University, Geelong 3220, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9699-9418","authenticated-orcid":false,"given":"Jonathan","family":"Kua","sequence":"additional","affiliation":[{"name":"IoT Research Lab, Deakin University, Geelong 3220, Australia"}]},{"given":"Deol","family":"Satish","sequence":"additional","affiliation":[{"name":"IoT Research Lab, Deakin University, Geelong 3220, Australia"}]},{"given":"Sebnem","family":"Ozer","sequence":"additional","affiliation":[{"name":"Comcast Corporation, Philadelphia, PA 19103, USA"}]},{"given":"Jeff","family":"Howe","sequence":"additional","affiliation":[{"name":"Comcast Corporation, Philadelphia, PA 19103, USA"}]},{"given":"Anwar","family":"Walid","sequence":"additional","affiliation":[{"name":"Amazon Science, New York, NY 10001, USA"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/3009824","article-title":"BBR: Congestion-based congestion control","volume":"60","author":"Cardwell","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_2","unstructured":"Dong, M., Li, Q., Zarchy, D., Godfrey, P.B., and Schapira, M. (2015, January 4\u20136). PCC: Re-architecting congestion control for consistent high performance. Proceedings of the 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15), Oakland, CA, USA."},{"key":"ref_3","unstructured":"Dong, M., Meng, T., Zarchy, D., Arslan, E., Gilad, Y., Godfrey, B., and Schapira, M. (2018, January 9\u201311). PCC Vivace: Online-Learning Congestion Control. Proceedings of the 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18), Renton, WA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Arun, V., and Balakrishnan, H. (2018, January 9\u201311). Copa: Practical Delay-Based Congestion Control for the Internet. Proceedings of the 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18), Renton, WA, USA.","DOI":"10.1145\/3232755.3232783"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gilad, T., Rozen-Schiff, N., Godfrey, P.B., Raiciu, C., and Schapira, M. (2020, January 1\u20134). MPCC: Online learning multipath transport. Proceedings of the 16th International Conference on emerging Networking EXperiments and Technologies, Barcelona, Spain.","DOI":"10.1145\/3386367.3433030"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ford, A., Raiciu, C., Handley, M.J., and Bonaventure, O. (2013). TCP Extensions for Multipath Operation with Multiple Addresses, IETF. RFC 6824.","DOI":"10.17487\/rfc6824"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"996","DOI":"10.1109\/TMC.2021.3085598","article-title":"Learning to Harness Bandwidth with Multipath Congestion Control and Scheduling","volume":"22","author":"Pokhrel","year":"2023","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"6088","DOI":"10.1109\/TVT.2019.2912851","article-title":"Mobility-Aware Multipath Communication for Unmanned Aerial Surveillance Systems","volume":"68","author":"Pokhrel","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Tian, H., Liao, X., Zeng, C., Sun, D., Zhang, J., and Chen, K. (2023). Efficient DRL-Based Congestion Control with Ultra-Low Overhead. IEEE\/ACM Trans. Netw., accepted for publication.","DOI":"10.1109\/TNET.2023.3330737"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1109\/LCOMM.2022.3144692","article-title":"End-to-End Congestion Control to Provide Deterministic Latency Over Internet","volume":"26","author":"Liu","year":"2022","journal-title":"IEEE Commun. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"10299","DOI":"10.1109\/JIOT.2021.3056466","article-title":"Multipath TCP Meets Transfer Learning: A Novel Edge-Based Learning for Industrial IoT","volume":"8","author":"Pokhrel","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_12","first-page":"11767","article-title":"Softmax deep double deterministic policy gradients","volume":"33","author":"Pan","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"762","DOI":"10.1109\/COMST.2022.3217613","article-title":"AI-driven Packet Forwarding with Programmable Data Plane: A Survey","volume":"25","author":"Quan","year":"2022","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_14","unstructured":"Wu, H., Alay, O., Brunstrom, A., Caso, G., and Ferlin, S. (2022). Falcon: Fast and accurate multipath scheduling using offline and online learning. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2621","DOI":"10.1109\/JSAC.2019.2933761","article-title":"SmartCC: A Reinforcement Learning Approach for Multipath TCP Congestion Control in Heterogeneous Networks","volume":"37","author":"Li","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1325","DOI":"10.1109\/JSAC.2019.2904358","article-title":"Experience-driven Congestion Control: When Multi-Path TCP Meets Deep Reinforcement Learning","volume":"37","author":"Xu","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Mai, T., Yao, H., Jing, Y., Xu, X., Wang, X., and Ji, Z. (2019, January 24\u201328). Self-learning Congestion Control of MPTCP in Satellites Communications. Proceedings of the 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC), Tangier, Morocco.","DOI":"10.1109\/IWCMC.2019.8766465"},{"key":"ref_18","unstructured":"Chung, J., Han, D., Kim, J., and Kim, C.K. (2017, January 13\u201316). Machine Learning Based Path Management for Mobile Devices Over MPTCP. Proceedings of the 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju, Republic of Korea."},{"key":"ref_19","unstructured":"Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 22\u201324). Deterministic Policy Gradient Algorithms. Proceedings of the 31st International Conference on Machine Learning, Beijing, China."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tiong, T., Saad, I., Teo, K.T.K., and Lago, H.B. (2020, January 28). Deep Reinforcement Learning with Robust Deep Deterministic Policy Gradient. Proceedings of the 2020 2nd International Conference on Electrical, Control and Instrumentation Engineering (ICECIE), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICECIE50279.2020.9309539"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6839","DOI":"10.1109\/TWC.2023.3246082","article-title":"EdAR: An Experience-Driven Multipath Scheduler for Seamless Handoff in Mobile Networks","volume":"22","author":"Han","year":"2023","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1879","DOI":"10.1109\/TNET.2022.3158678","article-title":"MuSher: An Agile Multipath-TCP Scheduler for Dual-Band 802.11ad\/ac Wireless LANs","volume":"30","author":"Aggarwal","year":"2022","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, C., Wang, H., Qian, F., Zheng, K., Wang, C., Mao, F., Guo, X., and Xu, C. (2023, January 2\u20136). Experience: A Three-Year Retrospective of Large-Scale Multipath Transport Deployment for Mobile Applications. Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, Madrid, Spain.","DOI":"10.1145\/3570361.3592506"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Song, C., Han, B., Ji, X., Li, Y., and Su, J. (2023). AI-driven Multipath Transmission: Empowering UAV-based Live Streaming. IEEE Netw., accepted for publication.","DOI":"10.1109\/MNET.2023.3321521"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"109470","DOI":"10.1016\/j.comnet.2022.109470","article-title":"Adaptive QoS-aware multipath congestion control for live streaming","volume":"220","author":"Ji","year":"2023","journal-title":"Comput. Netw."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"110047","DOI":"10.1016\/j.comnet.2023.110047","article-title":"SmartSBD: Smart shared bottleneck detection for efficient multipath congestion control over heterogeneous networks","volume":"237","author":"Dong","year":"2023","journal-title":"Comput. Netw."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1109\/TNET.2014.2379698","article-title":"Multipath TCP: Analysis, Design, and Implementation","volume":"24","author":"Peng","year":"2016","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1109\/TNET.2019.2911122","article-title":"Analysis and Design of a Latency Control Protocol for Multi-Path Data Delivery With Pre-Defined QoS Guarantees","volume":"27","author":"Chiariotti","year":"2019","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1651","DOI":"10.1109\/TNET.2013.2274462","article-title":"MPTCP is Not Pareto-optimal: Performance Issues and a Possible Solution","volume":"21","author":"Khalili","year":"2013","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Luo, J., Su, X., and Liu, B. (2019, January 7\u20139). A Reinforcement Learning Approach for Multipath TCP Data Scheduling. Proceedings of the 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.","DOI":"10.1109\/CCWC.2019.8666496"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1109\/TNET.2018.2884791","article-title":"Low-latency Scheduling in MPTCP","volume":"27","author":"Hurtig","year":"2018","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3100","DOI":"10.1109\/TMC.2017.2686379","article-title":"Low Delay Random Linear Coding and Scheduling over Multiple Interfaces","volume":"16","author":"Karzand","year":"2017","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lim, Y.S., Nahum, E.M., Towsley, D., and Gibbens, R.J. (2017, January 12\u201315). ECF: An MPTCP Path Scheduler to Manage Heterogeneous Paths. Proceedings of the 3th International Conference on emerging Networking Experiments and Technologies, Incheon, Republic of Korea.","DOI":"10.1145\/3143361.3143376"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1145\/2043164.2018467","article-title":"Improving datacenter performance and robustness with multipath TCP","volume":"41","author":"Raiciu","year":"2011","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"key":"ref_35","unstructured":"Walid, A., Peng, Q., Hwang, J., and Low, S. (2016). Internet Engineering Task Force, Internet-Draft Draft-Walid-Mptcp-Congestion-Control-04, IETF."},{"key":"ref_36","unstructured":"Goyal, P., Agarwal, A., Netravali, R., Alizadeh, M., and Balakrishnan, H. (2019). ABC: A Simple Explicit Congestion Control Protocol for Wireless Networks. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1109\/JSAC.2019.2904350","article-title":"Dynamic TCP Initial Windows and Congestion Control Schemes through Reinforcement Learning","volume":"37","author":"Nie","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1145\/3453953.3453968","article-title":"A Rent-Seeking Framework for Multipath TCP","volume":"48","author":"Pokhrel","year":"2021","journal-title":"ACM SIGMETRICS Perform. Eval. Rev."},{"key":"ref_39","unstructured":"Liao, B., Zhang, G., Diao, Z., and Xie, G. (2020, January 22\u201326). Precise and Adaptable: Leveraging Deep Reinforcement Learning for GAP-based Multipath Scheduler. Proceedings of the 2020 IFIP Networking Conference (Networking), Paris, France."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Silva, F., Togou, M.A., and Muntean, G.M. (2020, January 15\u201319). AVIRA: Enhanced Multipath for Content-aware Adaptive Virtual Reality. Proceedings of the 2020 International Wireless Communications and Mobile Computing (IWCMC), Limassol, Cyprus.","DOI":"10.1109\/IWCMC48107.2020.9148293"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/2\/37\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:47:59Z","timestamp":1760104079000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/2\/37"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,23]]},"references-count":40,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,2]]}},"alternative-id":["fi16020037"],"URL":"https:\/\/doi.org\/10.3390\/fi16020037","relation":{},"ISSN":["1999-5903"],"issn-type":[{"type":"electronic","value":"1999-5903"}],"subject":[],"published":{"date-parts":[[2024,1,23]]}}}