{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,24]],"date-time":"2025-12-24T12:21:21Z","timestamp":1766578881157,"version":"3.37.3"},"reference-count":23,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T00:00:00Z","timestamp":1732579200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T00:00:00Z","timestamp":1732579200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"The National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["No.62276091","No.62276091","No.62276091"],"award-info":[{"award-number":["No.62276091","No.62276091","No.62276091"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Key Science and Technology Program of Henan Province","award":["No.232102210132","No.232102210132","No.232102210132"],"award-info":[{"award-number":["No.232102210132","No.232102210132","No.232102210132"]}]},{"name":"The Open Fund from Research Platform of Grain Information Processing Center in Henan University of Technology under Grant","award":["No.KFJJ-2021-104","No.KFJJ-2021-104","No.KFJJ-2021-104"],"award-info":[{"award-number":["No.KFJJ-2021-104","No.KFJJ-2021-104","No.KFJJ-2021-104"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Supercomput"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In wireless networks, the Contention Window (CW) is a crucial parameter for wireless channel sharing among numerous stations, directly influencing overall network performance. With the rapid growth and increasing diversity of traffic in wireless networks, differentiating and serving various types of traffic to enhance network performance has become a pressing issue that needs to be addressed. Deep Reinforcement Learning (DRL) optimizes decision-making through the interaction between an agent and its environment, enabling it to handle complex state spaces and high-dimensional data, making it particularly suitable for dynamically adjusting the contention window and adapting to environmental changes in real time. Access Category (AC) is used to differentiate various types of traffic to support different quality of service (QoS) requirements. Therefore, this paper proposes the integration of DRL techniques into the EDCA mechanism, utilizing AC to differentiate network traffic, while the DRL technology adjusts the contention window to adequately ensure QoS for different types of traffic and enhance overall network performance. This paper proposes a CW back off scheme based on DRL for differentiating ACs. The scheme uses DRL technology to observe the channel collision rate and sense the current network conditions, thereby adaptively adjusting the CW size for ACs. In addition, it dynamically adjusts the back off strategy according to the perceived current network conditions to optimize the data transmission process. In the range of 20\u2013120 stations, the scheme was tested in both single AC and multiple AC traffic scenarios, demonstrating excellent performance. The collision rate consistently remained below 18%, while the normalized throughput was maintained above 76%. Additionally, there is a significant improvement compared to existing deep reinforcement learning-based optimization schemes. Experimental results show that this scheme effectively discriminates between different ACs, resulting in lower latency and higher throughput for high-priority traffic. Furthermore, adaptively adjusting the CW size and improving the back off strategy maintains a low collision rate and stable throughput even under heavy load conditions, significantly improving the overall network performance.<\/jats:p>","DOI":"10.1007\/s11227-024-06634-4","type":"journal-article","created":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T06:01:11Z","timestamp":1732600871000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["PDCF-DRL: a contention window backoff scheme based on deep reinforcement learning for differentiating access categories"],"prefix":"10.1007","volume":"81","author":[{"given":"Zhibin","family":"Zuo","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Demin","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaowei","family":"Nie","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaoduo","family":"Pan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miaolei","family":"Deng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mimi","family":"Ma","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,11,26]]},"reference":[{"issue":"10","key":"6634_CR1","doi-asserted-by":"publisher","first-page":"293","DOI":"10.3390\/fi14100293","volume":"14","author":"E Mozaffariahrar","year":"2022","unstructured":"Mozaffariahrar E, Theoleyre F, Menth M (2022) A survey of Wi-Fi 6: technologies, advances, and challenges. Future Internet 14(10):293","journal-title":"Future Internet"},{"issue":"3","key":"6634_CR2","doi-asserted-by":"publisher","first-page":"1843","DOI":"10.1109\/COMST.2022.3179242","volume":"24","author":"S Szott","year":"2022","unstructured":"Szott S et al (2022) Wi-Fi meets ML: a survey on improving IEEE 802.11 performance with machine learning. IEEE Commun Surv Tutor 24(3):1843\u20131893","journal-title":"IEEE Commun Surv Tutor"},{"issue":"10","key":"6634_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3561381","volume":"55","author":"J Parmar","year":"2023","unstructured":"Parmar J et al (2023) Open-world machine learning: applications, challenges, and opportunities. ACM Comput Surv 55(10):1\u201337","journal-title":"ACM Comput Surv"},{"issue":"07","key":"6634_CR4","first-page":"3897","volume":"10","author":"K Sharifani","year":"2023","unstructured":"Sharifani K, Amini M (2023) Machine learning and deep learning: a review of methods and applications. World Inf Technol Eng J 10(07):3897\u20133904","journal-title":"World Inf Technol Eng J"},{"issue":"2","key":"6634_CR5","doi-asserted-by":"publisher","first-page":"1226","DOI":"10.1109\/COMST.2021.3063822","volume":"23","author":"A Feriani","year":"2021","unstructured":"Feriani A, Hossain E (2021) Single and multi-agent deep reinforcement learning for AI-enabled wireless networks: a tutorial. IEEE Commun Surv Tutor 23(2):1226\u20131252","journal-title":"IEEE Commun Surv Tutor"},{"key":"6634_CR6","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1109\/TMLCN.2023.3288090","volume":"1","author":"Y Huang","year":"2023","unstructured":"Huang Y, Chin K-W (2023) A three-tier deep learning-based channel access method for WiFi networks. IEEE Trans Mach Learn Commun Network 1:90\u2013106","journal-title":"IEEE Trans Mach Learn Commun Network"},{"issue":"4","key":"6634_CR7","doi-asserted-by":"publisher","first-page":"5064","DOI":"10.1109\/TNNLS.2022.3207346","volume":"35","author":"X Wang","year":"2022","unstructured":"Wang X et al (2022) Deep reinforcement learning: a survey. IEEE Trans Neural Netw Learn Syst 35(4):5064\u20135078","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"issue":"11","key":"6634_CR8","doi-asserted-by":"publisher","first-page":"13677","DOI":"10.1007\/s10489-022-04105-y","volume":"53","author":"A Oroojlooy","year":"2023","unstructured":"Oroojlooy A, Hajinezhad D (2023) A review of cooperative multi-agent deep reinforcement learning. Appl Intell 53(11):13677\u201313722","journal-title":"Appl Intell"},{"key":"6634_CR9","doi-asserted-by":"crossref","unstructured":"Majid A et al (2023) Deep reinforcement learning versus evolution strategies: a comparative survey. IEEE Trans Neural Netw Learn Syst","DOI":"10.1109\/TNNLS.2023.3264540"},{"key":"6634_CR10","doi-asserted-by":"crossref","unstructured":"Wydma\u0144ski W and Szott S (2021) Contention window optimization in IEEE 802.11 ax networks with deep reinforcement learning. In: 2021 IEEE Wireless Communications and Networking Conference (WCNC), pp 1\u20136","DOI":"10.1109\/WCNC49053.2021.9417575"},{"issue":"8","key":"6634_CR11","doi-asserted-by":"publisher","first-page":"1033","DOI":"10.1002\/dac.1210","volume":"24","author":"C-H Ke","year":"2011","unstructured":"Ke C-H et al (2011) A smart exponential-threshold-linear backoff mechanism for IEEE 802.11 WLANs. Int J Commun Syst 24(8):1033\u20131048","journal-title":"Int J Commun Syst"},{"issue":"1","key":"6634_CR12","first-page":"334","volume":"16","author":"C-H Ke","year":"2022","unstructured":"Ke C-H, Astuti L (2022) Applying deep reinforcement learning to improve throughput and reduce collision rate in IEEE 802.11 networks. KSII Trans Internet Inf Syst (TIIS) 16(1):334\u2013349","journal-title":"KSII Trans Internet Inf Syst (TIIS)"},{"issue":"13","key":"6634_CR13","doi-asserted-by":"publisher","first-page":"3002","DOI":"10.3390\/s19133002","volume":"19","author":"Y Cheng","year":"2019","unstructured":"Cheng Y et al (2019) Ca-CWA: channel-aware contention window adaption in IEEE 802.11 ah for soft real-time industrial applications. Sensors 19(13):3002","journal-title":"Sensors"},{"key":"6634_CR14","doi-asserted-by":"publisher","first-page":"168898","DOI":"10.1109\/ACCESS.2020.3023552","volume":"8","author":"A Krotov","year":"2020","unstructured":"Krotov A et al (2020) Rate control with spatial reuse for Wi-Fi 6 dense deployments. IEEE Access 8:168898\u2013168909","journal-title":"IEEE Access"},{"key":"6634_CR15","unstructured":"Sandholm T et al (2019) Learning to wait: Wi-Fi contention control using load-based predictions. arXiv preprint arXiv:1912.06747"},{"issue":"7","key":"6634_CR16","doi-asserted-by":"publisher","first-page":"5309","DOI":"10.1109\/JIOT.2020.3037774","volume":"8","author":"C Chen","year":"2020","unstructured":"Chen C et al (2020) Contention resolution in Wi-Fi 6-enabled Internet of Things based on deep learning. IEEE Internet Things J 8(7):5309\u20135320","journal-title":"IEEE Internet Things J"},{"key":"6634_CR17","doi-asserted-by":"publisher","first-page":"18402","DOI":"10.1109\/ACCESS.2021.3054015","volume":"9","author":"F Li","year":"2021","unstructured":"Li F et al (2021) Adaptive contention window MAC protocol in a global view for emerging trends networks. IEEE Access 9:18402\u201318423","journal-title":"IEEE Access"},{"key":"6634_CR18","doi-asserted-by":"publisher","first-page":"62880","DOI":"10.1109\/ACCESS.2021.3074812","volume":"9","author":"M Rasheed","year":"2021","unstructured":"Rasheed M et al (2021) ECM-MAC: an efficient collision mitigation strategy in contention-based MAC protocol. IEEE Access 9:62880\u201362889","journal-title":"IEEE Access"},{"key":"6634_CR19","doi-asserted-by":"crossref","unstructured":"Le H-C et al (2022) Efficient backoff priority-based medium access control mechanism for IoT sensor networks. Ann Comput Sci Inf Syst 33","DOI":"10.15439\/2022R24"},{"issue":"2","key":"6634_CR20","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1093\/comjnl\/bxaa147","volume":"64","author":"N Ranjan","year":"2021","unstructured":"Ranjan N (2021) Performance analysis of prioritization and contention control algorithm in wireless body area networks. Comput J 64(2):211\u2013223","journal-title":"Comput J"},{"key":"6634_CR21","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijleo.2022.170186","volume":"271","author":"K Hussain","year":"2022","unstructured":"Hussain K et al (2022) Starvation mitigation and priority aware of CSMA\/CA in WSN with implementing Markov chain model. Optik 271:170186","journal-title":"Optik"},{"key":"6634_CR22","doi-asserted-by":"crossref","unstructured":"Kwon J-H et al (2023) Reinforcement learning-based contention window adjustment for wireless body area networks. In: 2023 4th International Conference on Big Data Analytics and Practices (IBDAP), pp 1\u20134","DOI":"10.1109\/IBDAP58581.2023.10271988"},{"key":"6634_CR23","doi-asserted-by":"crossref","unstructured":"Malli M et al (2004) Adaptive fair channel allocation for QoS enhancement. In: 2004 IEEE International Conference on Communications (IEEE Cat. No. 04CH37577), vol 6, pp 3470\u20133475","DOI":"10.1109\/ICC.2004.1313189"}],"container-title":["The Journal of Supercomputing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11227-024-06634-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11227-024-06634-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11227-024-06634-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T08:32:55Z","timestamp":1732609975000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11227-024-06634-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,26]]},"references-count":23,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["6634"],"URL":"https:\/\/doi.org\/10.1007\/s11227-024-06634-4","relation":{},"ISSN":["0920-8542","1573-0484"],"issn-type":[{"type":"print","value":"0920-8542"},{"type":"electronic","value":"1573-0484"}],"subject":[],"published":{"date-parts":[[2024,11,26]]},"assertion":[{"value":"19 October 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 November 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"213"}}