{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T12:59:32Z","timestamp":1740142772464,"version":"3.37.3"},"reference-count":33,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2022,10,23]],"date-time":"2022-10-23T00:00:00Z","timestamp":1666483200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61972166"],"award-info":[{"award-number":["61972166"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Research Fund for MinJiang Scholarship Distinguished Professor","award":["2018026"],"award-info":[{"award-number":["2018026"]}]},{"name":"Provincial Key Laboratory of Computer Vision and Machine Learning of FuJian Province","award":["201910"],"award-info":[{"award-number":["201910"]}]},{"name":"Start-up Fund in HuaQiao University","award":["50Y17066"],"award-info":[{"award-number":["50Y17066"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,12,14]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The spatial and temporal uncertainty caused by large propagation delays is a fundamental feature of Underwater Acoustic Networks (UWANs), which seriously affects the performance of the UWANs and also brings challenges to the design of MAC protocols. In this paper, we develop an adaptive MAC protocol based on deep reinforcement learning for UWANs, called ARL-MAC protocol, to intelligently allocate time slots for nodes. Firstly, we design a reward mechanism based on the idea of Time-Domain Interference Alignment (TDIA). We determine the reward according to the combination of the node action and the feedback corresponding to the action. Then, we propose a flexible training mechanism to deal with the ever-changing underwater environment, which improves the fairness of time slot allocation. In addition, we introduce the Deep Recurrent Q-Network (DRQN) algorithm to solve the partially observable information issue. Finally, we evaluate the ARL-MAC protocol with the different number of nodes and changing network environment. Simulation results reveal that the ARL-MAC protocol outperforms other MAC protocols for UWANs in terms of throughput, collision rate and service fairness.<\/jats:p>","DOI":"10.1093\/comjnl\/bxac145","type":"journal-article","created":{"date-parts":[[2022,10,24]],"date-time":"2022-10-24T13:15:57Z","timestamp":1666617357000},"page":"3015-3028","source":"Crossref","is-referenced-by-count":2,"title":["An Adaptive MAC Protocol Based on Time-Domain Interference Alignment for UWANs"],"prefix":"10.1093","volume":"66","author":[{"given":"Nan","family":"Zhao","sequence":"first","affiliation":[{"name":"Department of Computer Science and Technology, Dalian University of Technology , Dalian, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nianmin","family":"Yao","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Dalian University of Technology , Dalian, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhenguo","family":"Gao","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Huaqiao University , Xiamen, China"},{"name":"Provincial Key Laboratory of Computer Vision and Machine Learning of FuJian Province , Xiamen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2022,10,23]]},"reference":[{"key":"2023121811455067200_ref1","first-page":"795","article-title":"Underwater acoustic wireless sensor networks: advances and future trends in physical","volume":"14","author":"Climent","year":"2014","journal-title":"MAC and routing layers. Sensors"},{"key":"2023121811455067200_ref2","doi-asserted-by":"crossref","first-page":"13126","DOI":"10.1109\/JIOT.2021.3081918","article-title":"Localization algorithm for underwater sensor network: A review","volume":"8","author":"Luo","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"2023121811455067200_ref3","doi-asserted-by":"crossref","DOI":"10.1016\/j.jnca.2019.102446","article-title":"Survey on high reliability wireless communication for underwater sensor networks","volume":"148","author":"Li","year":"2019","journal-title":"Journal of Network and Computer Applications"},{"key":"2023121811455067200_ref4","doi-asserted-by":"crossref","DOI":"10.1002\/dac.3350","article-title":"A survey on deployment techniques, localization algorithms, and research challenges for underwater acoustic sensor networks","volume":"30","author":"Tuna","year":"2017","journal-title":"International Journal of Communication Systems"},{"key":"2023121811455067200_ref5","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1109\/COMST.2021.3134955","article-title":"Reliable data collection techniques in underwater wireless sensor networks: A survey","volume":"24","author":"Wei","year":"2021","journal-title":"IEEE Communications Surveys & Tutorials"},{"key":"2023121811455067200_ref6","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1109\/COMST.2017.2768802","article-title":"State-of-the-art medium access control (MAC) protocols for underwater acoustic networks: A survey based on a MAC reference model","volume":"20","author":"Jiang","year":"2017","journal-title":"IEEE Communications Surveys & Tutorials"},{"key":"2023121811455067200_ref7","doi-asserted-by":"crossref","first-page":"3247","DOI":"10.1109\/TITS.2020.2980422","article-title":"Volunteer assisted collaborative offloading and resource allocation in vehicular edge computing","volume":"22","author":"Zeng","year":"2020","journal-title":"IEEE Transactions on Intelligent Transportation Systems"},{"key":"2023121811455067200_ref8","doi-asserted-by":"crossref","first-page":"8271","DOI":"10.1109\/JIOT.2020.2989745","article-title":"Game-theoretic algorithm designs and analysis for interactions among contributors in mobile crowdsourcing with word of mouth","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"2023121811455067200_ref9","doi-asserted-by":"crossref","first-page":"10010","DOI":"10.1109\/JIOT.2020.2988697","article-title":"AUV-aided energy-efficient data collection in underwater acoustic sensor networks","volume":"7","author":"Zhuo","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"2023121811455067200_ref10","doi-asserted-by":"crossref","first-page":"12398","DOI":"10.1109\/JIOT.2021.3063462","article-title":"Medium access control under space-time coupling in underwater acoustic networks","volume":"8","author":"Chen","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"2023121811455067200_ref11","first-page":"156","article-title":"Throughput-efficient super-TDMA MAC transmission schedules in ad hoc linear underwater acoustic networks","volume":"42","author":"Lmai","year":"2016","journal-title":"IEEE Journal of Oceanic Engineering"},{"key":"2023121811455067200_ref12","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.adhoc.2016.05.003","article-title":"DAP-MAC: A delay-aware probability-based MAC protocol for underwater acoustic sensor networks","volume":"48","author":"Han","year":"2016","journal-title":"Ad Hoc Networks"},{"key":"2023121811455067200_ref13","doi-asserted-by":"crossref","first-page":"9798075","DOI":"10.1155\/2016\/9798075","article-title":"MHM: A multiple handshaking MAC protocol for underwater acoustic sensor networks","volume":"12","author":"Lin","year":"2016","journal-title":"International Journal of Distributed Sensor Networks"},{"key":"2023121811455067200_ref14","doi-asserted-by":"crossref","first-page":"3127","DOI":"10.1007\/s11276-021-02639-6","article-title":"Buffering_Slotted_ALOHA protocol for underwater acoustic sensor networks based on the slot status","volume":"27","author":"Khater","year":"2021","journal-title":"Wireless Networks"},{"key":"2023121811455067200_ref15","doi-asserted-by":"crossref","first-page":"12398","DOI":"10.1109\/JIOT.2021.3063462","article-title":"Medium access control under space-time coupling in underwater acoustic networks","volume":"8","author":"Chen","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"2023121811455067200_ref16","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1109\/JOE.2012.2203060","article-title":"Throughput of networks with large propagation delays","volume":"37","author":"Chitre","year":"2012","journal-title":"IEEE Journal of Oceanic Engineering"},{"key":"2023121811455067200_ref17","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1109\/JOE.2013.2293932","article-title":"Region of feasibility of interference alignment in underwater sensor networks","volume":"39","author":"Pandey","year":"2014","journal-title":"IEEE journal of oceanic Engineering"},{"volume-title":"Proceedings of the international conference on underwater networks & systems","year":"2014","author":"Zeng","key":"2023121811455067200_ref18"},{"volume-title":"Proceedings of the 10th International Conference on Underwater Networks & Systems","year":"2015","author":"Zhao","key":"2023121811455067200_ref19"},{"key":"2023121811455067200_ref20","doi-asserted-by":"crossref","first-page":"1625","DOI":"10.1109\/TMC.2020.3029844","article-title":"Deep reinforcement learning based MAC protocol for underwater acoustic networks","volume":"21","author":"Ye","year":"2022","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"2023121811455067200_ref21","doi-asserted-by":"crossref","first-page":"2584","DOI":"10.1109\/TWC.2015.2389220","article-title":"Collision tolerant and collision free packet scheduling for underwater acoustic localization","volume":"14","author":"Ramezani","year":"2015","journal-title":"IEEE Transactions on wireless Communications"},{"key":"2023121811455067200_ref22","doi-asserted-by":"crossref","first-page":"1480","DOI":"10.1109\/TWC.2016.2646682","article-title":"Leveraging the near-far effect for improved spatial-reuse scheduling in underwater acoustic networks","volume":"16","author":"Diamant","year":"2016","journal-title":"IEEE Transactions on Wireless Communications"},{"key":"2023121811455067200_ref23","doi-asserted-by":"crossref","first-page":"1131","DOI":"10.1109\/TCOMM.2017.2647940","article-title":"A distributed scheduling algorithm for underwater acoustic networks with large propagation delays","volume":"65","author":"Zeng","year":"2017","journal-title":"IEEE Transactions on Communications"},{"key":"2023121811455067200_ref24","doi-asserted-by":"crossref","first-page":"8990","DOI":"10.1109\/JIOT.2021.3056052","article-title":"Packet-level slot scheduling MAC protocol in underwater acoustic sensor networks","volume":"8","author":"Liu","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"2023121811455067200_ref25","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1109\/LNET.2019.2925685","article-title":"Linear TDA-MAC: Unsynchronized scheduling in linear underwater acoustic sensor networks","volume":"1","author":"Morozs","year":"2019","journal-title":"IEEE Networking Letters"},{"key":"2023121811455067200_ref26","doi-asserted-by":"crossref","first-page":"1058","DOI":"10.1007\/s12083-020-01058-2","article-title":"A message transmission scheduling algorithm based on time-domain interference alignment in UWANs","volume":"14","author":"Zhao","year":"2021","journal-title":"Peer-to-Peer Networking and Applications"},{"volume-title":"2016 IEEE Wireless Communications and Networking Conference","year":"2016","author":"Jiang","key":"2023121811455067200_ref27"},{"key":"2023121811455067200_ref28","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1109\/TCCN.2018.2809722","article-title":"Deep reinforcement learning for dynamic multichannel access in wireless networks","volume":"4","author":"Wang","year":"2018","journal-title":"IEEE Transactions on Cognitive Communications and Networking"},{"key":"2023121811455067200_ref29","doi-asserted-by":"crossref","first-page":"2239","DOI":"10.1109\/JSAC.2019.2933973","article-title":"Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks","volume":"37","author":"Nasir","year":"2019","journal-title":"IEEE Journal on Selected Areas in Communications"},{"key":"2023121811455067200_ref30","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1109\/TWC.2018.2879433","article-title":"Deep multi-user reinforcement learning for distributed dynamic spectrum access","volume":"18","author":"Naparstek","year":"2018","journal-title":"IEEE transactions on wireless communications"},{"key":"2023121811455067200_ref31","doi-asserted-by":"crossref","first-page":"48742","DOI":"10.1109\/ACCESS.2021.3068407","article-title":"A time-slotted data gathering medium access control protocol using Q-learning for underwater acoustic sensor networks","volume":"9","author":"Ahmed","year":"2021","journal-title":"IEEE Access"},{"volume-title":"2014 AAAI Spring Symposium Series","year":"2014","author":"Dewey","key":"2023121811455067200_ref32"},{"key":"2023121811455067200_ref33","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1007\/s00779-018-1136-3","article-title":"Node scheduling problem in underwater acoustic sensor network using genetic algorithm","volume":"22","author":"Sivakumar","year":"2018","journal-title":"Personal and Ubiquitous Computing"}],"container-title":["The Computer Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/66\/12\/3015\/54446572\/bxac145.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/66\/12\/3015\/54446572\/bxac145.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,18]],"date-time":"2023-12-18T11:46:48Z","timestamp":1702900008000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/comjnl\/article\/66\/12\/3015\/6770085"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,23]]},"references-count":33,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,10,23]]},"published-print":{"date-parts":[[2023,12,14]]}},"URL":"https:\/\/doi.org\/10.1093\/comjnl\/bxac145","relation":{},"ISSN":["0010-4620","1460-2067"],"issn-type":[{"type":"print","value":"0010-4620"},{"type":"electronic","value":"1460-2067"}],"subject":[],"published-other":{"date-parts":[[2023,12]]},"published":{"date-parts":[[2022,10,23]]}}}