{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T08:05:46Z","timestamp":1782288346736,"version":"3.54.5"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,1,3]],"date-time":"2023-01-03T00:00:00Z","timestamp":1672704000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,1,3]],"date-time":"2023-01-03T00:00:00Z","timestamp":1672704000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52171341"],"award-info":[{"award-number":["52171341"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61902431"],"award-info":[{"award-number":["61902431"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61972417"],"award-info":[{"award-number":["61972417"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"technology project of Qingdao west coast new area under grant","award":["2020-84"],"award-info":[{"award-number":["2020-84"]}]},{"name":"the science foundation of Shandong province under grant","award":["ZR2020MF005"],"award-info":[{"award-number":["ZR2020MF005"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["EURASIP J. Adv. Signal Process."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Due to the time-varying and space-varying characteristics of the underwater acoustic channel, the communication process may be seriously disturbed. Thus, the underwater acoustic communication system is facing the challenges of alleviating interference and improving communication quality and communication efficiency through adaptive modulation. In order to select the optimal modulation mode adaptively and maximize the system throughput ensuring that the bit error rate (BER) meets the transmission requirements, this paper introduces deep reinforcement learning (DRL) into orthogonal frequency division multiplexing acoustic communication system. The adaptive modulation is mapped into a Markov decision process with unknown state transition probability. Thereby, the underwater communication channel environment is regarded as the state of DRL, and the modulation mode is regarded as action. The system returns channel state information (CSI) and signal\u2013noise ratio in every time slot through the feedback link. Because the Deep Q-Network optimizes in the changing state space of each time slot, it is suitable for a variety of different CSI. Finally, simulations in different underwater environments (SWellEx-96) show that the proposed adaptive modulation scheme can obtain lower BER and improve the system throughput effectively.<\/jats:p>","DOI":"10.1186\/s13634-022-00961-5","type":"journal-article","created":{"date-parts":[[2023,1,3]],"date-time":"2023-01-03T06:04:17Z","timestamp":1672725857000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Deep reinforcement learning-based adaptive modulation for OFDM underwater acoustic communication system"],"prefix":"10.1186","volume":"2023","author":[{"given":"Xuerong","family":"Cui","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7091-8592","authenticated-orcid":false,"given":"Peihao","family":"Yan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juan","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shibao","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianhang","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,1,3]]},"reference":[{"issue":"1","key":"961_CR1","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1109\/MCOM.2009.4752682","volume":"47","author":"M Stojanovic","year":"2009","unstructured":"M. Stojanovic, J. Preisig, Underwater acoustic communication channels: propagation models and statistical characterization. IEEE Commun. Mag. 47(1), 84\u201389 (2009)","journal-title":"IEEE Commun. Mag."},{"key":"961_CR2","doi-asserted-by":"crossref","unstructured":"M. Sadeghi, M. Elamassie, M. Uysal, Adaptive OFDM-based acoustic underwater transmission: system design and experimental verification, in Proceedings of the 2017 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom), F 5\u20138 June 2017 (2017)","DOI":"10.1109\/BlackSeaCom.2017.8277660"},{"key":"961_CR3","doi-asserted-by":"crossref","unstructured":"M. Huda, N.B. Putri, T.B. Santoso, OFDM system with adaptive modulation for shallow water acoustic channel environment, in Proceedings of the 2017 IEEE International Conference on Communication, Networks and Satellite (Comnetsat), F 5\u20137 Oct. 2017 (2017)","DOI":"10.1109\/COMNETSAT.2017.8263573"},{"issue":"2","key":"961_CR4","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1109\/JOE.2014.2323365","volume":"40","author":"L Wan","year":"2015","unstructured":"L. Wan, H. Zhou, X. Xu et al., Adaptive modulation and coding for underwater acoustic OFDM. IEEE J. Ocean. Eng. 40(2), 327\u2013336 (2015)","journal-title":"IEEE J. Ocean. Eng."},{"issue":"4","key":"961_CR5","doi-asserted-by":"publisher","first-page":"588","DOI":"10.1049\/iet-com.2019.0376","volume":"14","author":"MJ Bocus","year":"2020","unstructured":"M.J. Bocus, A. Doufexi, D. Agrafiotis, Performance of OFDM-based massive MIMO OTFS systems for underwater acoustic communication. IET Commun. 14(4), 588\u2013593 (2020)","journal-title":"IET Commun."},{"issue":"2","key":"961_CR6","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1109\/JOE.2013.2253212","volume":"39","author":"A Radosevic","year":"2014","unstructured":"A. Radosevic, R. Ahmed, T.M. Duman et al., Adaptive OFDM modulation for underwater acoustic communications: design considerations and experimental results. IEEE J. Ocean. Eng. 39(2), 357\u2013370 (2014)","journal-title":"IEEE J. Ocean. Eng."},{"key":"961_CR7","doi-asserted-by":"publisher","first-page":"76340","DOI":"10.1109\/ACCESS.2021.3082766","volume":"9","author":"S Mangione","year":"2021","unstructured":"S. Mangione, G.E. Galioto, D. Croce et al., A channel-aware adaptive modem for underwater acoustic communications. IEEE Access 9, 76340\u201376353 (2021)","journal-title":"IEEE Access"},{"issue":"11","key":"961_CR8","doi-asserted-by":"publisher","first-page":"7655","DOI":"10.1109\/TWC.2021.3087191","volume":"20","author":"P Jiang","year":"2021","unstructured":"P. Jiang, T. Wang, B. Han et al., AI-aided online adaptive OFDM receiver: design and experimental results. IEEE Trans. Wirel. Commun. 20(11), 7655\u20137668 (2021)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"4","key":"961_CR9","doi-asserted-by":"publisher","first-page":"2816","DOI":"10.1109\/JIOT.2019.2962915","volume":"7","author":"H Wang","year":"2020","unstructured":"H. Wang, Y. Li, J. Qian, Self-adaptive resource allocation in underwater acoustic interference channel: a reinforcement learning approach. IEEE Internet Things J. 7(4), 2816\u20132827 (2020)","journal-title":"IEEE Internet Things J."},{"key":"961_CR10","doi-asserted-by":"crossref","unstructured":"Y. Mahmutoglu, K. Turk, E. Tugcu, Particle swarm optimization algorithm based decision feedback equalizer for underwater acoustic communication, in Proceedings of the 2016 39th International Conference on Telecommunications and Signal Processing (TSP), F 27\u201329 June 2016 (2016)","DOI":"10.1109\/TSP.2016.7760848"},{"key":"961_CR11","doi-asserted-by":"publisher","first-page":"108128","DOI":"10.1016\/j.apacoust.2021.108128","volume":"181","author":"Y Chen","year":"2021","unstructured":"Y. Chen, W. Yu, X. Sun et al., Environment-aware communication channel quality prediction for underwater acoustic transmissions: a machine learning method. Appl. Acoust. 181, 108128 (2021)","journal-title":"Appl. Acoust."},{"key":"961_CR12","doi-asserted-by":"publisher","first-page":"73957","DOI":"10.1109\/ACCESS.2020.2981973","volume":"8","author":"MSM Alamgir","year":"2020","unstructured":"M.S.M. Alamgir, M.N. Sultana, K. Chang, Link adaptation on an underwater communications network using machine learning algorithms: boosted regression tree approach. IEEE Access 8, 73957\u201373971 (2020)","journal-title":"IEEE Access"},{"issue":"10","key":"961_CR13","doi-asserted-by":"publisher","first-page":"6844","DOI":"10.1109\/TWC.2020.3006230","volume":"19","author":"J Huang","year":"2020","unstructured":"J. Huang, R. Diamant, Adaptive modulation for long-range underwater acoustic communication. IEEE Trans. Wirel. Commun. 19(10), 6844\u20136857 (2020)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"22","key":"961_CR14","doi-asserted-by":"publisher","first-page":"10881","DOI":"10.1109\/JSEN.2019.2932126","volume":"19","author":"Z Jin","year":"2019","unstructured":"Z. Jin, Q. Zhao, Y. Su, RCAR: a reinforcement-learning-based routing protocol for congestion-avoided underwater acoustic sensor networks. IEEE Sens. J. 19(22), 10881\u201310891 (2019)","journal-title":"IEEE Sens. J."},{"issue":"2","key":"961_CR15","doi-asserted-by":"publisher","first-page":"791","DOI":"10.1109\/JIOT.2020.3008178","volume":"8","author":"Y Su","year":"2021","unstructured":"Y. Su, M. Liwang, Z. Gao et al., Optimal cooperative relaying and power control for IoUT networks with reinforcement learning. IEEE Internet Things J. 8(2), 791\u2013801 (2021)","journal-title":"IEEE Internet Things J."},{"key":"961_CR16","doi-asserted-by":"publisher","first-page":"2541","DOI":"10.1109\/ACCESS.2017.2784239","volume":"6","author":"C Wang","year":"2018","unstructured":"C. Wang, Z. Wang, W. Sun et al., Reinforcement learning-based adaptive transmission in time-varying underwater acoustic channels. IEEE Access 6, 2541\u20132558 (2018)","journal-title":"IEEE Access"},{"key":"961_CR17","doi-asserted-by":"crossref","unstructured":"Q. Fu, A. Song, Adaptive modulation for underwater acoustic communications based on reinforcement learning, in Proceedings of the OCEANS 2018 MTS\/IEEE Charleston, F 22\u201325 Oct. 2018 (2018)","DOI":"10.1109\/OCEANS.2018.8604746"},{"key":"961_CR18","doi-asserted-by":"publisher","first-page":"67539","DOI":"10.1109\/ACCESS.2019.2918506","volume":"7","author":"W Su","year":"2019","unstructured":"W. Su, J. Lin, K. Chen et al., Reinforcement learning-based adaptive modulation and coding for efficient underwater communications. IEEE Access 7, 67539\u201367550 (2019)","journal-title":"IEEE Access"},{"issue":"4","key":"961_CR19","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1109\/JOE.2013.2278787","volume":"38","author":"P Qarabaqi","year":"2013","unstructured":"P. Qarabaqi, M. Stojanovic, Statistical characterization and computationally efficient modeling of a class of underwater acoustic communication channels. IEEE J. Ocean. Eng. 38(4), 701\u2013717 (2013)","journal-title":"IEEE J. Ocean. Eng."},{"key":"961_CR20","doi-asserted-by":"crossref","unstructured":"Z. Yi-Jia, Z. Lan-Yue, M. Jia-Xin, Modeling and estimation of the space-time varying channels, in Proceedings of the 2021 OES China Ocean Acoustics (COA), F 14\u201317 July 2021 (2021)","DOI":"10.1109\/COA50123.2021.9520079"},{"key":"961_CR21","doi-asserted-by":"crossref","first-page":"e4636","DOI":"10.1002\/ett.4636","volume":"33","author":"X Cui","year":"2022","unstructured":"X. Cui, P. Yan, J. Li et al., Timing estimation of multiple hyperbolic frequency-modulated signals based on multicarrier underwater acoustic communicatio006E. Trans. Emerg. Telecommun. Technol. 33, e4636 (2022)","journal-title":"Trans. Emerg. Telecommun. Technol."},{"issue":"1","key":"961_CR22","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1109\/TVT.2021.3130061","volume":"71","author":"X Feng","year":"2022","unstructured":"X. Feng, J.F. Wang, X.Y. Kuai et al., Message passing-based impulsive noise mitigation and channel estimation for underwater acoustic OFDM communications. IEEE Trans. Veh. Technol. 71(1), 611\u2013625 (2022)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"3","key":"961_CR23","doi-asserted-by":"publisher","first-page":"1708","DOI":"10.1109\/TSP.2009.2038424","volume":"58","author":"CR Berger","year":"2010","unstructured":"C.R. Berger, S. Zhou, J.C. Preisig et al., Sparse channel estimation for multicarrier underwater acoustic communication: from subspace methods to compressed sensing. IEEE Trans. Signal Process. 58(3), 1708\u20131721 (2010)","journal-title":"IEEE Trans. Signal Process."},{"issue":"9","key":"961_CR24","doi-asserted-by":"publisher","first-page":"6165","DOI":"10.1109\/TWC.2017.2720580","volume":"16","author":"P Chen","year":"2017","unstructured":"P. Chen, Y. Rong, S. Nordholm et al., Joint channel estimation and impulsive noise mitigation in underwater acoustic OFDM communication systems. IEEE Trans. Wirel. Commun. 16(9), 6165\u20136178 (2017)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"4","key":"961_CR25","doi-asserted-by":"publisher","first-page":"927","DOI":"10.1109\/JOE.2007.906409","volume":"32","author":"W Li","year":"2007","unstructured":"W. Li, J.C. Preisig, Estimation of rapidly time-varying sparse channels. IEEE J. Ocean. Eng. 32(4), 927\u2013939 (2007)","journal-title":"IEEE J. Ocean. Eng."},{"key":"961_CR26","doi-asserted-by":"crossref","unstructured":"S. Zhao, S. Yan, L. Xu, Doppler estimation based on HFM signal for underwater acoustic time-varying multipath channel, in Proceedings of the 2019 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), F 20\u201322 Sept. 2019 (2019)","DOI":"10.1109\/ICSPCC46631.2019.8960810"}],"container-title":["EURASIP Journal on Advances in Signal Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-022-00961-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13634-022-00961-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-022-00961-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,3]],"date-time":"2023-12-03T18:22:48Z","timestamp":1701627768000},"score":1,"resource":{"primary":{"URL":"https:\/\/asp-eurasipjournals.springeropen.com\/articles\/10.1186\/s13634-022-00961-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,3]]},"references-count":26,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["961"],"URL":"https:\/\/doi.org\/10.1186\/s13634-022-00961-5","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-1657231\/v1","asserted-by":"object"}]},"ISSN":["1687-6180"],"issn-type":[{"value":"1687-6180","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,3]]},"assertion":[{"value":"17 May 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 December 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 January 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"1"}}