{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T05:32:06Z","timestamp":1768800726222,"version":"3.49.0"},"reference-count":30,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T00:00:00Z","timestamp":1675123200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62071144"],"award-info":[{"award-number":["62071144"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Aiming at the existing Direction of Arrival (DOA) methods based on neural network, a large number of samples are required to achieve signal-scene adaptation and accurate angle estimation. In the coherent signal environment, the problems of a larger amount of training sample data are required. In this paper, the DOA of coherent signal is converted into the DOA parameter estimation of the angle interval of incident signal. The accurate estimation of coherent DOA under the condition of small samples based on meta\u2212reinforcement learning (MRL) is realized. The meta\u2212reinforcement learning method in this paper models the process of angle interval estimation of coherent signals as a Markov decision process. In the inner loop layer, the sequence to sequence (S2S) neural network is used to express the angular interval feature sequence of the incident signal DOA. The strategy learning of the existence of angle interval under small samples is realized through making full use of the context relevance of spatial spectral sequence through S2S neural network. Thus, according to the optimal strategy, the output sequence is sequentially determined to give the angle interval of the incident signal. Finally, DOA is obtained through one-dimensional spectral peak search according to the angle interval obtained. The experiment shows that the meta\u2212reinforcement learning algorithm based on S2S neural network can quickly converge to the optimal state by only updating the gradient of S2S neural network parameters with a small sample set when a new signal environment appears.<\/jats:p>","DOI":"10.3390\/s23031546","type":"journal-article","created":{"date-parts":[[2023,2,1]],"date-time":"2023-02-01T01:36:59Z","timestamp":1675215419000},"page":"1546","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Small Sample Coherent DOA Estimation Method Based on S2S Neural Network Meta Reinforcement Learning"],"prefix":"10.3390","volume":"23","author":[{"given":"Zihan","family":"Wu","sequence":"first","affiliation":[{"name":"School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China"},{"name":"School of Information Science and Engineering, Harbin Institute of Technology (Weihai), Weihai 264200, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jun","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China"},{"name":"School of Information Science and Engineering, Harbin Institute of Technology (Weihai), Weihai 264200, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1109\/TAP.1986.1143830","article-title":"Multiple emitter location and signal parameter estimation","volume":"34","author":"Schmidt","year":"1986","journal-title":"IEEE Trans. Antennas Propag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1049\/ip-f-1.1985.0038","article-title":"Performance evaluation of the modified FBLP method for angle of arrival estimation using real radar multipath data","volume":"132","author":"Haykin","year":"1985","journal-title":"IEE Proc. F Commun. Radar Signal Process."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1109\/TASSP.1985.1164649","article-title":"On spatial smoothing for direction-of-arrival estimation of coherent signals","volume":"33","author":"Shan","year":"1985","journal-title":"IEEE Trans. Acoust. Speech Signal Process."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1109\/78.193218","article-title":"Weighted subspace methods and spatial smoothing: Analysis and comparison","volume":"41","author":"Rao","year":"1993","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"984","DOI":"10.1109\/29.32276","article-title":"ESPRIT-estimation of signal parameters via rotational invariance techniques","volume":"37","author":"Roy","year":"1989","journal-title":"IEEE Trans. Acoust. Speech Signal Process."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1109\/TASSP.1985.1164583","article-title":"Adaptive beamforming for coherent signals and interference","volume":"33","author":"Shan","year":"1985","journal-title":"IEEE Trans. Acoust. Speech Signal Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2900","DOI":"10.1109\/TSP.2002.804075","article-title":"On conditions for the rank restoration in forward backward spatial smoothing","volume":"50","author":"Choi","year":"2002","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1049\/ip-rsn:20020483","article-title":"Subspace-based coherent source localization with forward\/backward covariance matrices","volume":"149","author":"Choi","year":"2002","journal-title":"IEE Proc. Radar Sonar Navig."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1208","DOI":"10.1109\/78.80975","article-title":"Improved Spatial Smoothing Techniques for DOA Estimation of Coherent Signals","volume":"39","author":"Du","year":"1991","journal-title":"IEEE Trans Signal Process."},{"key":"ref_10","first-page":"345","article-title":"One-vs-One Multiclass Least Squares Support Vector Machines for Direction of Arrival Estimation","volume":"18","author":"Rohwer","year":"2003","journal-title":"Appl. Comput. Electromagn. Soc. J."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1109\/APS.2004.1329637","article-title":"The use of machine learning in smart antennas","volume":"1","author":"Christodoulou","year":"2004","journal-title":"IEEE Antennas Propag. Soc. Symp."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2279","DOI":"10.1109\/TAP.2009.2024485","article-title":"An Innovative Multiresolution Approach for DOA Estimation Based on a Support Vector Classification","volume":"57","author":"Donelli","year":"2009","journal-title":"IEEE Trans. Antennas Propag."},{"key":"ref_13","first-page":"2279","article-title":"DOA estimation based on support vector machine-Large scale multiclass classification problem","volume":"57","author":"Du","year":"2011","journal-title":"IEEE Int. Conf. Signal Process. Commun. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1450","DOI":"10.1109\/LSP.2021.3096117","article-title":"Unsupervised Learning Strategy for Direction-of-arrival Estimation Network","volume":"28","author":"Yuan","year":"2021","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"7315","DOI":"10.1109\/TAP.2018.2874430","article-title":"Direction-of-arrival estimation based on deep neural networks with robustness to array imperfections","volume":"66","author":"Liu","year":"2018","journal-title":"IEEE Trans. Antennas Propag."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1688","DOI":"10.1109\/LSP.2019.2945115","article-title":"Deep convolution network for direction of arrival estimation with sparse prior","volume":"26","author":"Wu","year":"2019","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"5068","DOI":"10.1109\/TVT.2020.2977894","article-title":"Improved de-multipath neural network models with self-paced feature-to-feature learning for DOA estimation in multipath environment","volume":"69","author":"Xiang","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"82329","DOI":"10.1109\/ACCESS.2019.2924156","article-title":"A novel phase enhancement method for low-angle estimation based on supervised DNN learning","volume":"7","author":"Xiang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2296","DOI":"10.3390\/s20082296","article-title":"A-CRNN-Based Method for Coherent DOA Estimation with Unknown Source Number","volume":"20","author":"Yao","year":"2020","journal-title":"Sensors"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1118","DOI":"10.1109\/TSP.2022.3144033","article-title":"Deep Learning-Aided Coherent Direction-of-Arrival Estimation With the FTMR Algorithm","volume":"70","author":"Hoang","year":"2022","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Merkofer, J.P., Revach, G., Shlezinger, N., and van Sloun, R.J.G. (2022, January 22\u201327). Deep Augmented Music Algorithm for Data-Driven Doa Estimation. Proceedings of the ICASSP 2022\u20132022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.","DOI":"10.1109\/ICASSP43922.2022.9746637"},{"key":"ref_22","unstructured":"Houhong, X., Meibin, Q., Baixiao, C., and Zhuang, S. (2022). Signal separation and super-resolution DOA estimation based on multi-objective joint learning. Appl. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1109\/LCOMM.2020.3032733","article-title":"Angle separation learning for coherent DOA estimation with deep sparse prior","volume":"25","author":"Xiang","year":"2021","journal-title":"IEEE Commun. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5216","DOI":"10.1109\/JIOT.2021.3110138","article-title":"Average AoI Minimization in UAV-Assisted Data Collection With RF Wireless Power Transfer: A Deep Reinforcement Learning Scheme","volume":"9","author":"Liu","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"85350","DOI":"10.1109\/ACCESS.2021.3088124","article-title":"Task Offloading and Resource Scheduling in Hybrid Edge-Cloud Networks","volume":"9","author":"Zhang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1109\/TPDS.2020.3014896","article-title":"Fast Adaptive Task Offloading in Edge Computing Based on Meta Reinforcement Learning","volume":"32","author":"Wang","year":"2021","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/j.csl.2016.12.003","article-title":"Interactive neural machine translation","volume":"45","author":"Miguel","year":"2017","journal-title":"Comput. Speech Lang."},{"key":"ref_28","unstructured":"Nichol, A., Ansari, J., and Schulman, J. (2018). On first-order meta\u2212learning algorithms. arxiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2349","DOI":"10.1007\/s13042-021-01316-6","article-title":"Model-agnostic multi-stage loss optimization meta learning","volume":"12","author":"Yao","year":"2021","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kim, K.-S., and Choi, Y.-S. (2021). HyAdamC: A New Adam-Based Hybrid Optimization Algorithm for Convolution Neural Networks. Sensors, 21.","DOI":"10.3390\/s21124054"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1546\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:19:35Z","timestamp":1760120375000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1546"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,31]]},"references-count":30,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23031546"],"URL":"https:\/\/doi.org\/10.3390\/s23031546","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,31]]}}}