{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T00:36:05Z","timestamp":1760229365291,"version":"build-2065373602"},"reference-count":30,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,6,9]],"date-time":"2022-06-09T00:00:00Z","timestamp":1654732800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["2020R1F1A1071649","2021R1F1A1063273","4199990113966"],"award-info":[{"award-number":["2020R1F1A1071649","2021R1F1A1063273","4199990113966"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Ministry of Education, Korea","award":["2020R1F1A1071649","2021R1F1A1063273","4199990113966"],"award-info":[{"award-number":["2020R1F1A1071649","2021R1F1A1063273","4199990113966"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper proposes a low-complexity algorithm for a reinforcement learning-based channel estimator for multiple-input multiple-output systems. The proposed channel estimator utilizes detected symbols to reduce the channel estimation error. However, the detected data symbols may include errors at the receiver owing to the characteristics of the wireless channels. Thus, the detected data symbols are selectively used as additional pilot symbols. To this end, a Markov decision process (MDP) problem is defined to optimize the selection of the detected data symbols. Subsequently, a reinforcement learning algorithm is developed to solve the MDP problem with computational efficiency. The developed algorithm derives the optimal policy in a closed form by introducing backup samples and data subblocks, to reduce latency and complexity. Simulations are conducted, and the results show that the proposed channel estimator significantly reduces the minimum-mean square error of the channel estimates, thus improving the block error rate compared to the conventional channel estimation.<\/jats:p>","DOI":"10.3390\/s22124379","type":"journal-article","created":{"date-parts":[[2022,6,13]],"date-time":"2022-06-13T02:01:44Z","timestamp":1655085704000},"page":"4379","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["A Low-Complexity Algorithm for a Reinforcement Learning-Based Channel Estimator for MIMO Systems"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9629-7413","authenticated-orcid":false,"given":"Tae-Kyoung","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Electronic Engineering, Gachon University, Seongnam 13120, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1206-3805","authenticated-orcid":false,"given":"Moonsik","family":"Min","sequence":"additional","affiliation":[{"name":"School of Electronics Engineering, Kyungpook National University, Daegu 41566, Korea"},{"name":"School of Electronic and Electrical Engineering, Kyungpook National University, Daegu 41566, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1002\/bltj.2015","article-title":"Layered Space-Time Architecture for Wireless Communication in a Fading Environment When Using Multi-Element Antennas","volume":"1","author":"Foschini","year":"1996","journal-title":"Bell Labs Tech. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1002\/ett.4460100604","article-title":"Capacity of Multi-Antenna Gaussian Channels","volume":"10","author":"Telatar","year":"1999","journal-title":"Eur. Trans. Telecommun."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1073","DOI":"10.1109\/TIT.2003.810646","article-title":"Diversity and Multiplexing: A Fundamental Tradeoff in Multiple-Antenna Channels","volume":"49","author":"Zheng","year":"2003","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1109\/TWC.2017.2768423","article-title":"Massive MIMO Has Unlimited Capacity","volume":"17","author":"Hoydis","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1109\/MCOM.2014.6736761","article-title":"Massive MIMO for Next Generation Wireless Systems","volume":"52","author":"Larsson","year":"2014","journal-title":"IEEE Commun. Mag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1109\/JSTSP.2014.2317671","article-title":"An Overview of Massive MIMO: Benefits and Challenges","volume":"8","author":"Lu","year":"2014","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1109\/TWC.2003.819022","article-title":"Pilot-based Channel Estimation for OFDM Systems by Tracking the Delay-Subspace","volume":"3","author":"Simeone","year":"2004","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3065","DOI":"10.1109\/78.969514","article-title":"A Comparison of Pilot-Aided Channel Estimation Methods for OFDM System","volume":"49","author":"Morelli","year":"2001","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1109\/JCN.2014.000075","article-title":"Frequency Domain Channel Estimation for MIMO SC-FDMA Systems with CDM Pilots","volume":"16","author":"Kim","year":"2014","journal-title":"J. Commun. Netw."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"884","DOI":"10.1109\/TSP.2005.863008","article-title":"Training-based MIMO Channel Estimation: A Study of Estimator Tradeoffs and Optimal Training Signals","volume":"54","author":"Biguesh","year":"2006","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/COMST.2007.382406","article-title":"Channel Estimation for Wireless OFDM Systems","volume":"9","author":"Ozdemir","year":"2007","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2905","DOI":"10.1109\/TSP.2018.2799164","article-title":"Learning the MMSE Channel Estimator","volume":"66","author":"Neumann","year":"2018","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_13","unstructured":"Dowler, A., Nix, A., and McGeehan, J. (2003, January 1\u20135). Data-derived Iterative Channel Estimation with Channel Tracking for a Mobile Fourth Generation Wide Area OFDM System. Proceedings of the IEEE Global Telecommunications Conference (GLOBECOM), San Francisco, CA, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Le, H.A., Van Chien, T., Nguyen, T.H., Choo, H., and Nguyen, V.D. (2021). Machine Learning-Based 5G-and-Beyond Channel Estimation for MIMO-OFDM Communication Systems. Sensors, 21.","DOI":"10.3390\/s21144861"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Naeem, M., De Pietro, G., and Coronato, A. (2022). Application of Reinforcement Learning and Deep Learning in Multiple-Input and Multiple-Output (MIMO) Systems. Sensors, 22.","DOI":"10.3390\/s22010309"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, X., Wang, Q., Yang, H., and Ma, X. (2022, January 10\u201313). Data-Aided MIMO Channel Estimation by Clustering and Reinforcement-Learning. Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC), Austin, TX, USA.","DOI":"10.1109\/WCNC51071.2022.9771693"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4921","DOI":"10.1109\/TCOMM.2021.3071537","article-title":"A Reduced Complexity Ungerboeck Receiver for Quantized Wideband Massive SC-MIMO","volume":"69","year":"2021","journal-title":"IEEE Trans. Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2960","DOI":"10.1109\/TWC.2020.2969627","article-title":"Machine Learning-Based Channel Prediction in Massive MIMO with Channel Aging","volume":"19","author":"Yuan","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3174","DOI":"10.1109\/TWC.2008.070228","article-title":"Iterative Turbo Channel Estimation for OFDM System over Rapid Dispersive Fading Channel","volume":"7","author":"Zhao","year":"2008","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3111","DOI":"10.1109\/TSP.2014.2321120","article-title":"Data-Aided Channel Estimation in Large Antenna Systems","volume":"62","author":"Ma","year":"2014","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3032","DOI":"10.1109\/TSP.2015.2416684","article-title":"Iterative Channel Estimation Using Virtual Pilot Signals for MIMO-OFDM Systems","volume":"63","author":"Park","year":"2015","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1109\/TSP.2018.2879620","article-title":"Iterative Channel Estimation Using LSE and Sparse Message Passing for mmWave MIMO Systems","volume":"67","author":"Huang","year":"2018","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2397","DOI":"10.1109\/TCOMM.2017.2688447","article-title":"Expectation-Maximization-based Channel Estimation for Multiuser MIMO Systems","volume":"65","author":"Park","year":"2017","journal-title":"IEEE Trans. Commun."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1697","DOI":"10.1109\/49.947034","article-title":"Iterative Channel Estimation and Decoding of Pilot Symbol Assisted Turbo Codes Over Flat-Fading Channels","volume":"19","author":"Valenti","year":"2001","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2885","DOI":"10.1109\/TSP.2004.834270","article-title":"Soft Input Channel Estimation for Turbo Equalization","volume":"52","author":"Song","year":"2004","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jeon, Y.S., Li, J., Tavangaran, N., and Poor, H.V. (2020, January 7\u201311). Data-Aided Channel Estimator for MIMO Systems via Reinforcement Learning. Proceedings of the IEEE International Conference on Communications (ICC), Dublin, Ireland.","DOI":"10.1109\/ICC40277.2020.9149283"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1663","DOI":"10.1109\/TWC.2019.2956044","article-title":"Robust Data Detection for MIMO Systems with One-Bit ADCs: A Reinforcement Learning Approach","volume":"19","author":"Jeon","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_28","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1403","DOI":"10.1109\/TSP.2004.826182","article-title":"Optimal Insertion of Pilot Symbols for Transmissions over Time-Varying Flat Fading Channels","volume":"52","author":"Dong","year":"2004","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"6999","DOI":"10.1109\/TVT.2021.3090087","article-title":"Training Length Adaptation for Reinforcement Learning-Based Detection in Time-Varying Massive MIMO Systems With One-Bit ADCs","volume":"70","author":"Kim","year":"2021","journal-title":"IEEE Trans. Veh. Technol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4379\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:27:06Z","timestamp":1760138826000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4379"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,9]]},"references-count":30,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["s22124379"],"URL":"https:\/\/doi.org\/10.3390\/s22124379","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,6,9]]}}}