{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T16:14:54Z","timestamp":1779898494414,"version":"3.53.1"},"reference-count":39,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,6,18]],"date-time":"2023-06-18T00:00:00Z","timestamp":1687046400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["2021R1F1A1063273"],"award-info":[{"award-number":["2021R1F1A1063273"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["2023R1A2C1004034"],"award-info":[{"award-number":["2023R1A2C1004034"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["4199990113966"],"award-info":[{"award-number":["4199990113966"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Ministry of Education, Korea","award":["2021R1F1A1063273"],"award-info":[{"award-number":["2021R1F1A1063273"]}]},{"name":"Ministry of Education, Korea","award":["2023R1A2C1004034"],"award-info":[{"award-number":["2023R1A2C1004034"]}]},{"name":"Ministry of Education, Korea","award":["4199990113966"],"award-info":[{"award-number":["4199990113966"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper proposes a reinforcement learning-aided channel estimator for time-varying multi-input multi-output systems. The basic concept of the proposed channel estimator is the selection of the detected data symbol in the data-aided channel estimation. To achieve the selection successfully, we first formulate an optimization problem to minimize the data-aided channel estimation error. However, in time-varying channels, the optimal solution is difficult to derive because of its computational complexity and the time-varying nature of the channel. To address these difficulties, we consider a sequential selection for the detected symbols and a refinement for the selected symbols. A Markov decision process is formulated for sequential selection, and a reinforcement learning algorithm that efficiently computes the optimal policy is proposed with state element refinement. Simulation results demonstrate that the proposed channel estimator outperforms conventional channel estimators by efficiently capturing the variation of the channels.<\/jats:p>","DOI":"10.3390\/s23125689","type":"journal-article","created":{"date-parts":[[2023,6,19]],"date-time":"2023-06-19T02:29:19Z","timestamp":1687141759000},"page":"5689","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Reinforcement Learning-Aided Channel Estimator in Time-Varying MIMO Systems"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9629-7413","authenticated-orcid":false,"given":"Tae-Kyoung","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Electronic Engineering, Gachon University, Seongnam 13120, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1206-3805","authenticated-orcid":false,"given":"Moonsik","family":"Min","sequence":"additional","affiliation":[{"name":"School of Electronic and Electrical Engineering, Kyungpook National University, Daegu 41566, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"684","DOI":"10.1109\/JSAC.2003.810294","article-title":"Capacity Limits of MIMO Channels","volume":"21","author":"Goldsmith","year":"2003","journal-title":"IEEE J. Sel. Commun."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1073","DOI":"10.1109\/TIT.2003.810646","article-title":"Diversity and Multiplexing: A Fundamental Tradeoff in Multiple-Antenna Channels","volume":"49","author":"Zheng","year":"2003","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1109\/JPROC.2003.821915","article-title":"An Overview of MIMO Communications-a Key to Gigabit Wireless","volume":"92","author":"Paulraj","year":"2004","journal-title":"Proc. IEEE"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1109\/MCOM.2004.1341263","article-title":"Antenna Selection in MIMO Systems","volume":"42","author":"Sanayei","year":"2004","journal-title":"IEEE Commun. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1109\/MCOM.2014.6736761","article-title":"Massive MIMO for Next Generation Wireless Systems","volume":"52","author":"Larsson","year":"2014","journal-title":"IEEE Commun. Mag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1738","DOI":"10.1109\/COMST.2015.2425294","article-title":"Survey of Large-Scale MIMO Systems","volume":"17","author":"Zheng","year":"2015","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1941","DOI":"10.1109\/COMST.2015.2475242","article-title":"Fifty Years of MIMO Detection: The Road to Large-Scale MIMOs","volume":"17","author":"Yang","year":"2015","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3065","DOI":"10.1109\/78.969514","article-title":"A Comparison of Pilot-Aided Channel Estimation Methods for OFDM System","volume":"49","author":"Morelli","year":"2001","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1109\/TBC.2002.804034","article-title":"Channel Estimation Techniques Based on Pilot Arrangement in OFDM Systems","volume":"48","author":"Coleri","year":"2002","journal-title":"IEEE Trans. Broadcast."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1109\/TWC.2004.840235","article-title":"ICI Mitigation for Pilot-Aided OFDM Mobile Systems","volume":"4","author":"Mostofi","year":"2005","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"884","DOI":"10.1109\/TSP.2005.863008","article-title":"Training-based MIMO Channel Estimation: A Study of Estimator Tradeoffs and Optimal Training Signals","volume":"54","author":"Biguesh","year":"2006","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/COMST.2007.382406","article-title":"Channel Estimation for Wireless OFDM Systems","volume":"9","author":"Ozdemir","year":"2007","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"652","DOI":"10.1109\/LCOMM.2019.2898944","article-title":"Deep Learning-based Channel Estimation","volume":"23","author":"Soltani","year":"2019","journal-title":"IEEE Commun. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Le, H.A., Van Chien, T., Nguyen, T.H., Choo, H., and Nguyen, V.D. (2021). Machine Learning-Based 5G-and-Beyond Channel Estimation for MIMO-OFDM Communication Systems. Sensors, 21.","DOI":"10.3390\/s21144861"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2960","DOI":"10.1109\/TWC.2020.2969627","article-title":"Machine Learning-Based Channel Prediction in Massive MIMO with Channel Aging","volume":"19","author":"Yuan","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1697","DOI":"10.1109\/49.947034","article-title":"Iterative Channel Estimation and Decoding of Pilot Symbol Assisted Turbo Codes over Flat-Fading Channels","volume":"19","author":"Valenti","year":"2001","journal-title":"IEEE J. Sel. Commun."},{"key":"ref_17","unstructured":"Dowler, A., Nix, A., and McGeehan, J. (2003, January 1\u20135). Data-derived Iterative Channel Estimation with Channel Tracking for a Mobile Fourth Generation Wide Area OFDM System. Proceedings of the IEEE Global Telecommunications Conference (GLOBECOM), San Francisco, CA, USA."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1266","DOI":"10.1109\/TCOMM.2003.815062","article-title":"Joint Channel Estimation and Data Detection in Space-Time Communications","volume":"51","author":"Cozzo","year":"2003","journal-title":"IEEE Trans. Commun."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2885","DOI":"10.1109\/TSP.2004.834270","article-title":"Soft Input Channel Estimation for Turbo Equalization","volume":"52","author":"Song","year":"2004","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2993","DOI":"10.1109\/TSP.2007.893979","article-title":"Soft-Iterative Channel Estimation: Methods and Performance Analysis","volume":"55","author":"Nicoli","year":"2007","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3174","DOI":"10.1109\/TWC.2008.070228","article-title":"Iterative Turbo Channel Estimation for OFDM System over Rapid Dispersive Fading Channel","volume":"7","author":"Zhao","year":"2008","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4340","DOI":"10.1109\/TWC.2009.081448","article-title":"A Low-Complexity Iterative Channel Estimation and Detection Technique for Doubly Selective Channels","volume":"8","author":"Guo","year":"2009","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3111","DOI":"10.1109\/TSP.2014.2321120","article-title":"Data-Aided Channel Estimation in Large Antenna Systems","volume":"62","author":"Ma","year":"2014","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2541","DOI":"10.1109\/TSP.2015.2508786","article-title":"Bayes-Optimal Joint Channel-and-Data Estimation for Massive MIMO with Low-Precision ADCs","volume":"64","author":"Wen","year":"2015","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3032","DOI":"10.1109\/TSP.2015.2416684","article-title":"Iterative Channel Estimation Using Virtual Pilot Signals for MIMO-OFDM Systems","volume":"63","author":"Park","year":"2015","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1109\/TSP.2018.2879620","article-title":"Iterative Channel Estimation Using LSE and Sparse Message Passing for mmWave MIMO Systems","volume":"67","author":"Huang","year":"2018","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Li, X., Wang, Q., Yang, H., and Ma, X. (2022, January 10\u201313). Data-Aided MIMO Channel Estimation by Clustering and Reinforcement-Learning. Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC), Austin, TX, USA.","DOI":"10.1109\/WCNC51071.2022.9771693"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Naeem, M., De Pietro, G., and Coronato, A. (2022). Application of Reinforcement Learning and Deep Learning in Multiple-Input and Multiple-Output (MIMO) Systems. Sensors, 22.","DOI":"10.3390\/s22010309"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Oh, M.S., Hosseinalipour, S., Kim, T., Brinton, C.G., and Love, D.J. (2021, January 14\u201323). Channel Estimation via Successive Denoising in MIMO OFDM Systems: A Reinforcement Learning Approach. Proceedings of the IEEE International Conference on Communications (ICC), Montreal, QC, Canada.","DOI":"10.1109\/ICC42927.2021.9500671"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"10271","DOI":"10.1109\/TWC.2022.3183255","article-title":"Deep Reinforcement Learning based End-to-End Multi-User Channel Prediction and Beamforming","volume":"21","author":"Chu","year":"2022","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Jeon, Y.S., Li, J., Tavangaran, N., and Poor, H.V. (2020, January 27\u201329). Data-Aided Channel Estimator for MIMO Systems via Reinforcement Learning. Proceedings of the IEEE International Conference on Communications (ICC), Prayagraj, India.","DOI":"10.1109\/ICC40277.2020.9149283"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kim, T.K., and Min, M. (2022). A Low-Complexity Algorithm for Reinforcement Learning-Based Channel Estimator for MIMO Systems. Sensors, 21.","DOI":"10.3390\/s22124379"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kim, T.K., Jeon, Y.S., Li, J., Tavangaran, N., and Poor, H.V. (IEEE Trans. Wirel. Commun., 2022). Semi-Data-Aided Channel Estimation for MIMO Systems via Reinforcement Learning, IEEE Trans. Wirel. Commun., early access.","DOI":"10.1109\/TWC.2022.3227312"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1403","DOI":"10.1109\/TSP.2004.826182","article-title":"Optimal Insertion of Pilot Symbols for Transmissions over Time-Varying Flat Fading Channels","volume":"52","author":"Dong","year":"2004","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"6999","DOI":"10.1109\/TVT.2021.3090087","article-title":"Training Length Adaptation for Reinforcement Learning-Based Detection in Time-Varying Massive MIMO Systems With One-Bit ADCs","volume":"70","author":"Kim","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3376","DOI":"10.1109\/TSP.2015.2421471","article-title":"Predictive Coding of Bit Loading for Time Correlated MIMO Channels with A Decision Feedback Receiver","volume":"63","author":"Li","year":"2015","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2349","DOI":"10.1109\/TVT.2014.2345655","article-title":"Limited Feedback for Multicell Zero-Forcing Coordinated Beamforming in Time-Varying Channels","volume":"64","author":"Kim","year":"2015","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1109\/WCL.2013.100913.130565","article-title":"A Differential Codebook with Adaptive Scaling for Limited Feedback MU MISO Systems","volume":"3","author":"Mirza","year":"2014","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_39","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/12\/5689\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:55:52Z","timestamp":1760126152000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/12\/5689"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,18]]},"references-count":39,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["s23125689"],"URL":"https:\/\/doi.org\/10.3390\/s23125689","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,18]]}}}