{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T20:03:42Z","timestamp":1770494622984,"version":"3.49.0"},"reference-count":50,"publisher":"Wiley","issue":"6","license":[{"start":{"date-parts":[[2024,3,19]],"date-time":"2024-03-19T00:00:00Z","timestamp":1710806400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002551","name":"Seoul National University","doi-asserted-by":"publisher","award":["RS\u20102023\u201000257003"],"award-info":[{"award-number":["RS\u20102023\u201000257003"]}],"id":[{"id":"10.13039\/501100002551","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Intelligent Systems"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:p>Reinforcement learning (RL), exhibiting outstanding performance in various fields, requires large amounts of data for high performance. While exploration techniques address this requirement, conventional exploration methods have limitations: complexity of hardware implementation and significant hardware burden. Herein, in\u2010memory RL systems leveraging intrinsic 1\/<jats:italic>f<\/jats:italic> noise of synaptic ferroelectric field\u2010effect\u2010transistors (FeFETs) for efficient exploration are proposed. The electrical characteristics of fabricated FeFETs with low\u2010power operation capability verify their suitability for neuromorphic systems. The proposed system achieves comparable performance to the conventional exploration method without additional circuits. The intrinsic 1\/<jats:italic>f<\/jats:italic> noise of the FeFETs facilitates efficient exploration and offers significant advantages: efficiency in hardware implementation and simplicity in adjusting the 1\/<jats:italic>f<\/jats:italic> noise level for optimal performance. This approach effectively addresses the challenges of conventional exploration methods. The operation mechanism of the exploration method utilizing the 1\/<jats:italic>f<\/jats:italic> noise is systematically analyzed. The proposed in\u2010memory RL system demonstrates robustness and reliability to the device\u2010to\u2010device variation and the initial conductance distribution. This work provides further insights into the exploration methods of RL, paving the way for advanced in\u2010memory RL systems.<\/jats:p>","DOI":"10.1002\/aisy.202300763","type":"journal-article","created":{"date-parts":[[2024,3,19]],"date-time":"2024-03-19T15:13:51Z","timestamp":1710861231000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Toward Optimized In\u2010Memory Reinforcement Learning: Leveraging 1\/<i>f<\/i> Noise of Synaptic Ferroelectric Field\u2010Effect\u2010Transistors for Efficient Exploration"],"prefix":"10.1002","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4519-135X","authenticated-orcid":false,"given":"Jangsaeng","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering Inter\u2010University Semiconductor Research Center Seoul National University  Seoul 08826 Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9122-2458","authenticated-orcid":false,"given":"Wonjun","family":"Shin","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering Inter\u2010University Semiconductor Research Center Seoul National University  Seoul 08826 Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1657-3837","authenticated-orcid":false,"given":"Jiyong","family":"Yim","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering Hanyang University  Seoul 04763 Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7676-8938","authenticated-orcid":false,"given":"Dongseok","family":"Kwon","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering Inter\u2010University Semiconductor Research Center Seoul National University  Seoul 08826 Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daewoong","family":"Kwon","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering Hanyang University  Seoul 04763 Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3559-9802","authenticated-orcid":false,"given":"Jong\u2010Ho","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering Inter\u2010University Semiconductor Research Center Seoul National University  Seoul 08826 Republic of Korea"},{"name":"Ministry of Science and ICT  Sejong 30121 Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2024,3,19]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature24270"},{"key":"e_1_2_8_4_1","author":"Mnih V.","year":"2013","journal-title":"arXiv:1312.5602"},{"key":"e_1_2_8_5_1","unstructured":"Z.Wang T.Schaul M.Hessel H.Hasselt M.Lanctot N.Freitas inInt. Conf. Machine Learning International Conference on Machine Learning (ICML) New York NY2016."},{"key":"e_1_2_8_6_1","unstructured":"H.Van Hasselt A.Guez D.Silver inProc. of the AAAI Conf. on Artificial Intelligence AAAI Press Palo Alto CA2016."},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2022.03.003"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00056-4"},{"key":"e_1_2_8_9_1","author":"Amin S.","year":"2021","journal-title":"arXiv:2109.00157"},{"key":"e_1_2_8_10_1","author":"Wu N.","year":"2020","journal-title":"arXiv:2001.06930"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2016.09.141"},{"key":"e_1_2_8_12_1","author":"Gao B.","year":"2017","journal-title":"arXiv:1704.00805"},{"key":"e_1_2_8_13_1","author":"Plappert M.","year":"2017","journal-title":"arXiv:1706.01905"},{"key":"e_1_2_8_14_1","author":"Fortunato M.","year":"2017","journal-title":"arXiv:1706.10295"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.117343"},{"key":"e_1_2_8_16_1","unstructured":"R.Berdan T.Marukame S.Kabuyanagi K.Ota M.Saitoh S.Fujii J.Deguchi Y.Nishi inIEEE Symp. on VLSI Technology IEEE Kyoto Japan2019."},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41928-020-00523-3"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4020-5910-0"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-021-05699-z"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.11.016"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41928-018-0092-2"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1088\/0957-4484\/24\/38\/382001"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2015.2444094"},{"key":"e_1_2_8_24_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202000040"},{"key":"e_1_2_8_25_1","unstructured":"S. A.McKee inProc. of the 1st Conf. on Computing Frontiers Association for Computing Machinery New York NY2004."},{"key":"e_1_2_8_26_1","doi-asserted-by":"publisher","DOI":"10.1002\/advs.202207661"},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/LED.2021.3119281"},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/LED.2022.3176238"},{"key":"e_1_2_8_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TED.2015.2513744"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/LED.2022.3147259"},{"key":"e_1_2_8_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/LED.2021.3127175"},{"key":"e_1_2_8_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TED.2015.2475598"},{"key":"e_1_2_8_33_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6641\/ac251c"},{"key":"e_1_2_8_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2018.11.014"},{"key":"e_1_2_8_35_1","unstructured":"P. U.Diehl D.Neil J.Binas M.Cook S.\u2010C.Liu M.Pfeiffer inInt. Joint Conf. on Neural Networks IEEE Killarney Ireland2015."},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2017.00682"},{"key":"e_1_2_8_37_1","unstructured":"J.Kang Z.Yu L.Wu Y.Fang Z.Wang Y.Cai Z.Ji J.Zhang R.Wang Y.Yang R.Huang inIEEE Int. Electron Devices Meeting IEEE San Francisco CA2017."},{"key":"e_1_2_8_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/LED.2018.2869072"},{"key":"e_1_2_8_39_1","unstructured":"K.Ni A.Gupta O.Prakash S.Thomann X. S.Hu H.Amrouch inIEEE Int. Reliability Physics Symp. IEEE Dallas TX2020."},{"key":"e_1_2_8_40_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202100199"},{"key":"e_1_2_8_41_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-21057-x"},{"key":"e_1_2_8_42_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6641\/ac6ae0"},{"key":"e_1_2_8_43_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-51814-5"},{"key":"e_1_2_8_44_1","unstructured":"D.Niu Y.Chen C.Xu Y.Xie inProc. of the 47th Design Automation Conf. Association for Computing Machinery New York NY2010."},{"key":"e_1_2_8_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TED.2006.880165"},{"key":"e_1_2_8_46_1","unstructured":"F.Salehuddin I.Ahmad F. A.Hamid A.Zaharim A. M. A.Hamid P. S.Menon H. A.Elgomati B. Y.Majlis P. R.Apte inIEEE Int. Conf. Semiconductor Electronics IEEE Kuala Lumpur Malaysia2012."},{"key":"e_1_2_8_47_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-018-04485-1"},{"key":"e_1_2_8_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/JETCAS.2019.2933148"},{"key":"e_1_2_8_49_1","unstructured":"K.He X.Zhang S.Ren J.Sun inProc. of the IEEE Int. Conf. on Computer Vision (ICCV) IEEE Santiago Chile2015."},{"key":"e_1_2_8_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.sse.2019.107742"},{"key":"e_1_2_8_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-018-3659-y"}],"container-title":["Advanced Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202300763","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,7]],"date-time":"2025-10-07T23:05:41Z","timestamp":1759878341000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202300763"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,19]]},"references-count":50,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["10.1002\/aisy.202300763"],"URL":"https:\/\/doi.org\/10.1002\/aisy.202300763","archive":["Portico"],"relation":{},"ISSN":["2640-4567","2640-4567"],"issn-type":[{"value":"2640-4567","type":"print"},{"value":"2640-4567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,19]]},"assertion":[{"value":"2023-11-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"2300763"}}