{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,11]],"date-time":"2025-12-11T21:01:31Z","timestamp":1765486891536,"version":"build-2065373602"},"reference-count":35,"publisher":"Wiley","issue":"7","license":[{"start":{"date-parts":[[2023,4,7]],"date-time":"2023-04-07T00:00:00Z","timestamp":1680825600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100014718","name":"Innovative Research Group Project of the National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62274002","61874001","62004001"],"award-info":[{"award-number":["62274002","61874001","62004001"]}],"id":[{"id":"10.13039\/100014718","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003995","name":"Natural Science Foundation of Anhui Province","doi-asserted-by":"publisher","award":["2108085MF228"],"award-info":[{"award-number":["2108085MF228"]}],"id":[{"id":"10.13039\/501100003995","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Intelligent Systems"],"published-print":{"date-parts":[[2023,7]]},"abstract":"<jats:sec><jats:label\/><jats:p>A memristor\u2010based reinforcement learning (RL) system has shown outstanding performance in achieving efficient autonomous decision\u2010making and edge computing. Sarsa (<jats:italic>\u03bb<\/jats:italic>) is a classical multistep RL algorithm that records state with <jats:italic>\u03bb<\/jats:italic> decay and guides policy updates, significantly improving the algorithm convergence speed. However, <jats:italic>\u03bb<\/jats:italic> decay implementation of traditional computing hardware is confined by the extensive computation of power exponential decay. Herein, the value update equation for Sarsa (<jats:italic>\u03bb<\/jats:italic>) is implemented by using the topological structure of the memristor array, without complex circuits. Where, most importantly, the critical <jats:italic>\u03bb<\/jats:italic> decay function is realized by a TiO<jats:sub><jats:italic>x<\/jats:italic><\/jats:sub>\u2010based memristor with conductance decay property. The energy required for floating\u2010point operations can be significantly reduced while accelerating the convergence speed. Then, a path planning task is demonstrated based on intrinsic conductance decay property and shows outstanding performance. Finally, the information of rounds used for the task is obtained, which is based on the intrinsic decay property of the TiO<jats:sub><jats:italic>x<\/jats:italic><\/jats:sub>\u2010based memristor, maps into a 32\u2009\u00d7\u200932 memristor array in parallel to calculate the value of each round. The results indicate that the experimental data have similar results to the simulations. Herein, thus, it provides a hardware\u2010enabled scheme for the memristor\u2010based RL algorithm implementation.<\/jats:p><\/jats:sec>","DOI":"10.1002\/aisy.202200455","type":"journal-article","created":{"date-parts":[[2023,4,8]],"date-time":"2023-04-08T00:05:43Z","timestamp":1680912343000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Intrinsic Decay Property of Ti\/TiO<sub><i>x<\/i><\/sub>\/Pt Memristor for Reinforcement Learning"],"prefix":"10.1002","volume":"5","author":[{"given":"Yuehua","family":"Dai","sequence":"first","affiliation":[{"name":"School of Integrated Circuits Anhui University  Hefei Anhui 230601 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenbin","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Integrated Circuits Anhui University  Hefei Anhui 230601 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhe","family":"Feng","sequence":"additional","affiliation":[{"name":"School of Integrated Circuits Anhui University  Hefei Anhui 230601 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zuyu","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Integrated Circuits Anhui University  Hefei Anhui 230601 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yunlai","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Integrated Circuits Anhui University  Hefei Anhui 230601 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fei","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Integrated Circuits Anhui University  Hefei Anhui 230601 China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6465-5570","authenticated-orcid":false,"given":"Zuheng","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Integrated Circuits Anhui University  Hefei Anhui 230601 China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2023,4,7]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1002\/jeab.587"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.3758\/CABN.9.4.343"},{"key":"e_1_2_8_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2854283"},{"key":"e_1_2_8_5_1","doi-asserted-by":"publisher","DOI":"10.1002\/int.22375"},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-009-0191-x"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2018.07.015"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3038605"},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compchemeng.2019.05.029"},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2014.02.024"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/0303-2647(95)01551-5"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-015-0665-y"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2935255"},{"key":"e_1_2_8_14_1","first-page":"9949","volume":"31","author":"Lu T.","year":"2018","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.3390\/sym11091139"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2018.02.032"},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41928-017-0006-8"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-03453-y"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-04362-w"},{"key":"e_1_2_8_20_1","volume":"13","author":"Chang M.-F.","year":"2022","journal-title":"Nat. Commun."},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41578-022-00434-z"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/adma.202107811"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41928-020-00523-3"},{"key":"e_1_2_8_24_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41928-019-0221-6"},{"key":"e_1_2_8_25_1","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.abl5068"},{"key":"e_1_2_8_26_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abe6818"},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSI.2021.3137713"},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/pssa.201900938"},{"key":"e_1_2_8_29_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6463\/ac203c"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2018.2885317"},{"key":"e_1_2_8_31_1","doi-asserted-by":"publisher","DOI":"10.1002\/aelm.201800854"},{"key":"e_1_2_8_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TED.2012.2202319"},{"key":"e_1_2_8_33_1","unstructured":"Z.Wei T.Takagi Y.Kanzawa Y.Katoh T.Ninomiya K.Kawai S.Muraoka S.Mitani K.Katayama S.Fujii R.Miyanaga Y.Kawashima T.Mikawa K.Shimakawa K.Aono in4th IEEE Inter. Memory Workshop (IMW) IEEE Piscataway NJ2012.."},{"key":"e_1_2_8_34_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.210677"},{"key":"e_1_2_8_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40747-020-00138-3"},{"key":"e_1_2_8_36_1","volume":"6","author":"Lin H.","year":"2021","journal-title":"Adv. Mater. Technol."}],"container-title":["Advanced Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202200455","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,7]],"date-time":"2025-10-07T23:46:08Z","timestamp":1759880768000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202200455"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,7]]},"references-count":35,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2023,7]]}},"alternative-id":["10.1002\/aisy.202200455"],"URL":"https:\/\/doi.org\/10.1002\/aisy.202200455","archive":["Portico"],"relation":{},"ISSN":["2640-4567","2640-4567"],"issn-type":[{"type":"print","value":"2640-4567"},{"type":"electronic","value":"2640-4567"}],"subject":[],"published":{"date-parts":[[2023,4,7]]},"assertion":[{"value":"2022-12-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-04-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"2200455"}}