{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T14:21:52Z","timestamp":1753885312549,"version":"3.41.2"},"reference-count":27,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T00:00:00Z","timestamp":1619654400000},"content-version":"vor","delay-in-days":118,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002701","name":"Ministry of Education","doi-asserted-by":"publisher","award":["2019R1I1A1A01061824","2020R1C1C1007739","2020R1C1C1A01005229"],"award-info":[{"award-number":["2019R1I1A1A01061824","2020R1C1C1007739","2020R1C1C1A01005229"]}],"id":[{"id":"10.13039\/501100002701","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002641","name":"Konkuk University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002641","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Journal of Sensors"],"published-print":{"date-parts":[[2021,1]]},"abstract":"<jats:p>Experience replay memory in reinforcement learning enables agents to remember and reuse past experiences. Most of the reinforcement models are subject to single experience replay memory to operate agents. In this article, we propose a framework that accommodates doubly used experience replay memory, exploiting both important transitions and new transitions simultaneously. In numerical studies, the deep <jats:italic>Q<\/jats:italic>\u2010networks (DQN) equipped with double experience replay memory are examined under various scenarios. A self\u2010driving car requires an automated agent to figure out when to adequately change lanes on the real\u2010time basis. To this end, we apply our proposed agent to the simulation of urban mobility (SUMO) experiments. Besides, we also verify its applicability to reinforcement learning whose action space is discrete (e.g., computer game environments). Taken all together, we conclude that the proposed framework outperforms priorly known reinforcement learning models in the virtue of double experience replay memory.<\/jats:p>","DOI":"10.1155\/2021\/6652042","type":"journal-article","created":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T14:35:34Z","timestamp":1619706934000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Reinforcement Learning Guided by Double Replay Memory"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7251-5175","authenticated-orcid":false,"given":"Jiseong","family":"Han","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0543-2198","authenticated-orcid":false,"given":"Kichun","family":"Jo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7195-3349","authenticated-orcid":false,"given":"Wontaek","family":"Lim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3492-0839","authenticated-orcid":false,"given":"Yonghak","family":"Lee","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1666-6977","authenticated-orcid":false,"given":"Kyoungmin","family":"Ko","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4760-3058","authenticated-orcid":false,"given":"Eunseon","family":"Sim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8403-0834","authenticated-orcid":false,"given":"JunSang","family":"Cho","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0442-7795","authenticated-orcid":false,"given":"SungHwan","family":"Kim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2021,4,29]]},"reference":[{"key":"e_1_2_11_1_2","doi-asserted-by":"crossref","unstructured":"LaddhaA. KocamazM. K. Navarro-SermentL. E. andHebertM. Map supervised road detection 2016 IEEE Intelligent Vehicles Symposium (IV) 2016 Gothenburg Sweden 118\u2013123.","DOI":"10.1109\/IVS.2016.7535374"},{"key":"e_1_2_11_2_2","unstructured":"OpenStreetMap contributors Planet Dump 2017 https:\/\/planet.osm.org.https:\/\/www.openstreetmap.org."},{"key":"e_1_2_11_3_2","doi-asserted-by":"crossref","unstructured":"KocamazM. K. GongJ. andPiresB. R. Vision-based counting of pedestrians and cyclists 2016 IEEE winter conference on applications of computer vision (WACV) 2016 Lake Placid NY USA 1\u20138.","DOI":"10.1109\/WACV.2016.7477685"},{"key":"e_1_2_11_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2016.2597966"},{"key":"e_1_2_11_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2018.06.007"},{"key":"e_1_2_11_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/MITS.2017.2709782"},{"key":"e_1_2_11_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2018.2804891"},{"key":"e_1_2_11_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2017.2771144"},{"key":"e_1_2_11_9_2","doi-asserted-by":"publisher","DOI":"10.1080\/15472450.2013.810994"},{"key":"e_1_2_11_10_2","doi-asserted-by":"crossref","unstructured":"XuD. DingZ. ZhaoH. MozeM. AiounF. andGuillemardF. Naturalistic lane change analysis for human-like trajectory generation 2018 IEEE Intelligent Vehicles Symposium (IV) 2018 Changshu China 1393\u20131399.","DOI":"10.1109\/IVS.2018.8500690"},{"key":"e_1_2_11_11_2","doi-asserted-by":"crossref","unstructured":"JeongS.-G. KimJ. KimS. andMinJ. End-to-end learning of image based lane-change decision 2017 IEEE intelligent vehicles symposium (IV) 2017 Los Angeles CA USA 1602\u20131607.","DOI":"10.1109\/IVS.2017.7995938"},{"key":"e_1_2_11_12_2","unstructured":"FridmanL. TerwilligerJ. andJenikB. Deeptraffic: crowdsourced hyperparameter tuning of deep reinforcement learning systems for multi-agent dense traffic navigation 2018 https:\/\/arxiv.org\/abs\/1801.02805."},{"key":"e_1_2_11_13_2","doi-asserted-by":"crossref","unstructured":"WangP. ChanC.-Y. andde La FortelleA. A reinforcement learning based approach for automated lane change maneuvers 2018 IEEE Intelligent Vehicles Symposium (IV) 2018 Changshu China 1379\u20131384.","DOI":"10.1109\/IVS.2018.8500556"},{"key":"e_1_2_11_14_2","doi-asserted-by":"crossref","unstructured":"WolfP. KurzerK. WingertT. KuhntF. andZollnerJ. M. Adaptive behavior generation for autonomous driving using deep reinforcement learning with compact semantic states 2018 IEEE Intelligent Vehicles Symposium (IV) 2018 Changshu China 993\u20131000.","DOI":"10.1109\/IVS.2018.8500427"},{"key":"e_1_2_11_15_2","doi-asserted-by":"crossref","unstructured":"YouC. LuJ. FilevD. andTsiotrasP. Highway traffic modeling and decision making for autonomous vehicle using reinforcement learning 2018 IEEE Intelligent Vehicles Symposium (IV) 2018 Changshu China 1227\u20131232.","DOI":"10.1109\/IVS.2018.8500675"},{"key":"e_1_2_11_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.1998.712192"},{"key":"e_1_2_11_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_2_11_18_2","unstructured":"SchaulT. QuanJ. AntonoglouI. andSilverD. Prioritized experience replay 2015 https:\/\/arxiv.org\/abs\/1511.05952."},{"key":"e_1_2_11_19_2","doi-asserted-by":"crossref","unstructured":"HouY. LiuL. WeiQ. XuX. andChenC. A novel DDPG method with prioritized experience replay 2017 IEEE International Conference on Systems Man and Cybernetics (SMC) 2017 Banff AB Canada 316\u2013321.","DOI":"10.1109\/SMC.2017.8122622"},{"key":"e_1_2_11_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2956703"},{"key":"e_1_2_11_21_2","unstructured":"LiuZ. ZhouH. ChenB. ZhongS. HebertM. andZhaoD. Safe model-based reinforcement learning with robust cross-entropy method 2020 https:\/\/arxiv.org\/abs\/2010.07968."},{"key":"e_1_2_11_22_2","unstructured":"BrockmanG. CheungV. PetterssonL. SchneiderJ. SchulmanJ. TangJ. andZarembaW. Openai gym 2016 https:\/\/arxiv.org\/abs\/1606.01540."},{"key":"e_1_2_11_23_2","unstructured":"KingmaD. P.andBaJ. Adam: a method for stochastic optimization 2014 https:\/\/arxiv.org\/abs\/1412.6980."},{"key":"e_1_2_11_24_2","unstructured":"KrajzewiczD. HertkornG. RosselC. andWagnerP. Sumo (simulation of urban mobility)-an open-source traffic simulation Proceedings of the 4th Middle East Symposium on Simulation and Modelling 2002 183\u2013187."},{"key":"e_1_2_11_25_2","doi-asserted-by":"crossref","unstructured":"WegenerA. Pi\u00f3rkowskiM. RayaM. Hellbr\u00fcckH. FischerS. andHubauxJ. P. Traci: an interface for coupling road traffic and network simulators Proceedings of the 11th Communications and Networking Simulation Symposium (CNS\u203208) April 2008 Ottawa Canada 155\u2013163.","DOI":"10.1145\/1400713.1400740"},{"key":"e_1_2_11_26_2","doi-asserted-by":"crossref","unstructured":"IseleD.andCosgunA. Selective experience replay for lifelong learning The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) 2018.","DOI":"10.1609\/aaai.v32i1.11595"},{"key":"e_1_2_11_27_2","doi-asserted-by":"crossref","unstructured":"ZhaD. LaiK.-H. ZhouK. andHuX. Experience replay optimization 2019 https:\/\/arxiv.org\/abs\/1906.08387.","DOI":"10.24963\/ijcai.2019\/589"}],"container-title":["Journal of Sensors"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/js\/2021\/6652042.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/js\/2021\/6652042.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2021\/6652042","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T00:47:10Z","timestamp":1722905230000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/2021\/6652042"}},"subtitle":[],"editor":[{"given":"Ismail","family":"Butun","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2021,1]]},"references-count":27,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,1]]}},"alternative-id":["10.1155\/2021\/6652042"],"URL":"https:\/\/doi.org\/10.1155\/2021\/6652042","archive":["Portico"],"relation":{},"ISSN":["1687-725X","1687-7268"],"issn-type":[{"type":"print","value":"1687-725X"},{"type":"electronic","value":"1687-7268"}],"subject":[],"published":{"date-parts":[[2021,1]]},"assertion":[{"value":"2020-11-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-03-24","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"6652042"}}