{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T10:48:22Z","timestamp":1784630902871,"version":"3.55.0"},"reference-count":43,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,9,6]],"date-time":"2023-09-06T00:00:00Z","timestamp":1693958400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>In recent years, sensor components similar to human sensory functions have been rapidly developed in the hardware field, enabling the acquisition of information at a level beyond that of humans, and in the software field, artificial intelligence technology has been utilized to enable cognitive abilities and decision-making such as prediction, analysis, and judgment. These changes are being utilized in various industries and fields. In particular, new hardware and software technologies are being rapidly applied to robotics products, showing a level of performance and completeness that was previously unimaginable. In this paper, we researched the topic of establishing an optimal path plan for autonomous driving using LiDAR sensors and deep reinforcement learning in a workplace without map and grid coordinates for mobile robots, which are widely used in logistics and manufacturing sites. For this purpose, we reviewed the hardware configuration of mobile robots capable of autonomous driving, checked the characteristics of the main core sensors, and investigated the core technologies of autonomous driving. In addition, we reviewed the appropriate deep reinforcement learning algorithm to realize the autonomous driving of mobile robots, defined a deep neural network for autonomous driving data conversion, and defined a reward function for path planning. The contents investigated in this paper were built into a simulation environment to verify the autonomous path planning through experiment, and an additional reward technique \u201cVelocity Range-based Evaluation Method\u201d was proposed for further improvement of performance indicators required in the real field, and the effectiveness was verified. The simulation environment and detailed results of experiments are described in this paper, and it is expected as guidance and reference research for applying these technologies in the field.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1210442","type":"journal-article","created":{"date-parts":[[2023,9,6]],"date-time":"2023-09-06T17:42:02Z","timestamp":1694022122000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Velocity range-based reward shaping technique for effective map-less navigation with LiDAR sensor and deep reinforcement learning"],"prefix":"10.3389","volume":"17","author":[{"given":"HyeokSoo","family":"Lee","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jongpil","family":"Jeong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,9,6]]},"reference":[{"key":"B1","article-title":"\u201cDesign and Implementation of AMR Robot Based on RGBD, VSLAM and SLAM,\u201d","author":"Adithya","year":"2021","journal-title":"2021 9th International Conference on Orange Technology (ICOT)"},{"key":"B2","author":"Baek","year":"2021","journal-title":"A Comparative Research on SLAM Algorithm for Autonomous Driving of Indoor Mobile Robots"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1109\/SMC.2018.00221","article-title":"\u201cRobust 2D indoor localization through laser SLAM and visual SLAM fusion,\u201d","author":"Chan","year":"2018","journal-title":"2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC)"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1109\/ICOT54518.2021.9680621","article-title":"\u201cDesign and Implementation of AMR Robot Based on RGBD, VSLAM and SLAM,\u201d","author":"Chen","year":"2021","journal-title":"2021 9th International Conference on Orange Technology (ICOT)"},{"key":"B5","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1016\/j.ejor.2021.01.019","article-title":"Planning and control of autonomous mobile robots for intralogistics : literature review and research agenda","volume":"294","author":"Giuseppe","year":"2021","journal-title":"Eur. J. Oper. Res."},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561188","article-title":"\u201cDeep reinforcement learning for mapless navigation of a hybrid aerial underwater vehicle with medium transition,\u201d","author":"Grando","year":"2021","journal-title":"2021 IEEE International Conference on Robotics and Automation (ICRA)"},{"key":"B7","article-title":"\u201cSoft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor,\u201d","author":"Haarnoja","year":"","journal-title":"2018 35th International Conference on Machine Learning (ICLR 2018)"},{"key":"B8","author":"Haarnoja","year":""},{"key":"B9","article-title":"\u201cMobile robot navigation based on deep reinforcement learning with 2D-LiDAR sensor using stochastic approach,\u201d","author":"Han","year":"2021","journal-title":"2021 IEEE International Conference on Intelligence and Safety for Robotics (ISR)"},{"key":"B10","doi-asserted-by":"publisher","first-page":"48","DOI":"10.5302\/J.ICROS.2014.13.1910","article-title":"Mobile robot path optimization technique based on reinforcement learning algorithm in warehouse environment","volume":"20","author":"Han","year":"2014","journal-title":"J. Inst. Control Robot. Syst"},{"key":"B11","doi-asserted-by":"publisher","first-page":"698","DOI":"10.1177\/0278364920987859","article-title":"How to train your robot with deep reinforcement learning; lessons we've learned","volume":"40","author":"Ibarz","year":"2021","journal-title":"SAGE J"},{"key":"B12","unstructured":"International Federation of Robotics2022"},{"key":"B13","article-title":"\u201cArchitecture and design considerations for an autonomous mobile robot,\u201d","author":"Juan","year":"2021","journal-title":"2021 5th IEEE URUCON"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1109\/ICECA52323.2021.9675949","article-title":"\u201cReinforcement Learning for Navigation of Mobile Robot with LiDAR,\u201d","author":"Kim","year":"2021","journal-title":"2021 5th International Conference on Electronics, Communication and Aerospace Technology (ICECA)"},{"key":"B15","author":"Kim","year":"2002","journal-title":"A study on the path planning and control of the small mobile robot"},{"key":"B16","doi-asserted-by":"publisher","first-page":"77","DOI":"10.32971\/als.2019.006","article-title":"The concept of autonomous systems in industry 4.0","volume":"12","author":"Kov\u00e1cs","year":"2018","journal-title":"Adv. Logist. Syst"},{"key":"B17","doi-asserted-by":"publisher","first-page":"4703","DOI":"10.3390\/app12094703","article-title":"MARL-based dual reward model on segmented actions for multiple mobile robots in automated warehouse environment","volume":"12","author":"Lee","year":"2022","journal-title":"Appl. Sci."},{"key":"B18","doi-asserted-by":"publisher","first-page":"1209","DOI":"10.3390\/app11031209","article-title":"Mobile robot path optimization technique based on reinforcement learning algorithm in warehouse environment","volume":"11","author":"Lee","year":"2021","journal-title":"Appl. Sci."},{"key":"B19","article-title":"\u201cVirtual-to-real deep reinforcement learning: continuous control of mobile robots for mapless navigation,\u201d","author":"Lei","year":"2017","journal-title":"2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)"},{"key":"B20","article-title":"\u201cContinuous Control with Deep Reinforcement Learning,\u201d","author":"Lillicrap","year":"2016","journal-title":"4th International Conference on Learning Representations (ICLR 2016)"},{"key":"B21","author":"Liu","year":"2021","journal-title":"Autonomous driving segway robots"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1016\/j.rcim.2022.102360","article-title":"Robot learning towards smart robotic manufacturing: a review","author":"Liu","year":"2022","journal-title":"Robot. Comput. Integr. Manufact"},{"key":"B23","unstructured":"2017"},{"key":"B24","article-title":"\u201cAsynchronous Methods for Deep Reinforcement Learning,\u201d","author":"Mnih","year":"2016","journal-title":"2016 the 33rd International Conference on Machine Learning (ICML 2016)"},{"key":"B25","article-title":"\u201cDesign of an Autonomous Mobile Robot Based on ROS,\u201d","author":"Murat","year":"2017","journal-title":"2017 International Artificial Intelligence and Data Processing Symposium (IDAP)"},{"key":"B26","unstructured":"2018"},{"key":"B27","article-title":"\u201cTemporal Difference Rewards for End-to-end Vision-based Active Robot Tracking Using Deep Reinforcement Learning,\u201d","author":"Pavlos","year":"2021","journal-title":"2021 International Conference on Emerging Techniques in Computational Intelligence (ICETCI)"},{"key":"B28","doi-asserted-by":"publisher","first-page":"6951","DOI":"10.3390\/app12146951","article-title":"A comprehensive study of mobile robot: history, developments, applications, and future research perspectives","volume":"12","author":"Raj","year":"2022","journal-title":"Appl. Sci."},{"key":"B29","doi-asserted-by":"publisher","first-page":"712","DOI":"10.1002\/rob.20310","article-title":"Performance of laser and radar ranging devices in adverse environmental conditions","volume":"26","author":"Ryde","year":"2009","journal-title":"J. Field Robot."},{"key":"B30","unstructured":"2021"},{"key":"B31","author":"Sharma","year":"2020","journal-title":"SAC-RL: Continuous Control of Wheeled Mobile Robot for Navigation in a Dynamic Environment"},{"key":"B32","unstructured":"SilverD.\n          UCL Course on RL2015"},{"key":"B33","unstructured":"2019"},{"key":"B34","author":"Sutton","year":"2018","journal-title":"Introduction to Reinforcement Learning."},{"key":"B35","unstructured":"VincentJ.\n          Amazon has made its own autonomous six-wheeled delivery robot2019"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1109\/ISCID.2019.00036","article-title":"\u201cApplication of DDPG-based collision avoidance algorithm in air traffic control,\u201d","author":"Wen","year":"2019","journal-title":"2019 12th International Symposium on Computational Intelligence and Design (ISCID)"},{"key":"B37","author":"West","year":"2016","journal-title":"Moving forward: Self-driving vehicles in China, Europe, Japan, Korea, and the United States"},{"key":"B38","unstructured":"LiDAR vs. RADAR: Detection, Tracking, and Imaging2021"},{"key":"B39","doi-asserted-by":"publisher","first-page":"1209","DOI":"10.3389\/fnbot.2020.00063","article-title":"The path planning of mobile robot by neural networks and hierarchical reinforcement learning","volume":"14","author":"Yu","year":"2020","journal-title":"Front. Neurorobot"},{"key":"B40","article-title":"\u201cDesign and implementation of AMR robot based on RGBD, VSLAM and SLAM,\u201d","author":"Zhang","year":"2019","journal-title":"2019 Chinese Control Conference (CCC)"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1109\/CCET50901.2020.9213166","article-title":"\u201cRobot path planning method based on deep reinforcement learning,\u201d","author":"Zhang","year":"2020","journal-title":"2020 3rd International Conference on Computer and Communication Engineering Technology (CCET)"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN52387.2021.9533570","article-title":"\u201cPath planning for mobile robots based on TPR-DDPG,\u201d","author":"Zhao","year":"2021","journal-title":"2021 International Joint Conference on Neural Networks (IJCNN)"},{"key":"B43","doi-asserted-by":"publisher","first-page":"674","DOI":"10.26599\/TST.2021.9010012","article-title":"Deep reinforcement learning based mobile robot navigation: a review","volume":"26","author":"Zhu","year":"2021","journal-title":"Tsinghua Sci. Technol"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1210442\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,6]],"date-time":"2023-09-06T17:42:11Z","timestamp":1694022131000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1210442\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,6]]},"references-count":43,"alternative-id":["10.3389\/fnbot.2023.1210442"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1210442","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,6]]},"article-number":"1210442"}}