{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T16:03:46Z","timestamp":1758816226978},"reference-count":19,"publisher":"IGI Global","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2013,1,1]]},"abstract":"<p>The goal of this paper is to improve the performance of the well known Q learning algorithm, the robust technique of Machine learning to facilitate path planning in an environment. Until this time the Q learning algorithms like Classical Q learning(CQL)algorithm and Improved Q learning (IQL) algorithm deal with an environment without obstacles, while in a real environment an agent has to face obstacles very frequently. Hence this paper considers an environment with number of obstacles and has coined a new parameter, called \u2018immediate penalty\u2019 due to collision with an obstacle. Further the proposed technique has replaced the scalar \u2018immediate reward\u2019 function by \u2018effective immediate reward\u2019 function which consists of two fuzzy parameters named as, \u2018immediate reward\u2019 and \u2018immediate penalty\u2019. The fuzzification of these two important parameters not only improves the learning technique, it also strikes a balance between exploration and exploitation, the most challenging problem of Reinforcement Learning. The proposed algorithm stores the Q value for the best possible action at a state; as well it saves significant path planning time by suggesting the best action to adopt at each state to move to the next state. Eventually, the agent becomes more intelligent as it can smartly plan a collision free path avoiding obstacles from distance. The validation of the algorithm is studied through computer simulation in a maze like environment and also on KheperaII platform in real time. An analysis reveals that the Q Table, obtained by the proposed Advanced Q learning (AQL) algorithm, when used for path-planning application of mobile robots outperforms the classical and improved Q-learning.<\/p>","DOI":"10.4018\/ijimr.2013010105","type":"journal-article","created":{"date-parts":[[2013,9,27]],"date-time":"2013-09-27T16:42:40Z","timestamp":1380300160000},"page":"53-73","source":"Crossref","is-referenced-by-count":21,"title":["An Advance Q Learning (AQL) Approach for Path Planning and Obstacle Avoidance of a Mobile Robot"],"prefix":"10.4018","volume":"3","author":[{"given":"Arpita","family":"Chakraborty","sequence":"first","affiliation":[{"name":"Bengal Institute of Technology, Kolkata, West Bengal, India"}]},{"given":"Jyoti Sekhar","family":"Banerjee","sequence":"additional","affiliation":[{"name":"Bengal Institute of Technology, Kolkata, West Bengal, India"}]}],"member":"2432","reference":[{"key":"ijimr.2013010105-0","author":"R. E.Bellman","year":"1957","journal-title":"Dynamic programming"},{"key":"ijimr.2013010105-1","doi-asserted-by":"publisher","DOI":"10.1109\/70.143360"},{"key":"ijimr.2013010105-2","doi-asserted-by":"publisher","DOI":"10.1201\/9781439821091"},{"key":"ijimr.2013010105-3","unstructured":"Chen, W., Guo, J., Li, X., & Wang, J. (2009). hybrid q-learning algorithm about cooperation in mas. In Proceedings of the Chinese Control and Decision Conference (CCDC)."},{"key":"ijimr.2013010105-4","doi-asserted-by":"crossref","unstructured":"Cho, K., Sung, Y., & Um, K. (2007). A production technique for a q-table with an influence map for speeding up q-learning. In Proceedings of the International Conference on Intelligent Pervasive Computing.","DOI":"10.1109\/IPC.2007.88"},{"key":"ijimr.2013010105-5","doi-asserted-by":"publisher","DOI":"10.5120\/8073-1468"},{"key":"ijimr.2013010105-6","doi-asserted-by":"crossref","DOI":"10.1016\/B978-1-4832-0774-2.50008-1","article-title":"Reinforcement learning for planning and Control","author":"T.Dean","year":"1993","journal-title":"Machine learning methods for planning and scheduling"},{"key":"ijimr.2013010105-7","doi-asserted-by":"crossref","unstructured":"Gerke, M., & Hoyer, H. (1997). Planning of optimal paths for autonomous agents moving in inhomogeneous environments. In Proceedings of the 8th International Conference on Advanced Robotics (pp.347-352).","DOI":"10.1109\/ICAR.1997.620205"},{"key":"ijimr.2013010105-8","unstructured":"Hsu, Y. P., Jiang, W. C., & Lin, H. Y. (2008). A CMAC-Q-learning based dyna agent. In Proceedings of the SICE Annual Conference (pp. 2946\u20132950). The University Electro-Communications, Tokyo, Japan."},{"key":"ijimr.2013010105-9","doi-asserted-by":"crossref","unstructured":"Jeni, L. A., Korondi, Z. I. P., & Hierarchical, H. H. (2007). Reinforcement learning for robot navigation using the intelligent space concept. In Proceedings of the 11th International Conference on Intelligent Engineering Systems, Budapest, Hungary.","DOI":"10.1109\/INES.2007.4283689"},{"key":"ijimr.2013010105-10","unstructured":"Martin, T. M., & Rodriguez, R. (2007). Navigation of autonomous vehicles in unknown environments using reinforcement learning. In Proceedings of the IEEE Intelligent Vehicles Symposium, Istanbul, Turkey."},{"key":"ijimr.2013010105-11","doi-asserted-by":"crossref","unstructured":"Masoumzadeh, S. S., Taghizadeh, G., Meshgi, K., & Shiry, S. (2009). Deep blue: A fuzzy q-learning enhanced active queue management scheme. In Proceedings of the International Conference on Adaptive and Intelligent Systems.","DOI":"10.1109\/ICAIS.2009.17"},{"key":"ijimr.2013010105-12","doi-asserted-by":"crossref","unstructured":"Moll, M., & Kavraki, L. E. (2004). Path planning for minimal energy curves of constant length. In Proceedings of the IEEE International Conference on Robotics and Automation (pp.2826-2831).","DOI":"10.1109\/ROBOT.2005.1570428"},{"key":"ijimr.2013010105-13","doi-asserted-by":"crossref","unstructured":"Pandey, D., & Pandey, P. (2010). Approximate q-learning: An introduction. In Proceedings of the Second International Conference on Machine Learning and Computing.","DOI":"10.1109\/ICMLC.2010.38"},{"issue":"6","key":"ijimr.2013010105-14","first-page":"674","article-title":"Path planning for a robot manipulator based on probabilistic roadmap and reinforcement learning.","volume":"5","author":"J. J.Park","year":"2007","journal-title":"International Journal of Control, Automation,and Systems"},{"key":"ijimr.2013010105-15","doi-asserted-by":"crossref","unstructured":"Regele, R., & Levi, P. (2006). Cooperative multi-robot path planning by heuristic priority adjustment, In Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems.","DOI":"10.1109\/IROS.2006.282480"},{"key":"ijimr.2013010105-16","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"issue":"1","key":"ijimr.2013010105-17","article-title":"Adaptive evolutionary planner\/ navigator for mobile robots.","volume":"1","author":"J.Xiao","year":"1997","journal-title":"IEEE Transactions on Evolutionary Computation"},{"key":"ijimr.2013010105-18","doi-asserted-by":"crossref","unstructured":"Zhou, Y., & Er, M. J. (2007). A novel q-learning approach with continuous states and actions. In Proceedings of the 16th IEEE International Conference on Control Applications Part of IEEE Multi-conference on Systems and Control, Singapore.","DOI":"10.1109\/CCA.2007.4389199"}],"container-title":["International Journal of Intelligent Mechatronics and Robotics"],"original-title":[],"language":"ng","link":[{"URL":"https:\/\/www.igi-global.com\/viewtitle.aspx?TitleId=87481","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,1]],"date-time":"2022-06-01T15:00:19Z","timestamp":1654095619000},"score":1,"resource":{"primary":{"URL":"https:\/\/services.igi-global.com\/resolvedoi\/resolve.aspx?doi=10.4018\/ijimr.2013010105"}},"subtitle":[""],"short-title":[],"issued":{"date-parts":[[2013,1,1]]},"references-count":19,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2013,1]]}},"URL":"https:\/\/doi.org\/10.4018\/ijimr.2013010105","relation":{},"ISSN":["2156-1664","2156-1656"],"issn-type":[{"value":"2156-1664","type":"print"},{"value":"2156-1656","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,1,1]]}}}