{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T11:57:15Z","timestamp":1785412635238,"version":"3.56.0"},"reference-count":35,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T00:00:00Z","timestamp":1781654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Institute of Information & Communications Technology Planning & Evaluation","award":["RS-2025-25463654"],"award-info":[{"award-number":["RS-2025-25463654"]}]},{"award":["RS-2025-25463654"],"award-info":[{"award-number":["RS-2025-25463654"]}],"id":[{"id":"https:\/\/ror.org\/01g0hqq23","id-type":"ROR","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003668","name":"Korea Institute of Planning and Evaluation for Technology in Food, Agriculture and Forestry","doi-asserted-by":"publisher","award":["RS-2025-02315577"],"award-info":[{"award-number":["RS-2025-02315577"]}],"id":[{"id":"10.13039\/501100003668","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Long-horizon mobile manipulation requires a robot to execute a sequence of heterogeneous subtasks such as navigation, picking, and articulated-object manipulation in indoor environments. Standard reinforcement learning suffers from reward sparsity and inefficient exploration in this setting, and hierarchical methods often fail at the hand-off between consecutive subtasks when the terminal state of one subtask is kinematically infeasible for the next. We propose a motion planning-augmented hierarchical reinforcement learning architecture to resolve the fundamental trade-offs between sample efficiency and hand-off reliability in long-horizon mobile manipulation. The mission is decomposed into subtasks via a Semi-Markov Decision Process; within each subtask, a collision-free reference trajectory generated by RRT* in the full joint configuration space is embedded into the reward as a per-step shaping signal; and a region-goal mechanism, defined analytically from inverse kinematics feasibility, replaces rigid coordinate hand-offs with a continuous feasible region. The architecture is evaluated in the ManiSkill-HAB simulation under teleport-free sequential execution and challenging initialization. The proposed method improves subtask success rate and sample efficiency over the baseline across all six evaluated subtasks, and the advantage compounds along the long-horizon task chain.<\/jats:p>","DOI":"10.3390\/s26123845","type":"journal-article","created":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T08:22:56Z","timestamp":1781684576000},"page":"3845","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Motion Planning-Augmented Hierarchical Reinforcement Learning for Long-Horizon Mobile Manipulation"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-4789-2866","authenticated-orcid":false,"given":"Hyungtai","family":"Kim","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5025-8785","authenticated-orcid":false,"given":"Mun-Taek","family":"Choi","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,17]]},"reference":[{"key":"ref_1","unstructured":"Shukla, A., Tao, S., and Su, H. (2025). ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks. Proceedings of the International Conference on Learning Representations (ICLR), OpenReview.net."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Xia, F., Li, C., Mart\u00edn-Mart\u00edn, R., Litany, O., Toshev, A., and Savarese, S. (2021). Relmogen: Integrating motion generation in reinforcement learning for mobile manipulation. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA48506.2021.9561315"},{"key":"ref_3","first-page":"2628","article-title":"ARCH: Hierarchical Hybrid Learning for Long-Horizon Contact-Rich Robotic Assembly","volume":"Volume 305","author":"Sun","year":"2025","journal-title":"Proceedings of the 9th Conference on Robot Learning (CoRL), Seoul, Republic of Korea, 27\u201330 September 2025"},{"key":"ref_4","unstructured":"Yamada, J., Lee, Y., Salhotra, G., Pertsch, K., Pflueger, M., Sukhatme, G., Lim, J., and Englert, P. (2021). Motion planner augmented reinforcement learning for robot manipulation in obstructed environments. Proceedings of the Conference on Robot Learning, PMLR."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Faust, A., Ramirez, O., Fiser, M., Oslund, K., Francis, A., Davidson, J., and Tapia, L. (2018). PRM-RL: Long-range robotic navigation tasks by combining reinforcement learning and sampling-based planning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA.2018.8461096"},{"key":"ref_6","first-page":"251","article-title":"Habitat 2.0: Training home assistants to rearrange their habitat","volume":"Volume 34","author":"Szot","year":"2021","journal-title":"Proceedings of the Advances in Neural Information Processing Systems (NeurIPS)"},{"key":"ref_7","unstructured":"Kindle, J., Furrer, F., Novkovic, T., Chung, J.J., Siegwart, R., and Nieto, J. (2020). Whole-Body Control of a Mobile Manipulator using End-to-End Reinforcement Learning. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0004-3702(99)00052-1","article-title":"Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning","volume":"112","author":"Sutton","year":"1999","journal-title":"Artif. Intell."},{"key":"ref_9","unstructured":"Konidaris, G., and Barto, A. (2009). Skill discovery in continuous reinforcement learning domains using skill chaining. Proceedings of the Advances in Neural Information Processing Systems, Curran Associates Inc."},{"key":"ref_10","unstructured":"Bhaskar, A., Mahammad, Z., Jadhav, S.R., and Tokekar, P. (2024). PLANRL: A Motion Planning and Imitation Learning Framework to Bootstrap Reinforcement Learning. arXiv."},{"key":"ref_11","unstructured":"Dalal, M., Chiruvolu, T., Chaplot, D.S., and Salakhutdinov, R. (2024). Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks. Proceedings of the International Conference on Learning Representations (ICLR), OpenReview.net."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gao, J., Ye, W., Guo, J., and Li, Z. (2020). Deep Reinforcement Learning for Indoor Mobile Robot Path Planning. Sensors, 20.","DOI":"10.3390\/s20195493"},{"key":"ref_13","first-page":"1943","article-title":"Deep Reactive Planning in Dynamic Environments","volume":"Volume 155","author":"Ota","year":"2021","journal-title":"Proceedings of the 2020 Conference on Robot Learning. PMLR; Machine Learning Research"},{"key":"ref_14","unstructured":"Kolomeytsev, Y., and Golembiovsky, D. (2025). Hybrid Motion Planning with Deep Reinforcement Learning for Mobile Robot Navigation. arXiv."},{"key":"ref_15","unstructured":"Freitag, K., Ceder, K., Laezza, R., \u00c5kesson, K., and Chehreghani, M.H. (2024). Curriculum Reinforcement Learning for Complex Reward Functions. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Song, S., Bihl, T., and Liu, J. (2026). Coulomb force-guided deep reinforcement learning for effective and explainable robotic motion planning. Front. Robot. AI, 12.","DOI":"10.3389\/frobt.2025.1697155"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"114284","DOI":"10.1016\/j.engappai.2026.114284","article-title":"A trend-aware reinforcement learning approach for adaptive motion planning of robotic manipulators in dynamic environments","volume":"171","author":"Wang","year":"2026","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J.A., Solowjow, E., and Levine, S. (2019). Residual Reinforcement Learning for Robot Control. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA.2019.8794127"},{"key":"ref_19","unstructured":"Jauhri, S., Prasad, V., and Chalvatzaki, G. (2026). Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers. arXiv."},{"key":"ref_20","unstructured":"Gu, J., Chaplot, D.S., Su, H., and Malik, J. (2023). Multi-skill Mobile Manipulation for Object Rearrangement. Proceedings of the International Conference on Learning Representations (ICLR), OpenReview.net."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Puterman, M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons.","DOI":"10.1002\/9780470316887"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"LaValle, S.M. (2006). Planning Algorithms, Cambridge University Press.","DOI":"10.1017\/CBO9780511546877"},{"key":"ref_23","unstructured":"LaValle, S.M. (1998). Rapidly-Exploring Random Trees: A New Tool for Path Planning, Department of Computer Science, Iowa State University. Technical Report TR 98-11."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1177\/0278364911406761","article-title":"Sampling-Based Algorithms for Optimal Motion Planning","volume":"30","author":"Karaman","year":"2011","journal-title":"Int. J. Robot. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"566","DOI":"10.1109\/70.508439","article-title":"Probabilistic Roadmaps for Path Planning in High-Dimensional Configuration Spaces","volume":"12","author":"Kavraki","year":"1996","journal-title":"IEEE Trans. Robot. Autom."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Huang, X., Batra, D., Rai, A., and Szot, A. (2023). Skill Transformer: A Monolithic Policy for Mobile Manipulation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), IEEE.","DOI":"10.1109\/ICCV51070.2023.00996"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sundaralingam, B., Hari, S.K.S., Fishman, A., Garrett, C., Van Wyk, K., Blukis, V., Millane, A., Oleynikova, H., Handa, A., and Ramos, F. (2023). CuRobo: Parallelized collision-free robot motion generation. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA48891.2023.10160765"},{"key":"ref_28","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_29","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the International Conference on Machine Learning (ICML), PMLR."},{"key":"ref_30","unstructured":"Lin, T., Sachdev, K., Fan, L., Malik, J., and Zhu, Y. (2025). Sim-to-real reinforcement learning for vision-based dexterous manipulation on humanoids. arXiv."},{"key":"ref_31","first-page":"100235","article-title":"An adaptive compensation strategy for sensors based on the degree of degradation","volume":"5","author":"Li","year":"2025","journal-title":"Biomim. Intell. Robot."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"124616","DOI":"10.1016\/j.oceaneng.2026.124616","article-title":"Design and depth control of a beaver-inspired underwater robot based on center-of-mass adjustment","volume":"352","author":"Chen","year":"2026","journal-title":"Ocean Eng."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4302","DOI":"10.1017\/S0263574725102865","article-title":"A Novel Semi-Coupled Hierarchical Motion Planning Framework for Cooperative Transportation of Multiple Mobile Manipulators","volume":"43","author":"Zhang","year":"2025","journal-title":"Robotica"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zang, X., Zhang, X., and Zhao, J. (2023). A Hierarchical Motion Planning Method for Mobile Manipulator. Sensors, 23.","DOI":"10.3390\/s23156952"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1109\/MRA.2012.2205651","article-title":"The Open Motion Planning Library","volume":"19","author":"Moll","year":"2012","journal-title":"IEEE Robot. Autom. Mag."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/12\/3845\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T08:32:49Z","timestamp":1781685169000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/12\/3845"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,17]]},"references-count":35,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["s26123845"],"URL":"https:\/\/doi.org\/10.3390\/s26123845","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,17]]}}}