{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T10:06:25Z","timestamp":1760609185405,"version":"build-2065373602"},"reference-count":48,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2019,11,7]],"date-time":"2019-11-07T00:00:00Z","timestamp":1573084800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003130","name":"Fonds Wetenschappelijk Onderzoek","doi-asserted-by":"publisher","award":["1S40217N"],"award-info":[{"award-number":["1S40217N"]}],"id":[{"id":"10.13039\/501100003130","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>By coupling a robot to a smart environment, the robot can sense state beyond the perception range of its onboard sensors and gain greater actuation capabilities. Nevertheless, incorporating the states and actions of Internet of Things (IoT) devices into the robot\u2019s onboard planner increases the computational load, and thus can delay the execution of a task. Moreover, tasks may be frequently replanned due to the unanticipated actions of humans. Our framework aims to mitigate these inadequacies. In this paper, we propose a continual planning framework, which incorporates the sensing and actuation capabilities of IoT devices into a robot\u2019s state estimation, task planing and task execution. The robot\u2019s onboard task planner queries a cloud-based framework for actuators, capable of the actions the robot cannot execute. Once generated, the plan is sent to the cloud back-end, which will inform the robot if any IoT device reports a state change affecting its plan. Moreover, a Hierarchical Continual Planning in the Now approach was developed in which tasks are split-up into subtasks. To delay the planning of actions that will not be promptly executed, and thus to reduce the frequency of replanning, the first subtask is planned and executed before the subsequent subtask is. Only information relevant to the current (sub)task is provided to the task planner. We apply our framework to a smart home and office scenario in which the robot is tasked with carrying out a human\u2019s requests. A prototype implementation in a smart home, and simulator-based evaluation results, are presented to demonstrate the effectiveness of our framework.<\/jats:p>","DOI":"10.3390\/s19224856","type":"journal-article","created":{"date-parts":[[2019,11,7]],"date-time":"2019-11-07T11:17:25Z","timestamp":1573125445000},"page":"4856","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Robot Assistance in Dynamic Smart Environments\u2014A Hierarchical Continual Planning in the Now Framework"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3195-2579","authenticated-orcid":false,"given":"Helen","family":"Harman","sequence":"first","affiliation":[{"name":"IDLab, Department of Information Technology, Ghent University\u2014imec, Technologiepark 126, B-9052 Ghent, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Keshav","family":"Chintamani","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information Technology, Ghent University\u2014imec, Technologiepark 126, B-9052 Ghent, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9569-9373","authenticated-orcid":false,"given":"Pieter","family":"Simoens","sequence":"additional","affiliation":[{"name":"IDLab, Department of Information Technology, Ghent University\u2014imec, Technologiepark 126, B-9052 Ghent, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,11,7]]},"reference":[{"unstructured":"Cashmore, M., and Magazzeni, D. (2019, November 07). ICRA 2017 Tutorial on AI Planning for Robotics. Available online: http:\/\/kcl-planning.github.io\/ROSPlan\/\/demos\/conference_pages\/tutorialICRA2017.html.","key":"ref_1"},{"key":"ref_2","first-page":"70","article-title":"Casper: Space Exploration Through Continuous Planning","volume":"16","author":"Knight","year":"2001","journal-title":"IEEE Intell. Syst."},{"key":"ref_3","first-page":"1203","article-title":"Shakey 2016\u2014How Much Does it Take to Redo Shakey the Robot?","volume":"2","author":"Speck","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"unstructured":"Dornhege, C., and Hertle, A. (2013, January 25\u201327). Integrated Symbolic Planning in the Tidyup-Robot Project. Proceedings of the AAAI Spring Symposium: Designing Intelligent Robots, Palo Alto, CA, USA.","key":"ref_4"},{"doi-asserted-by":"crossref","unstructured":"Cashmore, M., Fox, M., Long, D., Magazzeni, D., Ridder, B., Carreraa, A., Palomeras, N., Hurt\u00f3s, N., and Carrerasa, M. (2015, January 7\u201311). ROSPlan: Planning in the Robot Operating System. Proceedings of the Twenty-Fifth International Conference on International Conference on Automated Planning and Scheduling, Jerusalem, Israel.","key":"ref_5","DOI":"10.1609\/icaps.v25i1.13699"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2787","DOI":"10.1016\/j.comnet.2010.05.010","article-title":"The Internet of Things: A Survey","volume":"54","author":"Atzori","year":"2010","journal-title":"Comput. Netw."},{"unstructured":"Vermesan, O., Br\u00f6ring, A., Tragos, E., Serrano, M., Bacciu, D., Chessa, S., Gallicchio, C., Micheli, A., Dragone, M., and Saffiotti, A. (2017). Internet of Robotic Things: Converging Sensing\/Actuating, Hypoconnectivity, Artificial Intelligence and IoT Platforms. Cognitive Hyperconnected Digital Transformation: Internet of Things Intelligence Evolution, River Publishers.","key":"ref_7"},{"doi-asserted-by":"crossref","unstructured":"Harman, H., Chintamani, K., and Simoens, P. (2017, January 5\u20137). Architecture for Incorporating Internet-of-Things Sensors and Actuators into Robot Task Planning in Dynamic Environments. Proceedings of the IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), Ottawa, ON, Canada.","key":"ref_8","DOI":"10.1109\/IRIS.2017.8250091"},{"doi-asserted-by":"crossref","unstructured":"Simoens, P., Dragone, M., and Saffiotti, A. (2018). The Internet of Robotic Things: A Review of Concept, Added Value and Applications. Int. J. Adv. Robot. Syst., 15.","key":"ref_9","DOI":"10.1177\/1729881418759424"},{"unstructured":"Papadakis, P., Lohr, C., Lujak, M., Karami, A., Kanellos, I., Lozenguez, G., and Fleury, A. (February, January 31). System Design for Coordinated Multi-Robot Assistance Deployment in Smart Spaces. Proceedings of the Second IEEE International Conference on Robotic Computing, Laguna Hills, CA, USA.","key":"ref_10"},{"unstructured":"Kovacs, D.L. (2012, January 25\u201329). A Multi-agent Extension of PDDL3.1. Proceedings of the Third ICAPS Workshop on the International Planning Competition (IPC), Sao Paulo, Brazil.","key":"ref_11"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/s10458-009-9081-1","article-title":"Continual Planning and Acting in Dynamic Multiagent Environments","volume":"19","author":"Brenner","year":"2009","journal-title":"Auton. Agents Multi-Agent Syst."},{"doi-asserted-by":"crossref","unstructured":"Bidot, J., and Biundo, S. (2011). Artificial Intelligence Planning for Ambient Environments. Next Generation Intelligent Environments, Springer.","key":"ref_13","DOI":"10.1007\/978-1-4614-1299-1_6"},{"doi-asserted-by":"crossref","unstructured":"Saffiotti, A., Broxvall, M., Gritti, M., LeBlanc, K., Lundh, R., Rashid, J., Seo, B.S., and Cho, Y.J. (2008, January 22\u201326). The PEIS-Ecology Project: Vision and Results. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Nice, France.","key":"ref_14","DOI":"10.1109\/IROS.2008.4650962"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1016\/j.robot.2008.06.006","article-title":"Autonomous Functional Configuration of a Network Robot System","volume":"56","author":"Lundh","year":"2008","journal-title":"Robot. Auton. Syst."},{"unstructured":"Broxvall, M., Gritti, M., Saffiotti, A., Seo, B.S., and Cho, Y.J. (2006, January 15\u201319). PEIS Ecology: Integrating Robots into Smart Environments. Proceedings of the IEEE International Conference on Robotics and Automation, Orlando, FL, USA.","key":"ref_16"},{"doi-asserted-by":"crossref","unstructured":"Buehler, J., and Pagnucco, M. (2014, January 27\u201331). A Framework for Task Planning in Heterogeneous Multi Robot Systems Based on Robot Capabilities. Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Qu\u00e9bec City, QC, Canada.","key":"ref_17","DOI":"10.1609\/aaai.v28i1.9056"},{"doi-asserted-by":"crossref","unstructured":"Blomqvist, E., Ciancarini, P., Poggi, F., and Vitali, F. (2016). IoT-O, a Core-Domain IoT Ontology to Represent Connected Devices Networks. Knowledge Engineering and Knowledge Management, Springer International Publishing.","key":"ref_18","DOI":"10.1007\/978-3-319-49004-5"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MRA.2016.2645444","article-title":"A Suite of Ontologies for Robotics and Automation [Industrial Activities]","volume":"24","author":"Fiorini","year":"2017","journal-title":"IEEE Robot. Autom. Mag."},{"doi-asserted-by":"crossref","unstructured":"Kunze, L., Roehm, T., and Beetz, M. (2011, January 9\u201313). Towards Semantic Robot Description Languages. Proceedings of the IEEE International Conference on Robotics and Automation, Shanghai, China.","key":"ref_20","DOI":"10.1109\/ICRA.2011.5980170"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1109\/TASE.2014.2377791","article-title":"RoboEarth Semantic Mapping: A Cloud Enabled Knowledge-based Approach","volume":"12","author":"Riazuelo","year":"2015","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"955","DOI":"10.1016\/j.robot.2008.08.007","article-title":"Robot Task Planning Using Semantic Maps","volume":"56","author":"Galindo","year":"2008","journal-title":"Robot. Auton. Syst."},{"unstructured":"K\u00f6ckemann, U., Alirezaie, M., Karlsson, L., and Loutfi, A. (2018, January 13). Integrating Ontologies for Context-based Constraint-based Planning. Proceedings of the Tenth International Workshop on Modelling and Reasoning in Context (IJCAI-MRC), Stockholm, Sweden.","key":"ref_23"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1613\/jair.1705","article-title":"The Fast Downward Planning System","volume":"26","author":"Helmert","year":"2006","journal-title":"J. Artif. Intell. Res."},{"doi-asserted-by":"crossref","unstructured":"Dornhege, C., Eyerich, P., Keller, T., Tr\u00fcg, S., Brenner, M., and Nebel, B. (2009, January 19\u201323). Semantic Attachments for Domain-independent Planning Systems. Proceedings of the Nineteenth International Conference on International Conference on Automated Planning and Scheduling, Thessaloniki, Greece.","key":"ref_25","DOI":"10.1609\/icaps.v19i1.13380"},{"unstructured":"Lallement, R., De Silva, L., and Alami, R. (2014, January 22\u201323). HATP: An HTN Planner for Robotics. Proceedings of the Second ICAPS Workshop on Planning and Robotics (PlanRob), Portsmouth, NH, USA.","key":"ref_26"},{"unstructured":"Myers, K., Smith, S.F., Hildum, D.W., Jarvis, P.A., and de Lacaze, R. (2014, January 12\u201314). Integrating Planning and Scheduling Through Adaptation of Resource Intensity Estimates. Proceedings of the Sixth European Conference on Planning, Toledo, Spain.","key":"ref_27"},{"doi-asserted-by":"crossref","unstructured":"Sukkerd, R., C\u00e1mara, J., Garlan, D., and Simmons, R. (2016, January 16). Multiscale Time Abstractions for Long-range Planning Under Uncertainty. Proceedings of the Second International Workshop on Software Engineering for Smart Cyber-Physical Systems, Austin, TX, USA.","key":"ref_28","DOI":"10.1145\/2897035.2897044"},{"doi-asserted-by":"crossref","unstructured":"Kaelbling, L.P., and Lozano-P\u00e9rez, T. (2011, January 9\u201313). Hierarchical Task and Motion Planning in the Now. Proceedings of the IEEE International Conference on Robotics and Automation, Shanghai, China.","key":"ref_29","DOI":"10.1109\/ICRA.2011.5980391"},{"doi-asserted-by":"crossref","unstructured":"Kaelbling, L.P., and Lozano-P\u00e9rez, T. (2016, January 16\u201321). Implicit Belief-space Pre-images for Hierarchical Planning and Execution. Proceedings of the IEEE International Conference on Robotics and Automation, Stockholm, Sweden.","key":"ref_30","DOI":"10.1109\/ICRA.2016.7487758"},{"unstructured":"Ramoly, N., Bouzeghoub, A., and Finance, B. (2016, January 21\u201322). Context-aware Planning by Refinement for Personal Robots in Smart Homes. Proceedings of the Forty-Seventh International Symposium on Robotics (ISR), Munich, Germany.","key":"ref_31"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1016\/j.robot.2016.04.009","article-title":"Planning and Execution Through Variable Resolution Planning","volume":"83","author":"Borrajo","year":"2016","journal-title":"Robot. Auton. Syst."},{"unstructured":"Rao, A.S., and Georgeff, M.P. (1995, January 12\u201314). BDI Agents: From Theory to Practice. Proceedings of the First International Conference on Multiagent Systems, San Francisco, CA, USA.","key":"ref_33"},{"unstructured":"M\u00fcller, J.P., Rao, A.S., and Singh, M.P. (1999). BDI Models and Systems: Reducing the Gap. Intelligent Agents V: Agents Theories, Architectures, and Languages, Springer.","key":"ref_34"},{"doi-asserted-by":"crossref","unstructured":"Eymann, T., Kl\u00fcgl, F., Lamersdorf, W., Klusch, M., and Huhns, M.N. (2005). A Goal Deliberation Strategy for BDI Agent Systems. Multiagent System Technologies, Springer.","key":"ref_35","DOI":"10.1007\/11550648"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1017\/S0269888913000337","article-title":"Planning in BDI Agents: A Survey of the Integration of Planning Algorithms and Agent Reasoning","volume":"30","author":"Meneguzzi","year":"2015","journal-title":"Knowl. Eng. Rev."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.artint.2015.02.002","article-title":"HTN Planning: Overview, Comparison, and Beyond","volume":"222","author":"Georgievski","year":"2015","journal-title":"Artif. Intell."},{"doi-asserted-by":"crossref","unstructured":"Cardoso, R.C., and Bordini, R.H. (2017). A Multi-agent Extension of a Hierarchical Task Network Planning Formalism. Adv. Distrib. Comput. Artif. Intell. J. (ADCAIJ), 6.","key":"ref_38","DOI":"10.14201\/ADCAIJ201762517"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1613\/jair.1141","article-title":"SHOP2: An HTN Planning System","volume":"20","author":"Nau","year":"2003","journal-title":"J. Artif. Intell. Res."},{"doi-asserted-by":"crossref","unstructured":"H\u00f6ller, D., Bercher, P., Behnke, G., and Biundo, S. (2018, January 24\u201329). A Generic Method to Guide HTN Progression Search with Classical Heuristics. Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling, Delft, The Netherlands.","key":"ref_40","DOI":"10.1609\/icaps.v28i1.13900"},{"doi-asserted-by":"crossref","unstructured":"Mandler, B., Marquez-Barja, J., Mitre Campista, M.E., Cag\u00e1\u0148ov\u00e1, D., Chaouchi, H., Zeadally, S., Badra, M., Giordano, S., Fazio, M., and Somov, A. (2016). Remote Management of a Large Set of Heterogeneous Devices Using Existing IoT Interoperability Platforms. Internet of Things: IoT Infrastructures, Springer International Publishing.","key":"ref_41","DOI":"10.1007\/978-3-319-47063-4"},{"doi-asserted-by":"crossref","unstructured":"Chamberlain, W., Leitner, J., Drummond, T., and Corke, P. (2016, January 21\u201324). A Distributed Robotic Vision Service. Proceedings of the IEEE International Conference on Robotics and Automation, Fort Worth, TX, USA.","key":"ref_42","DOI":"10.1109\/ICRA.2016.7487403"},{"doi-asserted-by":"crossref","unstructured":"Mokhtari, M., Abdulrazak, B., and Aloulou, H. (2017). Ambient Assisted Robot Object Search. Enhanced Quality of Life and Smart Living, Springer International Publishing.","key":"ref_43","DOI":"10.1007\/978-3-319-66188-9"},{"unstructured":"Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., and Ng, A.Y. (2009, January 17). ROS: An open-source Robot Operating System. Proceedings of the ICRA Workshop on Open Source Software, Kobe, Japan.","key":"ref_44"},{"unstructured":"Buehler, J., and Pagnucco, M. (2014, January 17\u201322). Planning and Execution of Robot Tasks Based on a Platform-independent Model of Robot Capabilities. Proceedings of the Twenty-first European Conference on Artificial Intelligence, Prague, Czech Republic.","key":"ref_45"},{"doi-asserted-by":"crossref","unstructured":"Hornung, A., B\u00f6ttcher, S., Schlagenhauf, J., Dornhege, C., Hertle, A., and Bennewitz, M. (2014, January 18\u201320). Mobile Manipulation in Cluttered Environments with Humanoids: Integrated Perception, Task Planning, and Action Execution. Proceedings of the IEEE-RAS International Conference on Humanoid Robots, Madrid, Spain.","key":"ref_46","DOI":"10.1109\/HUMANOIDS.2014.7041451"},{"doi-asserted-by":"crossref","unstructured":"Eyerich, P., Mattm\u00fcller, R., and R\u00f6ger, G. (2009, January 19\u201323). Using the Context-enhanced Additive Heuristic for Temporal and Numeric Planning. Proceedings of the Nineteenth International Conference on International Conference on Automated Planning and Scheduling, Thessaloniki, Greece.","key":"ref_47","DOI":"10.1609\/icaps.v19i1.13373"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1613\/jair.5328","article-title":"DESPOT: Online POMDP Planning with Regularization","volume":"58","author":"Ye","year":"2017","journal-title":"J. Artif. Intell. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/22\/4856\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:32:42Z","timestamp":1760189562000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/22\/4856"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,7]]},"references-count":48,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2019,11]]}},"alternative-id":["s19224856"],"URL":"https:\/\/doi.org\/10.3390\/s19224856","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,11,7]]}}}