{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T16:49:16Z","timestamp":1774630156383,"version":"3.50.1"},"reference-count":327,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,2,21]],"date-time":"2023-02-21T00:00:00Z","timestamp":1676937600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Manipulating cloth-like deformable objects (CDOs) is a long-standing problem in the robotics community. CDOs are flexible (non-rigid) objects that do not show a detectable level of compression strength while two points on the article are pushed towards each other and include objects such as ropes (1D), fabrics (2D) and bags (3D). In general, CDOs\u2019 many degrees of freedom (DoF) introduce severe self-occlusion and complex state\u2013action dynamics as significant obstacles to perception and manipulation systems. These challenges exacerbate existing issues of modern robotic control methods such as imitation learning (IL) and reinforcement learning (RL). This review focuses on the application details of data-driven control methods on four major task families in this domain: cloth shaping, knot tying\/untying, dressing and bag manipulation. Furthermore, we identify specific inductive biases in these four domains that present challenges for more general IL and RL algorithms.<\/jats:p>","DOI":"10.3390\/s23052389","type":"journal-article","created":{"date-parts":[[2023,2,22]],"date-time":"2023-02-22T02:08:34Z","timestamp":1677031714000},"page":"2389","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9290-467X","authenticated-orcid":false,"given":"Halid Abdulrahim","family":"Kadi","sequence":"first","affiliation":[{"name":"School of Computer Science, University of St Andrews, Jack Cole Building, North Haugh, St Andrews KY16 9SX, UK"}]},{"given":"Kasim","family":"Terzi\u0107","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of St Andrews, Jack Cole Building, North Haugh, St Andrews KY16 9SX, UK"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Tamei, T., Matsubara, T., Rai, A., and Shibata, T. (2011, January 26\u201328). Reinforcement learning of clothing assistance with a dual-arm robot. Proceedings of the 2011 11th IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia.","DOI":"10.1109\/Humanoids.2011.6100915"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Varier, V.M., Rajamani, D.K., Goldfarb, N., Tavakkolmoghaddam, F., Munawar, A., and Fischer, G.S. (September, January 31). Collaborative suturing: A reinforcement learning approach to automate hand-off task in suturing for surgical robots. Proceedings of the 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Naples, Italy.","DOI":"10.1109\/RO-MAN47096.2020.9223543"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Li, Z., Cheng, X., Peng, X.B., Abbeel, P., Levine, S., Berseth, G., and Sreenath, K. (June, January 20). Reinforcement learning for robust parameterized locomotion control of bipedal robots. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9560769"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4909","DOI":"10.1109\/TITS.2021.3054625","article-title":"Deep reinforcement learning for autonomous driving: A survey","volume":"23","author":"Kiran","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., and Levine, S. (2018, January 26\u201330). Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. Proceedings of the Robotics: Science and Systems (RSS), Pittsburgh, PA, USA.","DOI":"10.15607\/RSS.2018.XIV.049"},{"key":"ref_6","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"698","DOI":"10.1177\/0278364920987859","article-title":"How to train your robot with deep reinforcement learning: Lessons we have learned","volume":"40","author":"Ibarz","year":"2021","journal-title":"Int. J. Robot. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2300000053","article-title":"An algorithmic perspective on imitation learning","volume":"7","author":"Osa","year":"2018","journal-title":"Found. Trends\u00ae Robot."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Maitin-Shepard, J., Cusumano-Towner, M., Lei, J., and Abbeel, P. (2010, January 3\u20137). Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding. Proceedings of the 2010 IEEE International Conference on Robotics and Automation (ICRA), Anchorage, AK, USA.","DOI":"10.1109\/ROBOT.2010.5509439"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Doumanoglou, A., Kargakos, A., Kim, T.K., and Malassiotis, S. (June, January 31). Autonomous active recognition and unfolding of clothes using random decision forests and probabilistic planning. Proceedings of the 2014 IEEE international conference on robotics and automation (ICRA), Hong Kong, China.","DOI":"10.1109\/ICRA.2014.6906974"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"82","DOI":"10.3389\/frobt.2020.00082","article-title":"Modeling of deformable objects for robotic manipulation: A tutorial and review","volume":"7","author":"Guler","year":"2020","journal-title":"Front. Robot. AI"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1177\/0278364918779698","article-title":"Robotic manipulation and sensing of deformable objects in domestic and industrial applications: A survey","volume":"37","author":"Sanchez","year":"2018","journal-title":"Int. J. Robot. Res."},{"key":"ref_13","unstructured":"Henrich, D., and W\u00f6rn, H. (2012). Robot Manipulation of Deformable Objects, Springer Science & Business Media."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1108\/01439910210440255","article-title":"Industrial applications of automatic manipulation of flexible materials","volume":"29","author":"Saadat","year":"2002","journal-title":"Ind. Robot. Int. J."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.rcim.2011.08.002","article-title":"Survey on model-based manipulation planning of deformable objects","volume":"28","year":"2012","journal-title":"Robot.-Comput.-Integr. Manuf."},{"key":"ref_16","unstructured":"Khalil, F.F., and Payeur, P. (2010). Dexterous Robotic Manipulation of Deformable Objects with Multi-Sensory Feedback\u2014A Review, INTECH Open Access Publisher."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1109\/MRA.2022.3147415","article-title":"Challenges and outlook in robotic manipulation of deformable objects","volume":"29","author":"Zhu","year":"2022","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1109\/TRO.2020.2986921","article-title":"A grasping-centered analysis for cloth manipulation","volume":"36","author":"Torras","year":"2020","journal-title":"IEEE Trans. Robot."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"eabd8803","DOI":"10.1126\/scirobotics.abd8803","article-title":"Modeling, learning, perception, and control methods for deformable object manipulation","volume":"6","author":"Yin","year":"2021","journal-title":"Sci. Robot."},{"key":"ref_20","unstructured":"Matas, J., James, S., and Davison, A.J. (2018, January 29\u201331). Sim-to-real reinforcement learning for deformable object manipulation. Proceedings of the Conference on Robot Learning, Zurich, Switzerland."},{"key":"ref_21","unstructured":"Seita, D., Jamali, N., Laskey, M., Tanwani, A.K., Berenstein, R., Baskaran, P., Iba, S., Canny, J., and Goldberg, K. (2019, January 6\u201310). Deep transfer learning of pick points on fabric for robot bed-making. Proceedings of the The International Symposium of Robotics Research, Hanoi, Vietnam."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, Y., Hu, X., Xu, D., Yue, Y., Grinspun, E., and Allen, P.K. (2016, January 16\u201321). Multi-sensor surface analysis for robotic ironing. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487788"},{"key":"ref_23","unstructured":"Wang, W., Berenson, D., and Balkcom, D. (2015, January 26\u201330). An online method for tight-tolerance insertion tasks for string and rope. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Jackson, R.C., Desai, V., Castillo, J.P., and \u00c7avu\u015fo\u011flu, M.C. (2016, January 9\u201314). Needle-tissue interaction force state estimation for robotic surgical suturing. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Republic of Korea.","DOI":"10.1109\/IROS.2016.7759539"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1088\/0370-1301\/64\/9\/301","article-title":"The mechanical properties of metals","volume":"64","author":"Mott","year":"1951","journal-title":"Proc. Phys. Soc. Sect. B"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2601097.2601152","article-title":"Unified particle physics for real-time applications","volume":"33","author":"Macklin","year":"2014","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_27","unstructured":"Li, Y., Wu, J., Tedrake, R., Tenenbaum, J.B., and Torralba, A. (2019, January 20\u201324). Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids. Proceedings of the 2019 IEEE International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1145\/1073204.1073216","article-title":"Meshless deformations based on shape matching","volume":"24","author":"Heidelberger","year":"2005","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_29","unstructured":"Lin, X., Wang, Y., Olkin, J., and Held, D. (2021, January 8\u201311). Softgym: Benchmarking deep reinforcement learning for deformable object manipulation. Proceedings of the Conference on Robot Learning (CoRL), London, UK."},{"key":"ref_30","unstructured":"(2023, February 15). NVIDIA PhysX 4.5 and 5.0 SDK. Available online: https:\/\/developer.nvidia.com\/physx-sdk."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2746","DOI":"10.1109\/LRA.2020.2972852","article-title":"Learning to collaborate from simulation for robot-assisted dressing","volume":"5","author":"Clegg","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3272127.3275048","article-title":"Learning to dress: Synthesizing human dressing motion via deep reinforcement learning","volume":"37","author":"Clegg","year":"2018","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_33","unstructured":"Coumans, E., and Bai, Y. (2023, February 15). PyBullet, a Python Module for Physics Simulation for Games, Robotics and Machine Learning. Available online: http:\/\/pybullet.org."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Seita, D., Florence, P., Tompson, J., Coumans, E., Sindhwani, V., Goldberg, K., and Zeng, A. (June, January 30). Learning to rearrange deformable cables, fabrics, and bags with goal-conditioned transporter networks. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561391"},{"key":"ref_35","unstructured":"Community, B.O. (2018). Blender\u2014A 3D Modelling and Rendering Package, Blender Foundation, Stichting Blender Foundation."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sundaresan, P., Grannen, J., Thananjeyan, B., Balakrishna, A., Ichnowski, J., Novoseller, E.R., Hwang, M., Laskey, M., Gonzalez, J.E., and Goldberg, K. (2021). Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies. arXiv.","DOI":"10.15607\/RSS.2021.XVII.013"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hoque, R., Seita, D., Balakrishna, A., Ganapathi, A., Tanwani, A.K., Jamali, N., Yamane, K., Iba, S., and Goldberg, K. (2020). VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation. arXiv.","DOI":"10.15607\/RSS.2020.XVI.034"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1007\/s10514-021-10001-0","article-title":"Visuospatial foresight for physical sequential fabric manipulation","volume":"46","author":"Hoque","year":"2022","journal-title":"Auton. Robot."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Todorov, E., Erez, T., and Tassa, Y. (2012, January 7\u201312). MuJoCo: A physics engine for model-based control. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6386109"},{"key":"ref_40","unstructured":"Faure, F., Duriez, C., Delingette, H., Allard, J., Gilles, B., Marchesseau, S., Talbot, H., Courtecuisse, H., Bousquet, G., and Peterlik, I. (2012). Soft Tissue Biomechanical Modeling for Computer Assisted Surgery, Springer."},{"key":"ref_41","unstructured":"Studio, V.M. (2023, February 15). Unified Particle Physics for Unity. Available online: http:\/\/obi.virtualmethodstudio.com\/."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Weng, Z., Paus, F., Varava, A., Yin, H., Asfour, T., and Kragic, D. (October, January 27). Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects. Proceedings of the 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.","DOI":"10.1109\/IROS51168.2021.9636660"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Fan, Z., Shao, W., Hayashi, T., and Ohashi, T. (2022). Untying cable by combining 3D deep neural network with deep reinforcement learning. Adv. Robot., 1\u201315.","DOI":"10.1080\/01691864.2022.2126729"},{"key":"ref_44","unstructured":"Narain, R., Samii, A., Pfaff, T., and O\u2019Brien, J. (2014). ARCSim: Adaptive Refining and Coarsening Simulator, University of California."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Yang, S., Liang, J., and Lin, M.C. (2017, January 22\u201329). Learning-based cloth material recovery from video. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.470"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Bai, Y., and Liu, C.K. (2014, January 3\u20135). Coupling cloth and rigid bodies for dexterous manipulation. Proceedings of the Seventh International Conference on Motion in Games, Guanajuato, Mexico.","DOI":"10.1145\/2668064.2668066"},{"key":"ref_47","first-page":"1395","article-title":"A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms","volume":"22","author":"Kroemer","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Kroemer, O., Ugur, E., Oztop, E., and Peters, J. (2012, January 14\u201318). A kernel-based approach to direct action perception. Proceedings of the 2012 IEEE international Conference on Robotics and Automation (ICRA), St. Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224957"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"538","DOI":"10.7210\/jrsj.3.538","article-title":"Hand eye coordination in rope handling","volume":"3","author":"Inaba","year":"1985","journal-title":"J. Robot. Soc. Jpn."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/S0921-8890(99)00115-3","article-title":"Planning strategy for task of unfolding clothes","volume":"32","author":"Hamajima","year":"2000","journal-title":"Robot. Auton. Syst."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"618","DOI":"10.20965\/jrm.2006.p0618","article-title":"Clothes folding task by tool-using robot","volume":"18","author":"Osawa","year":"2006","journal-title":"J. Robot. Mechatronics"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Yamakawa, Y., Namiki, A., and Ishikawa, M. (2010, January 18\u201322). Motion planning for dynamic knotting of a flexible rope with a high-speed robot arm. Proceedings of the 2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan.","DOI":"10.1109\/IROS.2010.5651168"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Tassa, Y., Mansard, N., and Todorov, E. (June, January 31). Control-limited differential dynamic programming. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.","DOI":"10.1109\/ICRA.2014.6907001"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Scholz, J., and Stilman, M. (2010, January 6\u20138). Combining motion planning and optimization for flexible robot manipulation. Proceedings of the 2010 10th IEEE-RAS International Conference on Humanoid Robots, Nashville, TN, USA.","DOI":"10.1109\/ICHR.2010.5686849"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Erickson, Z., Clegg, A., Yu, W., Turk, G., Liu, C.K., and Kemp, C.C. (June, January 29). What does the person feel? Learning to infer applied forces during robot-assisted dressing. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989718"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Gao, Y., Hendricks, L.A., Kuchenbecker, K.J., and Darrell, T. (2016, January 16\u201321). Deep learning for tactile understanding from visual and haptic data. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487176"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Lee, M.A., Zhu, Y., Srinivasan, K., Shah, P., Savarese, S., Fei-Fei, L., Garg, A., and Bohg, J. (2019, January 20\u201324). Making sense of vision and touch: Self-supervised learning of multimodal representations for contact-rich tasks. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793485"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Sung, J., Lenz, I., and Saxena, A. (June, January 29). Deep multimodal embedding: Manipulating novel objects with point-clouds, language and trajectories. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989325"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Pan, Y., Cheng, C.A., Saigol, K., Lee, K., Yan, X., Theodorou, E.A., and Boots, B. (2017). Agile Autonomous Driving using End-to-End Deep Imitation Learning. arXiv.","DOI":"10.15607\/RSS.2018.XIV.056"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Codevilla, F., M\u00fcller, M., L\u00f3pez, A., Koltun, V., and Dosovitskiy, A. (2018, January 21\u201325). End-to-end driving via conditional imitation learning. Proceedings of the 2018 International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460487"},{"key":"ref_61","unstructured":"Morita, T., Takamatsu, J., Ogawara, K., Kimura, H., and Ikeuchi, K. (2003, January 14\u201319). Knot planning from observation. Proceedings of the 2003 IEEE International Conference on Robotics and Automation (Cat. No. 03CH37422), Taipei, Taiwan."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Zhao, W., Queralta, J.P., and Westerlund, T. (2020, January 1\u20134). Sim-to-real transfer in deep reinforcement learning for robotics: A survey. Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence (SSCI), Canberra, Australia.","DOI":"10.1109\/SSCI47803.2020.9308468"},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Vithayathil Varghese, N., and Mahmoud, Q.H. (2020). A survey of multi-task deep reinforcement learning. Electronics, 9.","DOI":"10.3390\/electronics9091363"},{"key":"ref_64","unstructured":"Stooke, A., Lee, K., Abbeel, P., and Laskin, M. (2021, January 18\u201324). Decoupling representation learning from reinforcement learning. Proceedings of the International Conference on Machine Learning (ICML), online."},{"key":"ref_65","unstructured":"Pong, V.H., Nair, A.V., Smith, L.M., Huang, C., and Levine, S. (2022, January 17\u201323). Offline meta-reinforcement learning with online self-supervision. Proceedings of the International Conference on Machine Learning, Baltimore, MA, USA."},{"key":"ref_66","first-page":"14656","article-title":"What matters for adversarial imitation learning?","volume":"34","author":"Orsini","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_67","first-page":"4565","article-title":"Generative adversarial imitation learning","volume":"29","author":"Ho","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_68","first-page":"3016","article-title":"Visual adversarial imitation learning using variational models","volume":"34","author":"Rafailov","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_69","first-page":"679","article-title":"A Markovian decision process","volume":"6","author":"Bellman","year":"1957","journal-title":"J. Math. Mech."},{"key":"ref_70","unstructured":"Howard, R.A. (1960). Dynamic Programming and Markov Processes, John Wiley."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MASSP.1986.1165342","article-title":"An introduction to hidden Markov models","volume":"3","author":"Rabiner","year":"1986","journal-title":"IEEE ASSP Mag."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum likelihood from incomplete data via the EM algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Kapusta, A., Yu, W., Bhattacharjee, T., Liu, C.K., Turk, G., and Kemp, C.C. (2016, January 26\u201331). Data-driven haptic perception for robot-assisted dressing. Proceedings of the 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), New York, NY, USA.","DOI":"10.1109\/ROMAN.2016.7745158"},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Cusumano-Towner, M., Singh, A., Miller, S., O\u2019Brien, J.F., and Abbeel, P. (July, January 29). Bringing clothing into desired configurations with limited perception. Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), Zurich, Switzerland.","DOI":"10.1109\/ICRA.2011.5980327"},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/S0004-3702(98)00023-X","article-title":"Planning and acting in partially observable stochastic domains","volume":"101","author":"Kaelbling","year":"1998","journal-title":"Artif. Intell."},{"key":"ref_76","unstructured":"Hallak, A., Castro, D.D., and Mannor, S. (2015). Contextual Markov Decision Processes. arXiv."},{"key":"ref_77","doi-asserted-by":"crossref","unstructured":"Detry, R., Ek, C.H., Madry, M., and Kragic, D. (2013, January 6\u201310). Learning a dictionary of prototypical grasp-predicting parts from grasping experience. Proceedings of the 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6630635"},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0004-3702(99)00052-1","article-title":"Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning","volume":"112","author":"Sutton","year":"1999","journal-title":"Artif. Intell."},{"key":"ref_79","unstructured":"Eysenbach, B., Gupta, A., Ibarz, J., and Levine, S. (2019, January 6\u20139). Diversity is All You Need: Learning Skills without a Reward Function. Proceedings of the International Conference on Learning Representations (ICLR), New Orleans, LA, USA."},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Pastor, P., Hoffmann, H., Asfour, T., and Schaal, S. (2009, January 12\u201317). Learning and generalization of motor skills by learning from demonstration. Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA), Kobe, Japan.","DOI":"10.1109\/ROBOT.2009.5152385"},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. (2017, January 24\u201328). Domain randomization for transferring deep neural networks from simulation to the real world. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202133"},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1613\/jair.1.11436","article-title":"Blind spot detection for safe sim-to-real transfer","volume":"67","author":"Ramakrishnan","year":"2020","journal-title":"J. Artif. Intell. Res."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"1033","DOI":"10.1109\/21.179842","article-title":"System identification and control using genetic algorithms","volume":"22","author":"Kristinsson","year":"1992","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1109\/JPROC.2020.3004555","article-title":"A comprehensive survey on transfer learning","volume":"109","author":"Zhuang","year":"2020","journal-title":"Proc. IEEE"},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"911","DOI":"10.1109\/LRA.2021.3052391","article-title":"Data-efficient domain randomization with bayesian optimization","volume":"6","author":"Muratore","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1109\/TMECH.2015.2465849","article-title":"A review of algorithms for compliant control of stiff and fixed-compliance robots","volume":"21","author":"Calanca","year":"2015","journal-title":"IEEE\/ASME Trans. Mechatronics"},{"key":"ref_87","first-page":"1437","article-title":"A comprehensive survey on safe reinforcement learning","volume":"16","year":"2015","journal-title":"J. Mach. Learn. Res."},{"key":"ref_88","first-page":"1040","article-title":"Learning from demonstration","volume":"9","author":"Schaal","year":"1996","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_89","doi-asserted-by":"crossref","unstructured":"Russell, S. (1998, January 24\u201326). Learning agents for uncertain environments. Proceedings of the Eleventh Annual Conference on Computational Learning Theory, Madison, WI, USA.","DOI":"10.1145\/279943.279964"},{"key":"ref_90","first-page":"51","article-title":"When Is a Linear Control System Optimal","volume":"86","year":"1963","journal-title":"J. Basic Eng."},{"key":"ref_91","doi-asserted-by":"crossref","unstructured":"Van Vinh, T., Tomizawa, T., Kudoh, S., and Suehiro, T. (2012, January 14\u201318). A new strategy for making a knot with a general-purpose arm. Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA), St. Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224852"},{"key":"ref_92","unstructured":"Suzuki, T., Ebihara, Y., and Shintani, K. (2005, January 18\u201321). Dynamic analysis of casting and winding with hyper-flexible manipulator. Proceedings of the ICAR\u201905, 12th International Conference on Advanced Robotics, Seattle, WA, USA."},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Das, N., Bechtle, S., Davchev, T., Jayaraman, D., Rai, A., and Meier, F. (2021, January 8\u201311). Model-based inverse reinforcement learning from visual demonstrations. Proceedings of the Conference on Robot Learning (CoRL), London, UK.","DOI":"10.1109\/ICRA48506.2021.9561396"},{"key":"ref_94","doi-asserted-by":"crossref","unstructured":"Furukawa, K., Michie, D., and Muggleton, S. (1999). Machine Intelligence 15, Oxford University Press.","DOI":"10.1093\/oso\/9780198538677.001.0001"},{"key":"ref_95","unstructured":"Baram, N., Anschel, O., Caspi, I., and Mannor, S. (2017, January 17\u201323). End-to-end differentiable adversarial imitation learning. Proceedings of the International Conference on Machine Learning, Baltimore, MA, USA."},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Nair, A., Chen, D., Agrawal, P., Isola, P., Abbeel, P., Malik, J., and Levine, S. (June, January 29). Combining self-supervised learning and imitation for vision-based rope manipulation. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989247"},{"key":"ref_97","unstructured":"Billard, A., Calinon, S., Dillmann, R., and Schaal, S. (2008). Springer Handbook of Robotics, Springer."},{"key":"ref_98","unstructured":"Attia, A., and Dayan, S. (2018). Global overview of imitation learning. arXiv."},{"key":"ref_99","first-page":"205","article-title":"Alvinn: An autonomous land vehicle in a neural network","volume":"1","author":"Pomerleau","year":"1988","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_100","unstructured":"Bagnell, J.A. (2015). An Invitation to Imitation, Carnegie-Mellon Univ Pittsburgh Pa Robotics Inst. Technical Report."},{"key":"ref_101","unstructured":"Ross, S., Gordon, G., and Bagnell, D. (2011, January 11\u201313). A reduction of imitation learning and structured prediction to no-regret online learning. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_102","first-page":"3149","article-title":"Imitation learning by coaching","volume":"25","author":"He","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_103","doi-asserted-by":"crossref","unstructured":"Venkatraman, A., Hebert, M., and Bagnell, J.A. (2015, January 25\u201330). Improving multi-step prediction of learned time series models. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA.","DOI":"10.1609\/aaai.v29i1.9590"},{"key":"ref_104","unstructured":"Zeng, A., Florence, P., Tompson, J., Welker, S., Chien, J., Attarian, M., Armstrong, T., Krasin, I., Duong, D., and Sindhwani, V. (2021, January 8\u201311). Transporter networks: Rearranging the visual world for robotic manipulation. Proceedings of the Conference on Robot Learning (CoRL), London, UK."},{"key":"ref_105","doi-asserted-by":"crossref","unstructured":"Kudoh, S., Gomi, T., Katano, R., Tomizawa, T., and Suehiro, T. (October, January 28). In-air knotting of rope by a dual-arm multi-finger robot. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7354262"},{"key":"ref_106","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/MRA.2010.936947","article-title":"Learning and reproduction of gestures by imitation","volume":"17","author":"Calinon","year":"2010","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_107","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.artint.2009.11.011","article-title":"Hidden semi-Markov models","volume":"174","author":"Yu","year":"2010","journal-title":"Artif. Intell."},{"key":"ref_108","doi-asserted-by":"crossref","first-page":"30","DOI":"10.3389\/frobt.2016.00030","article-title":"Learning controllers for reactive and proactive behaviors in human\u2014Robot collaboration","volume":"3","author":"Rozo","year":"2016","journal-title":"Front. Robot. AI"},{"key":"ref_109","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/S0005-1098(01)00174-1","article-title":"The explicit linear quadratic regulator for constrained systems","volume":"38","author":"Bemporad","year":"2002","journal-title":"Automatica"},{"key":"ref_110","first-page":"1547","article-title":"Learning attractor landscapes for learning motor primitives","volume":"15","author":"Ijspeert","year":"2002","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_111","unstructured":"Ijspeert, A.J., Nakanishi, J., and Schaal, S. (2002, January 11\u201315). Movement imitation with nonlinear dynamical systems in humanoid robots. Proceedings of the 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), Washington, DC, USA."},{"key":"ref_112","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1162\/NECO_a_00393","article-title":"Dynamical movement primitives: Learning attractor models for motor behaviors","volume":"25","author":"Ijspeert","year":"2013","journal-title":"Neural Comput."},{"key":"ref_113","doi-asserted-by":"crossref","unstructured":"Dario, P., and Chatila, R. (2005). Robotics Research. The Eleventh International Symposium: With 303 Figures, Springer.","DOI":"10.1007\/b97958"},{"key":"ref_114","first-page":"2616","article-title":"Probabilistic movement primitives","volume":"26","author":"Paraschos","year":"2013","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_115","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1109\/TRO.2011.2159412","article-title":"Learning stable nonlinear dynamical systems with gaussian mixture models","volume":"27","author":"Billard","year":"2011","journal-title":"IEEE Trans. Robot."},{"key":"ref_116","unstructured":"Schulman, J., Ho, J., Lee, C., and Abbeel, P. (2016). Robotics Research, Springer."},{"key":"ref_117","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/S1077-3142(03)00009-2","article-title":"A new point matching algorithm for non-rigid registration","volume":"89","author":"Chui","year":"2003","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_118","doi-asserted-by":"crossref","unstructured":"Lee, A.X., Lu, H., Gupta, A., Levine, S., and Abbeel, P. (2015, January 26\u201330). Learning force-based manipulation of deformable objects from multiple demonstrations. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7138997"},{"key":"ref_119","doi-asserted-by":"crossref","unstructured":"Lee, A.X., Goldstein, M.A., Barratt, S.T., and Abbeel, P. (2015, January 26\u201330). A non-rigid point and normal registration algorithm with applications to learning from demonstrations. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139289"},{"key":"ref_120","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_121","unstructured":"Bertsekas, D. (2012). Dynamic Programming and Optimal Control: Volume I, Athena Scientific."},{"key":"ref_122","unstructured":"Bertsekas, D. (2019). Reinforcement Learning and Optimal Control, Athena Scientific."},{"key":"ref_123","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_124","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016, January 2\u20134). Continuous control with deep reinforcement learning. Proceedings of the International Conference on Learning Representations (ICLR), San Juan, Puerto Rico."},{"key":"ref_125","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10\u201315). Soft actor\u2013critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the International Conference on Machine Learning (ICML), Stockholm, Sweden."},{"key":"ref_126","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_127","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_128","unstructured":"Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing function approximation error in actor\u2013critic methods. Proceedings of the International Conference on Machine Learning (ICML), Stockholm, Sweden."},{"key":"ref_129","first-page":"1057","article-title":"Policy gradient methods for reinforcement learning with function approximation","volume":"12","author":"Sutton","year":"1999","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_130","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_131","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1016\/j.neunet.2008.02.003","article-title":"Reinforcement learning of motor skills with policy gradients","volume":"21","author":"Peters","year":"2008","journal-title":"Neural Netw."},{"key":"ref_132","doi-asserted-by":"crossref","first-page":"1180","DOI":"10.1016\/j.neucom.2007.11.026","article-title":"Natural actor\u2013critic","volume":"71","author":"Peters","year":"2008","journal-title":"Neurocomputing"},{"key":"ref_133","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1162\/089976698300017746","article-title":"Natural gradient works efficiently in learning","volume":"10","author":"Amari","year":"1998","journal-title":"Neural Comput."},{"key":"ref_134","unstructured":"Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015, January 6\u201311). Trust region policy optimization. Proceedings of the International Conference on Machine Learning (ICML), Lille, France."},{"key":"ref_135","unstructured":"Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D.d.L., Budden, D., Abdolmaleki, A., Merel, J., and Lefrancq, A. (2018). Deepmind control suite. arXiv."},{"key":"ref_136","unstructured":"Laskin, M., Srinivas, A., and Abbeel, P. (2020, January 12\u201318). Curl: Contrastive unsupervised representations for reinforcement learning. Proceedings of the International Conference on Machine Learning (ICML), Vienna, Austria."},{"key":"ref_137","unstructured":"Zhang, A., McAllister, R.T., Calandra, R., Gal, Y., and Levine, S. (2021, January 3\u20137). Learning Invariant Representations for Reinforcement Learning without Reconstruction. Proceedings of the International Conference on Learning Representations (ICLR), online."},{"key":"ref_138","unstructured":"Oord, A.v.d., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv."},{"key":"ref_139","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation learning: A review and new perspectives","volume":"35","author":"Bengio","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_140","unstructured":"Yarats, D., Kostrikov, I., and Fergus, R. (2021, January 3\u20137). Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels. Proceedings of the International Conference on Learning Representations (ICLR), online."},{"key":"ref_141","first-page":"19884","article-title":"Reinforcement learning with augmented data","volume":"33","author":"Laskin","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_142","doi-asserted-by":"crossref","unstructured":"Hansen, N., and Wang, X. (June, January 30). Generalization in reinforcement learning by soft data augmentation. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561103"},{"key":"ref_143","unstructured":"Hjelm, R.D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., and Bengio, Y. (2019, January 6\u20139). Learning deep representations by mutual information estimation and maximization. Proceedings of the International Conference on Learning Representations (ICLR), New Orleans, LA, USA."},{"key":"ref_144","unstructured":"Hafner, D., Lillicrap, T., Ba, J., and Norouzi, M. (2020, January 26\u201330). Dream to Control: Learning Behaviors by Latent Imagination. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia."},{"key":"ref_145","unstructured":"Hafner, D., Lillicrap, T.P., Norouzi, M., and Ba, J. (2021, January 3\u20137). Mastering Atari with Discrete World Models. Proceedings of the International Conference on Learning Representations (ICLR), online."},{"key":"ref_146","unstructured":"Zhang, M., Vikram, S., Smith, L., Abbeel, P., Johnson, M., and Levine, S. (2019, January 10\u201315). Solar: Deep structured representations for model-based reinforcement learning. Proceedings of the International Conference on Machine Learning (ICML), Long Beach, CA, USA."},{"key":"ref_147","first-page":"741","article-title":"Stochastic latent actor\u2013critic: Deep reinforcement learning with a latent variable model","volume":"33","author":"Lee","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_148","unstructured":"Seo, Y., Lee, K., James, S.L., and Abbeel, P. (2022, January 17\u201323). Reinforcement learning with action-free pre-training from videos. Proceedings of the International Conference on Machine Learning (ICML), Baltimore, MA, USA."},{"key":"ref_149","doi-asserted-by":"crossref","unstructured":"Yarats, D., Zhang, A., Kostrikov, I., Amos, B., Pineau, J., and Fergus, R. (2021, January 2\u20139). Improving Sample Efficiency in Model-Free Reinforcement Learning from Images. Proceedings of the AAAI Conference on Artificial Intelligence, online.","DOI":"10.1609\/aaai.v35i12.17276"},{"key":"ref_150","first-page":"8769","article-title":"Unsupervised state representation learning in atari","volume":"32","author":"Anand","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_151","first-page":"11890","article-title":"Predictive information accelerates learning in rl","volume":"33","author":"Lee","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_152","doi-asserted-by":"crossref","unstructured":"Castro, P.S. (2020, January 7\u201312). Scalable methods for computing state similarity in deterministic markov decision processes. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i06.6564"},{"key":"ref_153","doi-asserted-by":"crossref","unstructured":"Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., and Abbeel, P. (2017). Asymmetric actor critic for image-based robot learning. arXiv.","DOI":"10.15607\/RSS.2018.XIV.008"},{"key":"ref_154","unstructured":"Badia, A.P., Sprechmann, P., Vitvitskyi, A., Guo, D., Piot, B., Kapturowski, S., Tieleman, O., Arjovsky, M., Pritzel, A., and Bolt, A. (2020). Never give up: Learning directed exploration strategies. arXiv."},{"key":"ref_155","unstructured":"Badia, A.P., Piot, B., Kapturowski, S., Sprechmann, P., Vitvitskyi, A., Guo, Z.D., and Blundell, C. (2020, January 12\u201318). Agent57: Outperforming the atari human benchmark. Proceedings of the International Conference on Machine Learning (ICML), Vienna, Austria."},{"key":"ref_156","unstructured":"Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., and Davidson, J. (2019, January 10\u201315). Learning latent dynamics for planning from pixels. Proceedings of the International Conference on Machine Learning (ICML), Long Beach, CA, USA."},{"key":"ref_157","unstructured":"Lee, A.X., Zhang, R., Ebert, F., Abbeel, P., Finn, C., and Levine, S. (2018). Stochastic adversarial video prediction. arXiv."},{"key":"ref_158","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1038\/s41586-020-03051-4","article-title":"Mastering atari, go, chess and shogi by planning with a learned model","volume":"588","author":"Schrittwieser","year":"2020","journal-title":"Nature"},{"key":"ref_159","unstructured":"Shelhamer, E., Mahmoudieh, P., Argus, M., and Darrell, T. (2016). Loss is its own Reward: Self-Supervision for Reinforcement Learning. arXiv."},{"key":"ref_160","doi-asserted-by":"crossref","unstructured":"Song, X., Jiang, Y., Tu, S., Du, Y., and Neyshabur, B. (2020, January 26\u201330). Observational Overfitting in Reinforcement Learning. Proceedings of the International Conference on Learning Representations (ICLR), online.","DOI":"10.1109\/ICASSP40776.2020.9053257"},{"key":"ref_161","unstructured":"Packer, C., Gao, K., Kos, J., Kr\u00e4henb\u00fchl, P., Koltun, V., and Song, D. (2018). Assessing generalization in deep reinforcement learning. arXiv."},{"key":"ref_162","unstructured":"Henaff, O. (2020, January 12\u201318). Data-efficient image recognition with contrastive predictive coding. Proceedings of the International Conference on Machine Learning (ICML), Vienna, Austria."},{"key":"ref_163","unstructured":"Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020, January 12\u201318). A simple framework for contrastive learning of visual representations. Proceedings of the International Conference on Machine Learning (ICML), Vienna, Austria."},{"key":"ref_164","doi-asserted-by":"crossref","unstructured":"He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 13\u201319). Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"ref_165","doi-asserted-by":"crossref","first-page":"193907","DOI":"10.1109\/ACCESS.2020.3031549","article-title":"Contrastive representation learning: A framework and review","volume":"8","author":"Healy","year":"2020","journal-title":"IEEE Access"},{"key":"ref_166","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv."},{"key":"ref_167","unstructured":"Moerland, T.M., Broekens, J., and Jonker, C.M. (2020). Model-based reinforcement learning: A survey. arXiv."},{"key":"ref_168","unstructured":"Camacho, E.F., and Alba, C.B. (2013). Model Predictive Control, Springer Science & Business Media."},{"key":"ref_169","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TCIAIG.2012.2186810","article-title":"A survey of monte carlo tree search methods","volume":"4","author":"Browne","year":"2012","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_170","unstructured":"Moerland, T.M., Broekens, J., and Jonker, C.M. (2020). A framework for reinforcement learning and planning. arXiv."},{"key":"ref_171","doi-asserted-by":"crossref","unstructured":"Finn, C., and Levine, S. (June, January 29). Deep visual foresight for planning robot motion. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989324"},{"key":"ref_172","unstructured":"Yang, Y., Caluwaerts, K., Iscen, A., Zhang, T., Tan, J., and Sindhwani, V. (2020, January 14\u201316). Data efficient reinforcement learning for legged robots. Proceedings of the Conference on Robot Learning (CoRL), Cambridge, MA, USA."},{"key":"ref_173","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1207\/s15516709cog1603_1","article-title":"Forward models: Supervised learning with a distal teacher","volume":"16","author":"Jordan","year":"1992","journal-title":"Cogn. Sci."},{"key":"ref_174","unstructured":"Ha, D., and Schmidhuber, J. (2018). World models. arXiv."},{"key":"ref_175","unstructured":"Ebert, F., Finn, C., Dasari, S., Xie, A., Lee, A.X., and Levine, S. (2018). Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control. arXiv."},{"key":"ref_176","unstructured":"Babaeizadeh, M., Finn, C., Erhan, D., Campbell, R.H., and Levine, S. (May, January 30). Stochastic Variational Video Prediction. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_177","unstructured":"Denton, E., and Fergus, R. (2018, January 10\u201315). Stochastic video generation with a learned prior. Proceedings of the International Conference on Machine Learning (ICML), Stockholm, Sweden."},{"key":"ref_178","unstructured":"Chung, J., G\u00fcl\u00e7ehre, \u00c7., Cho, K., and Bengio, Y. (2014). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv."},{"key":"ref_179","unstructured":"Lin, X., Wang, Y., Huang, Z., and Held, D. (2022, January 15\u201318). Learning visible connectivity dynamics for cloth smoothing. Proceedings of the Conference on Robot Learning (CoRL), Auckland, New Zealand."},{"key":"ref_180","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana","year":"1997","journal-title":"Mach. Learn."},{"key":"ref_181","first-page":"819","article-title":"Sparse multi-task reinforcement learning","volume":"27","author":"Calandriello","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_182","first-page":"5048","article-title":"Hindsight experience replay","volume":"30","author":"Andrychowicz","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_183","unstructured":"Chane-Sane, E., Schmid, C., and Laptev, I. (2021, January 18\u201324). Goal-conditioned reinforcement learning with imagined subgoals. Proceedings of the International Conference on Machine Learning (ICML), online."},{"key":"ref_184","first-page":"15324","article-title":"Goal-conditioned imitation learning","volume":"32","author":"Ding","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_185","first-page":"1094","article-title":"Learning to achieve goals","volume":"2","author":"Kaelbling","year":"1993","journal-title":"IJCAI"},{"key":"ref_186","unstructured":"Lin, X., Baweja, H.S., and Held, D. (2019). Reinforcement learning without ground-truth state. arXiv."},{"key":"ref_187","first-page":"10265","article-title":"Policy continuation with hindsight inverse dynamics","volume":"32","author":"Sun","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_188","unstructured":"Eysenbach, B., Salakhutdinov, R., and Levine, S. (2021, January 3\u20137). C-Learning: Learning to Achieve Goals via Recursive Classification. Proceedings of the International Conference on Learning Representations (ICLR), online."},{"key":"ref_189","unstructured":"Schaul, T., Horgan, D., Gregor, K., and Silver, D. (2015, January 6\u201311). Universal value function approximators. Proceedings of the International Conference on Machine Learning (ICML), Lille, France."},{"key":"ref_190","first-page":"4026","article-title":"Deep exploration via bootstrapped DQN","volume":"29","author":"Osband","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_191","unstructured":"Mankowitz, D.J., Z\u00eddek, A., Barreto, A., Horgan, D., Hessel, M., Quan, J., Oh, J., van Hasselt, H., Silver, D., and Schaul, T. (2018). Unicorn: Continual Learning with a Universal, Off-policy Agent. arXiv."},{"key":"ref_192","unstructured":"Achiam, J., and Sastry, S. (2017). Surprise-based intrinsic motivation for deep reinforcement learning. arXiv."},{"key":"ref_193","unstructured":"McFarlane, R. (2018). A Survey of Exploration Strategies in Reinforcement Learning, McGill University."},{"key":"ref_194","unstructured":"Amin, S., Gomrokchi, M., Satija, H., van Hoof, H., and Precup, D. (2021). A survey of exploration methods in reinforcement learning. arXiv."},{"key":"ref_195","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.inffus.2022.03.003","article-title":"Exploration in deep reinforcement learning: A survey","volume":"85","author":"Ladosz","year":"2022","journal-title":"Inf. Fusion"},{"key":"ref_196","doi-asserted-by":"crossref","unstructured":"Lattimore, T., and Szepesv\u00e1ri, C. (2020). Bandit Algorithms, Cambridge University Press.","DOI":"10.1017\/9781108571401"},{"key":"ref_197","first-page":"397","article-title":"Using confidence bounds for exploitation-exploration trade-offs","volume":"3","author":"Auer","year":"2002","journal-title":"J. Mach. Learn. Res."},{"key":"ref_198","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1093\/biomet\/25.3-4.285","article-title":"On the likelihood that one unknown probability exceeds another in view of the evidence of two samples","volume":"25","author":"Thompson","year":"1933","journal-title":"Biometrika"},{"key":"ref_199","unstructured":"Aubret, A., Matignon, L., and Hassas, S. (2019). A survey on intrinsic motivation in reinforcement learning. arXiv."},{"key":"ref_200","doi-asserted-by":"crossref","unstructured":"Schmidhuber, J. (1991, January 14). A possibility for implementing curiosity and boredom in model-building neural controllers. Proceedings of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats, Paris, France.","DOI":"10.7551\/mitpress\/3115.003.0030"},{"key":"ref_201","unstructured":"Osband, I., and Van Roy, B. (2017, January 6\u201311). Why is posterior sampling better than optimism for reinforcement learning?. Proceedings of the International Conference on Machine Learning (ICML), Sydney, Australia."},{"key":"ref_202","doi-asserted-by":"crossref","unstructured":"Azizzadenesheli, K., Brunskill, E., and Anandkumar, A. (2018, January 11\u201316). Efficient exploration through bayesian deep q-networks. Proceedings of the 2018 Information Theory and Applications Workshop (ITA), San Diego, CA, USA.","DOI":"10.1109\/ITA.2018.8503252"},{"key":"ref_203","unstructured":"Plappert, M., Houthooft, R., Dhariwal, P., Sidor, S., Chen, R.Y., Chen, X., Asfour, T., Abbeel, P., and Andrychowicz, M. (May, January 30). Parameter Space Noise for Exploration. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_204","unstructured":"Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., and Fergus, R. (May, January 30). Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_205","unstructured":"Lee, L., Eysenbach, B., Parisotto, E., Xing, E., Levine, S., and Salakhutdinov, R. (2019). Efficient exploration via state marginal matching. arXiv."},{"key":"ref_206","unstructured":"Hazan, E., Kakade, S., Singh, K., and Van Soest, A. (2019, January 10\u201315). Provably efficient maximum entropy exploration. Proceedings of the International Conference on Machine Learning (ICML), Long Beach, CA, USA."},{"key":"ref_207","unstructured":"Yang, T., Tang, H., Bai, C., Liu, J., Hao, J., Meng, Z., and Liu, P. (2021). Exploration in deep reinforcement learning: A comprehensive survey. arXiv."},{"key":"ref_208","unstructured":"Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K.O., and Clune, J. (2019). Go-Explore: A New Approach for Hard-Exploration Problems. arXiv."},{"key":"ref_209","unstructured":"Jiang, N., Krishnamurthy, A., Agarwal, A., Langford, J., and Schapire, R.E. (2017, January 6\u201311). Contextual decision processes with low bellman rank are pac-learnable. Proceedings of the International Conference on Machine Learning (ICML), Sydney, Australia."},{"key":"ref_210","doi-asserted-by":"crossref","unstructured":"Machado, M.C., Bellemare, M.G., and Bowling, M. (2020, January 7\u201312). Count-based exploration with the successor representation. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i04.5955"},{"key":"ref_211","unstructured":"Burda, Y., Edwards, H., Storkey, A., and Klimov, O. (2018). Exploration by random network distillation. arXiv."},{"key":"ref_212","first-page":"9191","article-title":"Visual reinforcement learning with imagined goals","volume":"31","author":"Nair","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_213","unstructured":"Pong, V.H., Dalal, M., Lin, S., Nair, A., Bahl, S., and Levine, S. (2019). Skew-fit: State-covering self-supervised reinforcement learning. arXiv."},{"key":"ref_214","unstructured":"Lopes, M., Lang, T., Toussaint, M., and Oudeyer, P.Y. (2012, January 3\u20138). Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress. Proceedings of the Neural Information Processing Systems Conference (NIPS), Lake Tahoe, NV, USA."},{"key":"ref_215","unstructured":"Ziebart, B.D., Maas, A.L., Bagnell, J.A., and Dey, A.K. (2008, January 13\u201317). Maximum entropy inverse reinforcement learning. Proceedings of the AAAI, Chicago, IL, USA."},{"key":"ref_216","unstructured":"Levine, S. (2018). Reinforcement learning and control as probabilistic inference: Tutorial and review. arXiv."},{"key":"ref_217","unstructured":"Haarnoja, T., Tang, H., Abbeel, P., and Levine, S. (2017, January 6\u201311). Reinforcement learning with deep energy-based policies. Proceedings of the International Conference on Machine Learning (ICML), Sydney, Australia."},{"key":"ref_218","doi-asserted-by":"crossref","unstructured":"Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., and Osband, I. (2018, January 2\u20137). Deep q-learning from demonstrations. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11757"},{"key":"ref_219","unstructured":"Vecer\u00edk, M., Hester, T., Scholz, J., Wang, F., Pietquin, O., Piot, B., Heess, N., Roth\u00f6rl, T., Lampe, T., and Riedmiller, M.A. (2017). Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards. arXiv."},{"key":"ref_220","doi-asserted-by":"crossref","unstructured":"Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., and Abbeel, P. (2018, January 21\u201325). Overcoming exploration in reinforcement learning with demonstrations. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8463162"},{"key":"ref_221","doi-asserted-by":"crossref","unstructured":"Chebotar, Y., Kalakrishnan, M., Yahya, A., Li, A., Schaal, S., and Levine, S. (June, January 29). Path integral guided policy search. Proceedings of the 2017 IEEE international conference on robotics and automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989384"},{"key":"ref_222","doi-asserted-by":"crossref","unstructured":"Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., and Silver, D. (2018, January 2\u20137). Rainbow: Combining improvements in deep reinforcement learning. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11796"},{"key":"ref_223","doi-asserted-by":"crossref","unstructured":"Piot, B., Geist, M., and Pietquin, O. (2014, January 15\u201319). Boosted bellman residual minimization handling expert demonstrations. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Nancy, France.","DOI":"10.1007\/978-3-662-44851-9_35"},{"key":"ref_224","doi-asserted-by":"crossref","unstructured":"Takizawa, M., Yao, Z., Onda, H., Kudoh, S., and Suehiro, T. (2019, January 14\u201316). Learning from observation of tabletop knotting using a simple task model. Proceedings of the 2019 IEEE\/SICE International Symposium on System Integration (SII), Paris, France.","DOI":"10.1109\/SII.2019.8700429"},{"key":"ref_225","doi-asserted-by":"crossref","unstructured":"Wu, Y., Yan, W., Kurutach, T., Pinto, L., and Abbeel, P. (2019). Learning to manipulate deformable objects without demonstrations. arXiv.","DOI":"10.15607\/RSS.2020.XVI.065"},{"key":"ref_226","doi-asserted-by":"crossref","unstructured":"Jangir, R., Aleny\u00e0, G., and Torras, C. (August, January 31). Dynamic cloth manipulation with deep reinforcement learning. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9196659"},{"key":"ref_227","unstructured":"Ha, H., and Song, S. (2022, January 15\u201318). Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding. Proceedings of the Conference on Robot Learning (CoRL), Auckland, New Zealand."},{"key":"ref_228","unstructured":"Hietala, J., Blanco-Mulero, D., Alcan, G., and Kyrki, V. (2021). Closing the Sim2Real Gap in Dynamic Cloth Manipulation. arXiv."},{"key":"ref_229","unstructured":"Lee, R., Ward, D., Dasagi, V., Cosgun, A., Leitner, J., and Corke, P. (2021, January 8\u201311). Learning arbitrary-goal fabric folding with one hour of real robot experience. Proceedings of the Conference on Robot Learning (CoRL), London, UK."},{"key":"ref_230","doi-asserted-by":"crossref","unstructured":"Clegg, A., Yu, W., Erickson, Z., Tan, J., Liu, C.K., and Turk, G. (2017, January 24\u201328). Learning to navigate cloth using haptics. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206110"},{"key":"ref_231","doi-asserted-by":"crossref","unstructured":"Gonnochenko, A., Semochkin, A., Egorov, D., Statovoy, D., Zabihifar, S., Postnikov, A., Seliverstova, E., Zaidi, A., Stemmler, J., and Limkrailassiri, K. (2021, January 4\u20136). Coinbot: Intelligent Robotic Coin Bag Manipulation Using Artificial Brain. Proceedings of the 2021 7th International Conference on Automation, Robotics and Applications (ICARA), Prague, Czech Republic.","DOI":"10.1109\/ICARA51699.2021.9376455"},{"key":"ref_232","unstructured":"Xu, Z., Chi, C., Burchfiel, B., Cousineau, E., Feng, S., and Song, S. (July, January 27). DextAIRity: Deformable Manipulation Can be a Breeze. Proceedings of the Robotics: Science and Systems (RSS), New York, NY, USA."},{"key":"ref_233","unstructured":"Yan, W., Vangipuram, A., Abbeel, P., and Pinto, L. (2021, January 8\u201311). Learning predictive representations for deformable objects using contrastive estimation. Proceedings of the Conference on Robot Learning (CoRL), London, UK."},{"key":"ref_234","unstructured":"Ma, X., Hsu, D., and Lee, W.S. (2021). Learning Latent Graph Dynamics for Deformable Object Manipulation. arXiv."},{"key":"ref_235","unstructured":"Arnold, S., Tanaka, D., and Yamazaki, K. (2021). Cloth Manipulation Planning on Basis of Mesh Representations with Incomplete Domain Knowledge and Voxel-to-Mesh Estimation. arXiv."},{"key":"ref_236","doi-asserted-by":"crossref","unstructured":"Seita, D., Ganapathi, A., Hoque, R., Hwang, M., Cen, E., Tanwani, A.K., Balakrishna, A., Thananjeyan, B., Ichnowski, J., and Jamali, N. (2020, January 25\u201329). Deep imitation learning of sequential fabric smoothing from an algorithmic supervisor. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341608"},{"key":"ref_237","unstructured":"Weng, T., Bajracharya, S.M., Wang, Y., Agrawal, K., and Held, D. (2022, January 15\u201318). Fabricflownet: Bimanual cloth manipulation with a flow-based policy. Proceedings of the Conference on Robot Learning (CoRL), Auckland, New Zealand."},{"key":"ref_238","doi-asserted-by":"crossref","unstructured":"Teng, Y., Lu, H., Li, Y., Kamiya, T., Nakatoh, Y., Serikawa, S., and Gao, P. (2022). Multidimensional Deformable Object Manipulation Based on DN-Transporter Networks. IEEE Trans. Intell. Transp. Syst.","DOI":"10.1109\/TITS.2022.3168303"},{"key":"ref_239","doi-asserted-by":"crossref","unstructured":"Suzuki, K., Kanamura, M., Suga, Y., Mori, H., and Ogata, T. (October, January 27). In-air knotting of rope using dual-arm robot based on deep learning. Proceedings of the 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.","DOI":"10.1109\/IROS51168.2021.9635954"},{"key":"ref_240","unstructured":"Grannen, J., Sundaresan, P., Thananjeyan, B., Ichnowski, J., Balakrishna, A., Hwang, M., Viswanath, V., Laskey, M., Gonzalez, J.E., and Goldberg, K. (2020). Untangling dense knots by learning task-relevant keypoints. arXiv."},{"key":"ref_241","doi-asserted-by":"crossref","unstructured":"Viswanath, V., Grannen, J., Sundaresan, P., Thananjeyan, B., Balakrishna, A., Novoseller, E., Ichnowski, J., Laskey, M., Gonzalez, J.E., and Goldberg, K. (October, January 27). Disentangling Dense Multi-Cable Knots. Proceedings of the 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.","DOI":"10.1109\/IROS51168.2021.9636397"},{"key":"ref_242","unstructured":"Seita, D., Kerr, J., Canny, J., and Goldberg, K. (October, January 27). Initial Results on Grasping and Lifting Physical Deformable Bags with a Bimanual Robot. Proceedings of the IROS Workshop on Robotic Manipulation of Deformable Objects in Real-World Applications, Prague, Czech Republic."},{"key":"ref_243","doi-asserted-by":"crossref","unstructured":"Lee, A.X., Huang, S.H., Hadfield-Menell, D., Tzeng, E., and Abbeel, P. (2014, January 14\u201318). Unifying scene registration and trajectory optimization for learning from demonstrations with application to manipulation of deformable objects. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6943185"},{"key":"ref_244","doi-asserted-by":"crossref","unstructured":"Huang, S.H., Pan, J., Mulcaire, G., and Abbeel, P. (October, January 28). Leveraging appearance priors in non-rigid registration, with application to manipulation of deformable objects. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353475"},{"key":"ref_245","unstructured":"Tallec, C., Blier, L., and Ollivier, Y. (2019, January 10\u201315). Making deep q-learning methods robust to time discretization. Proceedings of the International Conference on Machine Learning (ICML), Long Beach, CA, USA."},{"key":"ref_246","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1080\/01691864.2013.777012","article-title":"Reinforcement learning of a motor skill for wearing a T-shirt using topology coordinates","volume":"27","author":"Matsubara","year":"2013","journal-title":"Adv. Robot."},{"key":"ref_247","doi-asserted-by":"crossref","unstructured":"Colom\u00e9, A., Planells, A., and Torras, C. (2015, January 26\u201330). A friction-model-based framework for reinforcement learning of robotic tasks in non-rigid environments. Proceedings of the 2015 IEEE international conference on robotics and automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139990"},{"key":"ref_248","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.robot.2017.03.017","article-title":"Learning adaptive dressing assistance from human demonstration","volume":"93","author":"Pignat","year":"2017","journal-title":"Robot. Auton. Syst."},{"key":"ref_249","doi-asserted-by":"crossref","first-page":"1156","DOI":"10.1080\/01691864.2019.1636715","article-title":"A framework for robotic clothing assistance by imitation learning","volume":"33","author":"Joshi","year":"2019","journal-title":"Adv. Robot."},{"key":"ref_250","doi-asserted-by":"crossref","unstructured":"Sundaresan, P., Grannen, J., Thananjeyan, B., Balakrishna, A., Laskey, M., Stone, K., Gonzalez, J.E., and Goldberg, K. (August, January 31). Learning rope manipulation policies using dense object descriptors trained on synthetic depth data. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197121"},{"key":"ref_251","doi-asserted-by":"crossref","first-page":"244","DOI":"10.20965\/jrm.1998.p0244","article-title":"Planning strategy for task untangling laundry-isolating clothes from a washed mass","volume":"10","author":"Hamajima","year":"1998","journal-title":"Robotics Mechatron."},{"key":"ref_252","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1177\/0278364911430417","article-title":"A geometric approach to robotic laundry folding","volume":"31","author":"Miller","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_253","doi-asserted-by":"crossref","first-page":"1461","DOI":"10.1109\/TRO.2016.2602376","article-title":"Folding clothes autonomously: A complete pipeline","volume":"32","author":"Doumanoglou","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_254","unstructured":"Kaneko, M., and Kakikura, M. (2001, January 28\u201329). Planning strategy for putting away laundry-isolating and unfolding task. Proceedings of the 2001 IEEE International Symposium on Assembly and Task Planning (ISATP2001). Assembly and Disassembly in the Twenty-First Century (Cat. No. 01TH8560), Fukuoka, Japan."},{"key":"ref_255","doi-asserted-by":"crossref","unstructured":"Willimon, B., Birchfield, S., and Walker, I. (2011, January 25\u201330). Classification of clothing using interactive perception. Proceedings of the 2011 IEEE International Conference on Robotics and Automation (IROS), San Francisco, CA, USA.","DOI":"10.1109\/ICRA.2011.5980336"},{"key":"ref_256","doi-asserted-by":"crossref","unstructured":"Sun, L., Aragon-Camarasa, G., Cockshott, P., Rogers, S., and Siebert, J.P. (2013, January 28\u201330). A heuristic-based approach for flattening wrinkled clothes. Proceedings of the Conference Towards Autonomous Robotic Systems (TAROS), Oxford, UK.","DOI":"10.1007\/978-3-662-43645-5_16"},{"key":"ref_257","doi-asserted-by":"crossref","unstructured":"Bersch, C., Pitzer, B., and Kammel, S. (2011, January 25\u201330). Bimanual robotic cloth manipulation for laundry folding. Proceedings of the 2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6095109"},{"key":"ref_258","unstructured":"Yamazaki, K., and Inaba, M. (2009, January 20\u201322). A Cloth Detection Method Based on Image Wrinkle Feature for Daily Assistive Robots. Proceedings of the International Conference on Machine Vision Applications (MVA), Yokohama, Japan."},{"key":"ref_259","doi-asserted-by":"crossref","unstructured":"Ramisa, A., Alenya, G., Moreno-Noguer, F., and Torras, C. (2012, January 14\u201318). Using depth and appearance features for informed robot grasping of highly wrinkled clothes. Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA), St. Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6225045"},{"key":"ref_260","doi-asserted-by":"crossref","unstructured":"Sun, L., Aragon-Camarasa, G., Rogers, S., and Siebert, J.P. (2015, January 26\u201330). Accurate garment surface analysis using an active stereo robot head with application to dual-arm flattening. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7138998"},{"key":"ref_261","doi-asserted-by":"crossref","unstructured":"Willimon, B., Birchfield, S., and Walker, I. (2011, January 25\u201330). Model for unfolding laundry using interactive perception. Proceedings of the 2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6095066"},{"key":"ref_262","unstructured":"Bell, M. (2010). Flexible Object Manipulation, Dartmouth College."},{"key":"ref_263","unstructured":"Berg, J.v.d., Miller, S., Goldberg, K., and Abbeel, P. (2010). Algorithmic Foundations of Robotics IX, Springer."},{"key":"ref_264","unstructured":"Farin, G. (2014). Curves and Surfaces for Computer-Aided Geometric Design: A Practical Guide, Elsevier."},{"key":"ref_265","doi-asserted-by":"crossref","unstructured":"Miller, S., Fritz, M., Darrell, T., and Abbeel, P. (July, January 29). Parametrized shape models for clothing. Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), Zurich, Switzerland.","DOI":"10.1109\/ICRA.2011.5980453"},{"key":"ref_266","doi-asserted-by":"crossref","unstructured":"Stria, J., Pr\u016f\u0161a, D., Hlav\u00e1\u010d, V., Wagner, L., Petr\u00edk, V., Krsek, P., and Smutn\u00fd, V. (2014, January 14\u201318). Garment perception and its folding using a dual-arm robot. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6942541"},{"key":"ref_267","doi-asserted-by":"crossref","unstructured":"Huang, Z., Lin, X., and Held, D. (2022). Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation. arXiv.","DOI":"10.15607\/RSS.2022.XVIII.011"},{"key":"ref_268","doi-asserted-by":"crossref","unstructured":"Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Van Der Smagt, P., Cremers, D., and Brox, T. (2015, January 7\u201313). Flownet: Learning optical flow with convolutional networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.316"},{"key":"ref_269","unstructured":"Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A., and Battaglia, P. (2021, January 3\u20137). Learning Mesh-Based Simulation with Graph Networks. Proceedings of the International Conference on Learning Representations (ICLR), online."},{"key":"ref_270","unstructured":"Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., Ying, R., Leskovec, J., and Battaglia, P. (2020, January 12\u201318). Learning to simulate complex physics with graph networks. Proceedings of the International Conference on Machine Learning (ICML), Vienna, Austria."},{"key":"ref_271","doi-asserted-by":"crossref","first-page":"8775","DOI":"10.1109\/LRA.2022.3187843","article-title":"Learning Deformable Object Manipulation from Expert Demonstrations","volume":"7","author":"Salhotra","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_272","doi-asserted-by":"crossref","unstructured":"Hoque, R., Shivakumar, K., Aeron, S., Deza, G., Ganapathi, A., Wong, A., Lee, J., Zeng, A., Vanhoucke, V., and Goldberg, K. (2022, January 23\u201327). Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics Research. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9981253"},{"key":"ref_273","unstructured":"Wong, A., Zeng, A., Bose, A., Wahid, A., Kalashnikov, D., Krasin, I., Varley, J., Lee, J., Tompson, J., and Attarian, M. (2023, February 15). PyReach\u2014Python Client SDK for Robot Remote Control. Available online: https:\/\/github.com\/google-research\/pyreach."},{"key":"ref_274","unstructured":"Crowell, R.H., and Fox, R.H. (2012). Introduction to Knot Theory, Springer Science & Business Media."},{"key":"ref_275","unstructured":"Wakamatsu, H., Tsumaya, A., Arai, E., and Hirai, S. (May, January 26). Planning of one-handed knotting\/raveling manipulation of linear objects. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), New Orleans, LA, USA."},{"key":"ref_276","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1142\/S021821659400023X","article-title":"Energy functions for polygonal knots","volume":"3","author":"Simon","year":"1994","journal-title":"J. Knot Theory Its Ramifications"},{"key":"ref_277","unstructured":"Scharein, R.G. (1998). Interactive Topological Drawing. [Ph.D. Thesis, University of British Columbia]."},{"key":"ref_278","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1142\/S0218216594000356","article-title":"Recognizing knots using simulated annealing","volume":"3","author":"Ligocki","year":"1994","journal-title":"J. Knot Theory Its Ramif."},{"key":"ref_279","unstructured":"Huang, M., Grzeszczuk, R.P., and Kauffman, L.H. (November, January 27). Untangling knots by stochastic energy optimization. Proceedings of the Seventh Annual IEEE Visualization\u201996, San Francisco, CA, USA."},{"key":"ref_280","doi-asserted-by":"crossref","first-page":"797","DOI":"10.1177\/0278364904045469","article-title":"Using motion planning for knot untangling","volume":"23","author":"Ladd","year":"2004","journal-title":"Int. J. Robot. Res."},{"key":"ref_281","unstructured":"Wakamatsu, H., Tsumaya, A., Arai, E., and Hirai, S. (2006, January 15\u201319). Manipulation planning for unraveling linear objects. Proceedings of the 2006 IEEE International Conference on Robotics and Automation (ICRA), Orlando, FL, USA."},{"key":"ref_282","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1177\/0278364906064819","article-title":"Knotting\/unknotting manipulation of deformable linear objects","volume":"25","author":"Wakamatsu","year":"2006","journal-title":"Int. J. Robot. Res."},{"key":"ref_283","unstructured":"Reidemeister, K. (1983). Knot Theory, BCS Associates."},{"key":"ref_284","doi-asserted-by":"crossref","unstructured":"Yamakawa, Y., Namiki, A., Ishikawa, M., and Shimojo, M. (2008, January 22\u201326). Knotting manipulation of a flexible rope by a multifingered hand system based on skill synthesis. Proceedings of the 2008 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Nice, France.","DOI":"10.1109\/IROS.2008.4650802"},{"key":"ref_285","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1109\/TMECH.2006.878557","article-title":"Manipulation of deformable linear objects using knot invariants to classify the object condition based on image sensor information","volume":"11","author":"Matsuno","year":"2006","journal-title":"IEEE\/ASME Trans. Mechatronics"},{"key":"ref_286","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1177\/027836499101000105","article-title":"A case study of flexible object manipulation","volume":"10","author":"Hopcroft","year":"1991","journal-title":"Int. J. Robot. Res."},{"key":"ref_287","unstructured":"Sundaresan, P., Goldberg, K., and Gonzalez, J. (2021). Robotic Untangling and Disentangling of Cables via Learned Manipulation and Recovery Strategies. [Master\u2019s Thesis, University of Berkeley]."},{"key":"ref_288","unstructured":"Wang, W., and Balkcom, D. (2016, January 16\u201321). Tying knot precisely. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden."},{"key":"ref_289","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1140\/epjb\/e2008-00443-y","article-title":"Curvature and torsion of the tight closed trefoil knot","volume":"66","author":"Baranska","year":"2008","journal-title":"Eur. Phys. J. B"},{"key":"ref_290","unstructured":"Rawdon, E.J. (1998). Ideal Knots, World Scientific."},{"key":"ref_291","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1080\/10586458.2011.544581","article-title":"Knot tightening by constrained gradient descent","volume":"20","author":"Ashton","year":"2011","journal-title":"Exp. Math."},{"key":"ref_292","unstructured":"Carlen, M., Laurie, B., Maddocks, J.H., and Smutny, J. (2005). Physical and Numerical Models in Knot Theory: Including Applications to the Life Sciences, World Scientific."},{"key":"ref_293","unstructured":"Fink, T., and Mao, Y. (2000). The 85 Ways to Tie a Tie: The Science and Aesthetics of Tie Knots, Broadway."},{"key":"ref_294","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/0166-8641(83)90004-4","article-title":"Classification of knot projections","volume":"16","author":"Dowker","year":"1983","journal-title":"Topol. Its Appl."},{"key":"ref_295","doi-asserted-by":"crossref","unstructured":"Schulman, J., Gupta, A., Venkatesan, S., Tayson-Frederick, M., and Abbeel, P. (2013, January 3\u20137). A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696945"},{"key":"ref_296","unstructured":"Lui, W.H., and Saxena, A. (2013, January 3\u20137). Tangled: Learning to untangle ropes with rgb-d perception. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan."},{"key":"ref_297","unstructured":"(2023, February 15). I-Dress: Assistive Interactive Robotic System for Support in Dressing. Available online: https:\/\/i-dress-project.iri.upc.edu\/."},{"key":"ref_298","doi-asserted-by":"crossref","unstructured":"Yamazaki, K., Oya, R., Nagahama, K., Okada, K., and Inaba, M. (2014, January 13\u201315). Bottom dressing by a life-sized humanoid robot provided failure detection and recovery functions. Proceedings of the 2014 IEEE\/SICE International Symposium on System Integration, Tokyo, Japan.","DOI":"10.1109\/SII.2014.7028101"},{"key":"ref_299","doi-asserted-by":"crossref","unstructured":"Klee, S.D., Ferreira, B.Q., Silva, R., Costeira, J.P., Melo, F.S., and Veloso, M. (2015, January 26\u201330). Personalized assistance for dressing users. Proceedings of the International Conference on Social Robotics (ICSR), Paris, France.","DOI":"10.1007\/978-3-319-25554-5_36"},{"key":"ref_300","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1109\/TRO.2017.2691721","article-title":"Bayesian nonparametric learning of cloth models for real-time state estimation","volume":"33","author":"Koganti","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_301","doi-asserted-by":"crossref","unstructured":"Chance, G., Camilleri, A., Winstone, B., Caleb-Solly, P., and Dogramadzi, S. (2016, January 6\u201329). An assistive robot to support dressing-strategies for planning and error handling. Proceedings of the 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), Singapore.","DOI":"10.1109\/BIOROB.2016.7523721"},{"key":"ref_302","doi-asserted-by":"crossref","unstructured":"Li, S., Figueroa, N., Shah, A.J., and Shah, J.A. (2021, January 12\u201316). Provably Safe and Efficient Motion Planning with Uncertain Human Dynamics. Proceedings of the Robotics: Science and Systems (RSS), online.","DOI":"10.15607\/RSS.2021.XVII.050"},{"key":"ref_303","doi-asserted-by":"crossref","unstructured":"Zhang, F., Cully, A., and Demiris, Y. (2017, January 24\u201328). Personalized robot-assisted dressing using user modeling in latent spaces. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206206"},{"key":"ref_304","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1109\/TRO.2019.2904461","article-title":"Probabilistic real-time user posture tracking for personalized robot-assisted dressing","volume":"35","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Robot."},{"key":"ref_305","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1002\/rob.20073","article-title":"Safe planning for human-robot interaction","volume":"22","author":"Croft","year":"2005","journal-title":"J. Robot. Syst."},{"key":"ref_306","doi-asserted-by":"crossref","unstructured":"Gao, Y., Chang, H.J., and Demiris, Y. (2016, January 9\u201314). Iterative path optimisation for personalised dressing assistance using vision and force information. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Republic of Korea.","DOI":"10.1109\/IROS.2016.7759647"},{"key":"ref_307","doi-asserted-by":"crossref","first-page":"2245","DOI":"10.1109\/LRA.2018.2812912","article-title":"Tracking human pose during robot-assisted dressing using single-axis capacitive proximity sensing","volume":"3","author":"Erickson","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_308","doi-asserted-by":"crossref","unstructured":"Schiavi, R., Bicchi, A., and Flacco, F. (2009, January 12\u201317). Integration of active and passive compliance control for safe human-robot coexistence. Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA), Kobe, Japan.","DOI":"10.1109\/ROBOT.2009.5152571"},{"key":"ref_309","doi-asserted-by":"crossref","first-page":"5","DOI":"10.5772\/61930","article-title":"Bottom dressing by a dual-arm robot using a clothing state estimation based on dynamic shape changes","volume":"13","author":"Yamazaki","year":"2016","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_310","doi-asserted-by":"crossref","unstructured":"Koganti, N., Tamei, T., Matsubara, T., and Shibata, T. (2013, January 4\u20136). Estimation of human cloth topological relationship using depth sensor for robotic clothing assistance. Proceedings of the Conference on Advances in Robotics, Pune, India.","DOI":"10.1145\/2506095.2506146"},{"key":"ref_311","doi-asserted-by":"crossref","unstructured":"Koganti, N., Tamei, T., Matsubara, T., and Shibata, T. (2014, January 25\u201329). Real-time estimation of human-cloth topological relationship using depth sensor for robotic clothing assistance. Proceedings of the The 23rd IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, UK.","DOI":"10.1109\/ROMAN.2014.6926241"},{"key":"ref_312","doi-asserted-by":"crossref","unstructured":"Koganti, N., Ngeo, J.G., Tomoya, T., Ikeda, K., and Shibata, T. (October, January 28). Cloth dynamics modeling in latent spaces and its application to robotic clothing assistance. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353860"},{"key":"ref_313","unstructured":"Gao, Y., Chang, H.J., and Demiris, Y. (October, January 28). User modelling for personalised dressing assistance by humanoid robots. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany."},{"key":"ref_314","doi-asserted-by":"crossref","unstructured":"Erickson, Z., Clever, H.M., Turk, G., Liu, C.K., and Kemp, C.C. (2018, January 21\u201325). Deep haptic model predictive control for robot-assisted dressing. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460656"},{"key":"ref_315","doi-asserted-by":"crossref","unstructured":"Yamazaki, K., Oya, R., Nagahama, K., and Inaba, M. (2013, January 15\u201317). A method of state recognition of dressing clothes based on dynamic state matching. Proceedings of the 2013 IEEE\/SICE International Symposium on System Integration, Kobe, Japan.","DOI":"10.1109\/SII.2013.6776728"},{"key":"ref_316","doi-asserted-by":"crossref","first-page":"13","DOI":"10.3389\/frobt.2017.00013","article-title":"A quantitative analysis of dressing dynamics for robotic dressing assistance","volume":"4","author":"Chance","year":"2017","journal-title":"Front. Robot. AI"},{"key":"ref_317","doi-asserted-by":"crossref","unstructured":"Yu, W., Kapusta, A., Tan, J., Kemp, C.C., Turk, G., and Liu, C.K. (June, January 29). Haptic simulation for robot-assisted dressing. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989716"},{"key":"ref_318","doi-asserted-by":"crossref","unstructured":"Shinohara, D., Matsubara, T., and Kidode, M. (2011, January 7\u201311). Learning motor skills with non-rigid materials by reinforcement learning. Proceedings of the 2011 IEEE International Conference on Robotics and Biomimetics, Karon Beach, Thailand.","DOI":"10.1109\/ROBIO.2011.6181709"},{"key":"ref_319","doi-asserted-by":"crossref","first-page":"1281","DOI":"10.1016\/S0893-6080(96)00043-3","article-title":"A kendama learning robot based on bi-directional theory","volume":"9","author":"Miyamoto","year":"1996","journal-title":"Neural Netw."},{"key":"ref_320","first-page":"3137","article-title":"A generalized path integral control approach to reinforcement learning","volume":"11","author":"Theodorou","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_321","unstructured":"Eickeler, S., Kosmala, A., and Rigoll, G. (1998, January 20). Hidden markov model based continuous online gesture recognition. Proceedings of the Fourteenth International Conference on Pattern Recognition (Cat. No. 98EX170), Brisbane, Australia."},{"key":"ref_322","first-page":"4555","article-title":"A survey on curriculum learning","volume":"44","author":"Wang","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_323","doi-asserted-by":"crossref","unstructured":"Twardon, L., and Ritter, H. (2016, January 9\u201314). Active boundary component models for robotic dressing assistance. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Republic of Korea.","DOI":"10.1109\/IROS.2016.7759436"},{"key":"ref_324","doi-asserted-by":"crossref","unstructured":"Kirchheim, A., Burwinkel, M., and Echelmeyer, W. (2008, January 3). Automatic unloading of heavy sacks from containers. Proceedings of the 2008 IEEE International Conference on Automation and Logistics, Qingdao, China.","DOI":"10.1109\/ICAL.2008.4636286"},{"key":"ref_325","unstructured":"Huang, G., Chen, D., Li, T., Wu, F., van der Maaten, L., and Weinberger, K. (May, January 30). Multi-Scale Dense Networks for Resource Efficient Image Classification. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_326","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA."},{"key":"ref_327","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1022140919877","article-title":"Recent advances in hierarchical reinforcement learning","volume":"13","author":"Barto","year":"2003","journal-title":"Discret. Event Dyn. Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2389\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:38:33Z","timestamp":1760121513000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2389"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,21]]},"references-count":327,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052389"],"URL":"https:\/\/doi.org\/10.3390\/s23052389","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,21]]}}}