{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T02:59:21Z","timestamp":1775789961890,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,6,17]],"date-time":"2020-06-17T00:00:00Z","timestamp":1592352000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Learning skills from kinesthetic demonstrations is a promising way of minimizing the gap between human manipulation abilities and those of robots. We propose an approach to learn sequential force interaction skills from such demonstrations. The demonstrations are decomposed into a set of movement primitives by inferring the underlying sequential structure of the task. The decomposition is based on a novel probability distribution which we call Directional Normal Distribution. The distribution allows infering the movement primitive\u2019s composition, i.e., its coordinate frames, control variables and target coordinates from the demonstrations. In addition, it permits determining an appropriate number of movement primitives for a task via model selection. After finding the task\u2019s composition, the system learns to sequence the resulting movement primitives in order to be able to reproduce the task on a real robot. We evaluate the approach on three different tasks, unscrewing a light bulb, box stacking and box flipping. All tasks are kinesthetically demonstrated and then reproduced on a Barrett WAM robot.<\/jats:p>","DOI":"10.3390\/robotics9020045","type":"journal-article","created":{"date-parts":[[2020,6,17]],"date-time":"2020-06-17T13:11:32Z","timestamp":1592399492000},"page":"45","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Learning Sequential Force Interaction Skills"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6419-8157","authenticated-orcid":false,"given":"Simon","family":"Manschitz","sequence":"first","affiliation":[{"name":"Honda Research Institute Europe, 63073 Offenbach, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8036-2519","authenticated-orcid":false,"given":"Michael","family":"Gienger","sequence":"additional","affiliation":[{"name":"Honda Research Institute Europe, 63073 Offenbach, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7257-5434","authenticated-orcid":false,"given":"Jens","family":"Kober","sequence":"additional","affiliation":[{"name":"Cognitive Robotics Department, Delft University of Technology, 2628 CD Delft, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5266-8091","authenticated-orcid":false,"given":"Jan","family":"Peters","sequence":"additional","affiliation":[{"name":"Institute for Intelligent Autonomous Systems, Technische Universit\u00e4t Darmstadt, 64289 Darmstadt, Germany"},{"name":"Max Planck Institute for Intelligent Systems, 72076 T\u00fcbingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Manschitz, S., Kober, J., Gienger, M., and Peters, J. (2016, January 9\u201314). Probabilistic Decomposition of Sequential Force Interaction Tasks into Movement Primitives. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759577"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Manschitz, S., Kober, J., Gienger, M., and Peters, J. (2014, January 14\u201318). Learning to Sequence Movement Primitives from Demonstrations. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6943187"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.robot.2015.07.005","article-title":"Learning movement primitive attractor goals and sequential skills from kinesthetic demonstrations","volume":"74","author":"Manschitz","year":"2015","journal-title":"Robot. Auton. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lioutikov, R., Neumann, G., Maeda, G., and Peters, J. (2015, January 3\u20135). Probabilistic Segmentation Applied to an Assembly Task. Proceedings of the 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, Korea.","DOI":"10.1109\/HUMANOIDS.2015.7363584"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lemme, A., Reinhart, R.F., and Steil, J.J. (2014, January 18\u201320). Self-supervised bootstrapping of a movement primitive library from complex trajectories. Proceedings of the 2014 IEEE-RAS International Conference on Humanoid Robots, Madrid, Spain.","DOI":"10.1109\/HUMANOIDS.2014.7041443"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"16:1","DOI":"10.1145\/2010325.2010326","article-title":"Emulating Human Observers with Bayesian Binning: Segmentation of Action Streams","volume":"8","author":"Endres","year":"2011","journal-title":"ACM Trans. Appl. Percept."},{"key":"ref_7","unstructured":"Krishnan, S., Garg, A., Patil, S., Lea, C., Hager, G., Abbeel, P., and Goldberg, K. (2015). Transition State Clustering: Unsupervised Surgical Trajectory Segmentation for Robot Learning. Int. J. Robot. Res."},{"key":"ref_8","unstructured":"Lafferty, J.D., McCallum, A., and Pereira, F.C.N. (July, January 28). Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. Proceedings of the 18th International Conference on Machine Learning 2001 (ICML 2001), Williamstown, MA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Baisero, A., Mollard, Y., Lopes, M., Toussaint, M., and Lutkebohle, I. (October, January 28). Temporal Segmentation of Pair-Wise Interaction Phases in Sequential Manipulation Demonstrations. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353415"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s10514-014-9397-9","article-title":"Autonomous framework for segmenting robot trajectories of manipulation task","volume":"38","author":"Lee","year":"2015","journal-title":"Auton. Robots"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"903","DOI":"10.1007\/s10514-015-9501-9","article-title":"A modular approach to learning manipulation strategies from human demonstration","volume":"40","author":"Huang","year":"2016","journal-title":"Auton. Robots"},{"key":"ref_12","unstructured":"Figueroa, N., and Billard, A. (2018, January 29\u201331). A Physically-Consistent Bayesian Non-Parametric Mixture Model for Dynamical System Learning. Proceedings of the 2nd Annual Conference on Robot Learning, Z\u00fcrich, Switzerland."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Gr\u00e4ve, K., and Behnke, S. (2012, January 7\u201312). Incremental action recognition and generalizing motion generation based on goal-directed features. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6386116"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1177\/0278364911426178","article-title":"Incremental learning of full body motion primitives and their sequencing through human motion observation","volume":"31","author":"Ott","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_15","unstructured":"Shiarlis, K., Wulfmeier, M., Salter, S., Whiteson, S., and Posner, I. (2018, January 10\u201315). TACO: Learning Task Decomposition via Temporal Alignment for Control. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1177\/0278364911428653","article-title":"Robot learning from demonstration by constructing skill trees","volume":"31","author":"Konidaris","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Niekum, S., Osentoski, S., Atkeson, C.G., and Barto, A.G. (2015, January 26\u201330). Online Bayesian Changepoint Detection for Articulated Motion Models. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139383"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/MC.2007.154","article-title":"A Language for Human Action","volume":"40","author":"Aloimonos","year":"2007","journal-title":"Computer"},{"key":"ref_19","first-page":"761","article-title":"Movement Segmentation and Recognition for Imitation Learning","volume":"22","author":"Meier","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Gutzeit, L., and Kirchner, E.A. (2016, January 27\u201328). Automatic Detection and Recognition of Human Movement Patterns in Manipulation Tasks. Proceedings of the 3rd International Conference on Physiological Computing Systems, Lissabon, Portugal.","DOI":"10.5220\/0005946500540063"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gutzeit, L., Fabisch, A., Petzoldt, C., Wiese, H., and Kirchner, F. (2019). Automated Robot Skill Learning from Demonstration for Various Robot Systems. KI 2019: Advances in Artificial Intelligence, Springer.","DOI":"10.1007\/978-3-030-30179-8_14"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ranchod, P., Rosman, B., and Konidaris, G. (October, January 28). Nonparametric Bayesian Reward Segmentation for Skill Discovery Using Inverse Reinforcement Learning. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353414"},{"key":"ref_23","unstructured":"Michini, B. (2013). Bayesian Nonparametric Reward Learning from Demonstration. [Ph.D. Thesis, Massachusetts Institute of Technology]."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1177\/0278364918784350","article-title":"SWIRL: A sequential windowed inverse reinforcement learning algorithm for robot tasks with delayed rewards","volume":"38","author":"Krishnan","year":"2019","journal-title":"Int. J. Robot. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1177\/0278364914554471","article-title":"Learning Grounded Finite-State Representations from Unstructured Demonstrations","volume":"34","author":"Niekum","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1109\/TRO.2016.2540623","article-title":"Learning Physical Collaborative Robot Behaviors From Human Demonstrations","volume":"32","author":"Rozo","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1007\/s10514-016-9556-2","article-title":"Probabilistic movement primitives for coordination of multiple human\u2013robot collaborative tasks","volume":"41","author":"Maeda","year":"2017","journal-title":"Auton. Robots"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1291","DOI":"10.1007\/s10514-018-9706-9","article-title":"Kinesthetic teaching and attentional supervision of structured tasks in human\u2013robot interaction","volume":"43","author":"Caccavale","year":"2019","journal-title":"Auton. Robots"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"159022","DOI":"10.1109\/ACCESS.2019.2947529","article-title":"Incremental Learning Introspective Movement Primitives From Multimodal Unstructured Demonstrations","volume":"7","author":"Wu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"833","DOI":"10.1177\/0278364919846363","article-title":"Kernelized movement primitives","volume":"38","author":"Huang","year":"2019","journal-title":"Int. J. Robot. Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2019\/5262859","article-title":"Learning Force-Relevant Skills from Human Demonstration","volume":"2019","author":"Gao","year":"2019","journal-title":"Complexity"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1007\/s10514-015-9435-2","article-title":"Adaptation of manipulation skills in physical contact with the environment to reference force profiles","volume":"39","author":"Nemec","year":"2015","journal-title":"Auton. Robots"},{"key":"ref_33","unstructured":"Ijspeert, A.J., Nakanishi, J., and Schaal, S. (2002, January 11\u201315). Movement imitation with nonlinear dynamical systems in humanoid robots. Proceedings of the 2002 IEEE International Conference on Robotics and Automation, Washington, DC, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Steinmetz, F., Montebelli, A., and Kyrki, V. (2015, January 3\u20135). Simultaneous kinesthetic teaching of positional and force requirements for sequential in-contact tasks. Proceedings of the 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, Korea.","DOI":"10.1109\/HUMANOIDS.2015.7363552"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1458","DOI":"10.1109\/TRO.2015.2495003","article-title":"Task Parameterization Using Continuous Constraints Extracted From Human Demonstrations","volume":"31","author":"Ureche","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Kober, J., Gienger, M., and Steil, J. (2015, January 26\u201330). Learning Movement Primitives for Force Interaction Tasks. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139639"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Pastor, P., Kalakrishnan, M., Righetti, L., and Schaal, S. (December, January 29). Towards Associative Skill Memories. Proceedings of the 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), Osaka, Japan.","DOI":"10.1109\/HUMANOIDS.2012.6651537"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"M\u00fchlig, M., Gienger, M., Steil, J.J., and Goerick, C. (2009, January 10\u201315). Automatic selection of task spaces for imitation learning. Proceedings of the 2009 IEEE\/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA.","DOI":"10.1109\/IROS.2009.5353894"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kappler, D., Pastor, P., Kalakrishnan, M., Wuthrich, M., and Schaal, S. (2015, January 13\u201317). Data-Driven Online Decision Making for Autonomous Manipulation. Proceedings of the Robotics: Science and Systems, Rome, Italy.","DOI":"10.15607\/RSS.2015.XI.044"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Ekvall, S., and Kragic, D. (2006, January 6\u20138). Learning Task Models from Multiple Human Demonstrations. Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication (ROMAN), Hatfield, UK.","DOI":"10.1109\/ROMAN.2006.314460"},{"key":"ref_41","unstructured":"Nicolescu, M.N., and Mataric, M.J. (July, January 14\u201318). Natural Methods for Robot Task Learning: Instructive Demonstrations, Generalization and Practice. Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, Melbourne, Australia."},{"key":"ref_42","unstructured":"Pardowitz, M., Zollner, R., and Dillmann, R. (2005, January 5). Learning sequential Constraints of Tasks from User Demonstrations. Proceedings of the 5th IEEE-RAS International Conference on Humanoid Robots, Tsukuba, Japan."},{"key":"ref_43","unstructured":"Medina, J.R., and Billard, A. (2017, January 13\u201315). Learning Stable Task Sequences from Demonstration with Linear Parameter Varying Systems and Hidden Markov Models. Proceedings of the 1st Annual Conference on Robot Learning, Mountain View, CA, USA."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"650","DOI":"10.1016\/j.conb.2006.10.005","article-title":"Control strategies in object manipulation tasks","volume":"16","author":"Flanagan","year":"2006","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_45","unstructured":"Higham, N. (1989). Matrix Nearness Problems and Applications. Applications of Matrix Theory, Oxford University Press."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"2280","DOI":"10.1016\/j.patcog.2014.01.005","article-title":"Automatic generation and detection of highly reliable fiducial markers under occlusion","volume":"47","year":"2014","journal-title":"Pattern Recognit."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Calinon, S., Li, Z., Alizadeh, T., Tsagarakis, N.G., and Caldwell, D.G. (December, January 29). Statistical dynamical systems for skills acquisition in humanoids. Proceedings of the 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), Osaka, Japan.","DOI":"10.1109\/HUMANOIDS.2012.6651539"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1007\/s12369-012-0160-0","article-title":"Keyframe-based Learning from Demonstration","volume":"4","author":"Akgun","year":"2012","journal-title":"Int. J. Soc. Robot."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/9\/2\/45\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:40:06Z","timestamp":1760175606000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/9\/2\/45"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,17]]},"references-count":48,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["robotics9020045"],"URL":"https:\/\/doi.org\/10.3390\/robotics9020045","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,17]]}}}