{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T16:50:08Z","timestamp":1791219008783,"version":"4.1.0"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T00:00:00Z","timestamp":1764892800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,12,8]],"date-time":"2025-12-08T00:00:00Z","timestamp":1765152000000},"content-version":"vor","delay-in-days":3,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Fraunhofer-Institut f\u00fcr Gie\u00dferei-, Composite undVerarbeitungstechnik IGCV"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Imitation learning provides an intuitive approach for robot programming by enabling robots to learn directly from human demonstrations. While recent visual imitation learning methods have shown promise, they often depend on large datasets, which limits their applicability in manufacturing scenarios where tasks and objects are highly specialized. This paper proposes a one-shot visual imitation learning framework that allows robots to acquire multi-step pick &amp; place tasks from a single video demonstration. The framework integrates hand detection, object detection, trajectory segmentation, and skill learning through Dynamic Movement Primitives (DMPs). Hand trajectories are mapped to the robot\u2019s end-effector, enabling the system to generalize to new object positions while significantly reducing data requirements. The approach is evaluated in simulation and achieves reliable reproduction of multi-step tasks. These results demonstrate the potential of one-shot visual imitation learning to reduce programming complexity and increase flexibility for industrial robot applications.<\/jats:p>","DOI":"10.1038\/s41598-025-30938-x","type":"journal-article","created":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T10:56:48Z","timestamp":1764932208000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Visual imitation learning from one-shot demonstration for multi-step robot pick and place tasks"],"prefix":"10.1038","volume":"15","author":[{"given":"Shuang","family":"Lu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"H\u00e4rdtlein","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johaness","family":"Schilp","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,12,5]]},"reference":[{"key":"30938_CR1","doi-asserted-by":"crossref","unstructured":"Beibl, J., Lee, J., Krause, D. & Moon, S.\u00a0K. Flexibility - grand challenge for product design and production: Review and status. Procedia CIRP, 119 91\u201396, https:\/\/doi.org\/10.1016\/j.procir.2023.05.003 (2023). The 33rd CIRP Design Conference.","DOI":"10.1016\/j.procir.2023.05.003"},{"key":"30938_CR2","doi-asserted-by":"publisher","first-page":"3742","DOI":"10.1109\/LRA.2019.2928782","volume":"4","author":"F Steinmetz","year":"2019","unstructured":"Steinmetz, F., Nitsch, V. & Stulp, F. Intuitive task-level programming by demonstration through semantic skill recognition. IEEE Robotics and Automation Letters 4, 3742\u20133749 (2019).","journal-title":"IEEE Robotics and Automation Letters"},{"key":"30938_CR3","unstructured":"Berg, J.\u00a0K. System zur aufgabenorientierten Programmierung f\u00fcr die Mensch-Roboter-Kooperation. Ph.D. thesis, Technische Universit\u00e4t M\u00fcnchen (2020)."},{"key":"30938_CR4","unstructured":"Saveriano, M. Robotic Tasks Acquisition via Human Guidance: Representation, Learning and Execution. Ph.D. thesis, Technische Universit\u00e4t M\u00fcnchen\u00a0(2017)."},{"key":"30938_CR5","doi-asserted-by":"crossref","unstructured":"Qiu, Z., Eiband, T., Li, S. & Lee, D. Hand pose-based task learning from visual observations with semantic skill extraction. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 596\u2013603 (IEEE, 2020).","DOI":"10.1109\/RO-MAN47096.2020.9223479"},{"key":"30938_CR6","unstructured":"Finn, C., Yu, T., Zhang, T., Abbeel, P. & Levine, S. One-shot visual imitation learning via meta-learning. In Conference on Robot Learning, 357\u2013368 (PMLR, 2017)."},{"key":"30938_CR7","doi-asserted-by":"crossref","unstructured":"Nagarajan, T., Feichtenhofer, C. & Grauman, K. Grounded human-object interaction hotspots from video. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 8688\u20138697 (2019).","DOI":"10.1109\/ICCV.2019.00878"},{"key":"30938_CR8","unstructured":"Zeng, Z. Semantic Robot Programming for Taskable Goal-Directed Manipulation. Ph.D. thesis, University of Michigan (2020)."},{"key":"30938_CR9","doi-asserted-by":"crossref","unstructured":"Xin, J., Wang, L., Xu, K., Yang, C. & Yin, B. Learning interaction regions and motion trajectories simultaneously from egocentric demonstration videos. IEEE Robotics Autom. Lett.\u00a0(2023).","DOI":"10.1109\/LRA.2023.3301307"},{"key":"30938_CR10","doi-asserted-by":"crossref","unstructured":"Damen, D. et al. Scaling egocentric vision: The epic-kitchens dataset. In European Conference on Computer Vision (ECCV). (2018).","DOI":"10.1007\/978-3-030-01225-0_44"},{"key":"30938_CR11","doi-asserted-by":"crossref","unstructured":"Jonnavittula, A., Parekh, S. & P.\u00a0Losey, D. View: Visual imitation learning with waypoints.\u00a0Auton. Robots.\u00a049, 5 (2025).","DOI":"10.1007\/s10514-024-10188-y"},{"key":"30938_CR12","doi-asserted-by":"publisher","first-page":"3888","DOI":"10.1109\/TRO.2023.3286074","volume":"39","author":"J Gao","year":"2023","unstructured":"Gao, J., Tao, Z., Jaquier, N. & Asfour, T. K-vil: Keypoints-based visual imitation learning. IEEE Transactions on Robotics 39, 3888\u20133908 (2023).","journal-title":"IEEE Transactions on Robotics"},{"key":"30938_CR13","doi-asserted-by":"crossref","unstructured":"Shan, D., Geng, J., Shu, M. & Fouhey, D.\u00a0F. Understanding human hands in contact at internet scale. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, 9869\u20139878 (2020).","DOI":"10.1109\/CVPR42600.2020.00989"},{"key":"30938_CR14","unstructured":"Ren, S., He, K., Girshick, R. & Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015)."},{"key":"30938_CR15","doi-asserted-by":"crossref","unstructured":"Wen, B., Lian, W., Bekris, K. & Schaal, S. You only demonstrate once: Category-level manipulation from single visual demonstration. Robotics: Science and Systems 2022 (2022).","DOI":"10.15607\/RSS.2022.XVIII.044"},{"key":"30938_CR16","doi-asserted-by":"crossref","unstructured":"Li, S. & Lee, D. Point-to-pose voting based hand pose estimation using residual permutation equivariant layer. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 11927\u201311936 (2019).","DOI":"10.1109\/CVPR.2019.01220"},{"key":"30938_CR17","doi-asserted-by":"crossref","unstructured":"Schaal, S. Dynamic movement primitives-a framework for motor control in humans and humanoid robotics. In Adaptive motion of animals and machines, 261\u2013280 (Springer).","DOI":"10.1007\/4-431-31381-8_23"},{"key":"30938_CR18","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1007\/s10514-018-9725-6","volume":"43","author":"M Kyrarini","year":"2019","unstructured":"Kyrarini, M., Haseeb, M. A., Risti\u0107-Durrant, D. & Gr\u00e4ser, A. Robot learning of industrial assembly task via human demonstrations. Autonomous Robots 43, 239\u2013257 (2019).","journal-title":"Autonomous Robots"},{"key":"30938_CR19","doi-asserted-by":"crossref","unstructured":"Zeng, Z., Zhou, Z., Sui, Z. & Jenkins, O.\u00a0C. Semantic robot programming for goal-directed manipulation in cluttered scenes. In 2018 IEEE international conference on robotics and automation (ICRA), 7462\u20137469 (IEEE, 2018).","DOI":"10.1109\/ICRA.2018.8460538"},{"key":"30938_CR20","volume-title":"Pddl| the planning domain definition language","author":"C Aeronautiques","year":"1998","unstructured":"Aeronautiques, C. et al. Pddl| the planning domain definition language (Tech. Rep, 1998)."},{"key":"30938_CR21","unstructured":"Backhaus, J. C.\u00a0S. Adaptierbares aufgabenorientiertes Programmiersystem f\u00fcr Montagesysteme. Ph.D. thesis, Technische Universit\u00e4t M\u00fcnchen (2016)."},{"key":"30938_CR22","doi-asserted-by":"crossref","unstructured":"Hoda\u0148, T. et al. Photorealistic image synthesis for object instance detection. In 2019 IEEE international conference on image processing (ICIP), 66\u201370 (IEEE, 2019).","DOI":"10.1109\/ICIP.2019.8803821"},{"key":"30938_CR23","doi-asserted-by":"crossref","unstructured":"Fouhey, D.\u00a0F., Kuo, W., Efros, A.\u00a0A. & Malik, J. From lifestyle vlogs to everyday interactions. In CVPR (2018).","DOI":"10.1109\/CVPR.2018.00524"},{"key":"30938_CR24","doi-asserted-by":"crossref","unstructured":"Li, Y., Ye, Z. & Rehg, J.\u00a0M. Delving into egocentric actions. In Proceedings of the IEEE conference on computer vision and pattern recognition, 287\u2013295 (2015).","DOI":"10.1109\/CVPR.2015.7298625"},{"key":"30938_CR25","doi-asserted-by":"crossref","unstructured":"Sigurdsson, G.\u00a0A., Gupta, A.\u00a0K., Schmid, C., Farhadi, A. & Karteek, A. Actor and observer: Joint modeling of first and third-person videos. 2018 IEEE\/CVF Conf. on Comput. Vis. Pattern Recognit.\u00a07396\u20137404 (2018).","DOI":"10.1109\/CVPR.2018.00772"},{"key":"30938_CR26","unstructured":"Lugaresi, C. et al. Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172 (2019)."},{"key":"30938_CR27","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O. et al. Imagenet large scale visual recognition challenge. Int. journal computer vision 115, 211\u2013252 (2015).","journal-title":"Int. journal computer vision"},{"key":"30938_CR28","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","volume":"88","author":"M Everingham","year":"2010","unstructured":"Everingham, M., Van Gool, L., Williams, C. K., Winn, J. & Zisserman, A. The pascal visual object classes (voc) challenge. Int. journal computer vision 88, 303\u2013338 (2010).","journal-title":"Int. journal computer vision"},{"key":"30938_CR29","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y. et al. Microsoft coco: Common objects in context. In Computer Vision\u2013ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, 740\u2013755 (Springer, 2014).","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"30938_CR30","unstructured":"Ding, J. et al. Object detection in aerial images: A large-scale benchmark and challenges. IEEE Transactions on Pattern Analysis and Machine Intelligence 1\u20131, 10.1109\/TPAMI.2021.3117983 (2021)."},{"key":"30938_CR31","unstructured":"Vanherle, B., Moonen, S., Reeth, F.\u00a0V. & Michiels, N. Analysis of training object detection models with synthetic data. In 33rd British Machine Vision Conference 2022, BMVC 2022, London, UK, November 21-24, 2022 (BMVA Press, 2022)."},{"key":"30938_CR32","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2023.103907","volume":"240","author":"H Schieber","year":"2024","unstructured":"Schieber, H., Demir, K. C., Kleinbeck, C., Yang, S. H. & Roth, D. Indoor synthetic data generation: A systematic review. Comput. Vis. Image Underst. 240, 103907 (2024).","journal-title":"Comput. Vis. Image Underst."},{"key":"30938_CR33","doi-asserted-by":"crossref","unstructured":"Singh, R. et al. Synthetica: Large scale synthetic data for robot perception. arXiv preprint arXiv:2410.21153 (2024).","DOI":"10.1109\/IROS60139.2025.11247126"},{"key":"30938_CR34","unstructured":"Borkman, S. et al. Unity perception: generate synthetic data for computer vision. arXiv preprint arXiv:2107.04259 (2021)."},{"key":"30938_CR35","doi-asserted-by":"crossref","unstructured":"Denninger, M. et al. Blenderproc2: A procedural pipeline for photorealistic rendering.\u00a0J. Open Source Softw.\u00a08, 4901, 10.21105\/joss.04901 (2023).","DOI":"10.21105\/joss.04901"},{"key":"30938_CR36","doi-asserted-by":"crossref","unstructured":"Castellano, R., Terreran, M. & Ghidoni, S. Performance evaluation of depth completion neural networks for various rgb-d camera technologies in indoor scenarios. In Basili, R., Lembo, D., Limongelli, C. & Orlandini, A. (eds.) AIxIA 2023 \u2013 Advances in Artificial Intelligence, 351\u2013364 (Springer Nature Switzerland, Cham, 2023).","DOI":"10.1007\/978-3-031-47546-7_24"},{"key":"30938_CR37","unstructured":"Kim, D. et al. Global-local path networks for monocular depth estimation with vertical cutdepth. CoRR abs\/2201.07436 (2022). 2201.07436."},{"key":"30938_CR38","unstructured":"Fabisch, A. Learning and generalizing behaviors for robots from human demonstration. Ph.D. thesis, Universit\u00e4t Bremen (2020)."},{"key":"30938_CR39","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1162\/NECO_a_00393","volume":"25","author":"AJ Ijspeert","year":"2013","unstructured":"Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P. & Schaal, S. Dynamical movement primitives: learning attractor models for motor behaviors. Neural computation 25, 328\u2013373 (2013).","journal-title":"Neural computation"},{"key":"30938_CR40","doi-asserted-by":"crossref","unstructured":"Pastor, P., Hoffmann, H., Asfour, T. & Schaal, S. Learning and generalization of motor skills by learning from demonstration. In 2009 IEEE International Conference on Robotics and Automation, 763\u2013768 (IEEE, 2009).","DOI":"10.1109\/ROBOT.2009.5152385"},{"key":"30938_CR41","doi-asserted-by":"publisher","first-page":"7770","DOI":"10.3390\/s21227770","volume":"21","author":"M Servi","year":"2021","unstructured":"Servi, M. et al. Metrological characterization and comparison of d415, d455, l515 realsense devices in the close range. Sensors 21, 7770 (2021).","journal-title":"Sensors"},{"key":"30938_CR42","doi-asserted-by":"crossref","unstructured":"Chen, Z., Liu, P., Wen, F., Wang, J. & Ying, R. Restoration of motion blur in time-of-flight depth image using data alignment. In 2020 International Conference on 3D Vision (3DV), 820\u2013828, 10.1109\/3DV50981.2020.00092 (2020).","DOI":"10.1109\/3DV50981.2020.00092"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-30938-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-30938-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-30938-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T23:02:01Z","timestamp":1765580521000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-30938-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,5]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["30938"],"URL":"https:\/\/doi.org\/10.1038\/s41598-025-30938-x","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,5]]},"assertion":[{"value":"5 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 November 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 December 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"43265"}}