{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:41:31Z","timestamp":1760146891929,"version":"build-2065373602"},"reference-count":32,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T00:00:00Z","timestamp":1735516800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"dAIedge project","award":["HORIZON-CL4-2022-HUMAN-02-02","101120726"],"award-info":[{"award-number":["HORIZON-CL4-2022-HUMAN-02-02","101120726"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>The demand for flexible automation in manufacturing has increased, incorporating vision-guided systems for object grasping. However, a key challenge is in-hand error, where discrepancies between the actual and estimated positions of an object in the robot\u2019s gripper impact not only the grasp but also subsequent assembly stages. Corrective strategies used to compensate for misalignment can increase cycle times or rely on pre-labeled datasets, offline training, and validation processes, delaying deployment and limiting adaptability in dynamic industrial environments. Our main contribution is an online self-supervised learning method that automates data collection, training, and evaluation in real time, eliminating the need for offline processes. Building on this, our system collects real-time data during each assembly cycle, using corrective strategies to adjust the data and autonomously labeling them via a self-supervised approach. It then builds and evaluates multiple regression models through an auto machine learning implementation. The system selects the best-performing model to correct the misalignment and dynamically chooses between corrective strategies and the learned model, optimizing the cycle times and improving the performance during the cycle, without halting the production process. Our experiments show a significant reduction in the cycle time while maintaining the performance.<\/jats:p>","DOI":"10.3390\/robotics14010004","type":"journal-article","created":{"date-parts":[[2024,12,31]],"date-time":"2024-12-31T13:26:25Z","timestamp":1735651585000},"page":"4","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Online Self-Supervised Learning for Accurate Pick Assembly Operation Optimization"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-1961-1490","authenticated-orcid":false,"given":"Sergio","family":"Vald\u00e9s","sequence":"first","affiliation":[{"name":"Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), 20009 Donostia-San Sebastian, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4494-6688","authenticated-orcid":false,"given":"Marco","family":"Ojer","sequence":"additional","affiliation":[{"name":"Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), 20009 Donostia-San Sebastian, Spain"},{"name":"Computer Science and Artifitial Intelligence Department, University of the Basque Country (UPV\/EHU), 20018 Donostia-San Sebastian, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8518-2293","authenticated-orcid":false,"given":"Xiao","family":"Lin","sequence":"additional","affiliation":[{"name":"Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), 20009 Donostia-San Sebastian, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3203","DOI":"10.1109\/ROBOT.1996.509200","article-title":"Vision-guided robotic grasping: Issues and experiments","volume":"Volume 4","author":"Smith","year":"1996","journal-title":"Proceedings of the IEEE International Conference on Robotics and Automation"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2579","DOI":"10.1109\/LRA.2021.3062350","article-title":"Multi-view object pose refinement with differentiable renderer","volume":"6","author":"Shugurov","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Yuan, S., Ge, Z., and Yang, L. (2023). Single-Camera Multi-View 6DoF pose estimation for robotic grasping. Front. Neurorobotics, 17.","DOI":"10.3389\/fnbot.2023.1136882"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"6299","DOI":"10.1109\/TIE.2017.2682002","article-title":"Compliance-based robotic peg-in-hole assembly strategy without force feedback","volume":"64","author":"Park","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4447","DOI":"10.1109\/LRA.2020.3000428","article-title":"Compliant peg-in-hole assembly using partial spiral force trajectory with tilted peg posture","volume":"5","author":"Park","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/978-3-319-27149-1_9","article-title":"Generic algorithm for peg-in-hole assembly tasks for pin alignments with impedance controlled robots","volume":"Volume 2","author":"Jokesch","year":"2016","journal-title":"Proceedings of the Robot 2015: Second Iberian Robotics Conference: Advances in Robotics"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Gibbons, O., Albini, A., and Maiolino, P. (June, January 29). A Tactile Feedback Insertion Strategy for Peg-in-Hole Tasks. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10160879"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"103651","DOI":"10.1016\/j.robot.2020.103651","article-title":"Skill learning for robotic assembly based on visual perspectives and force sensing","volume":"135","author":"Song","year":"2021","journal-title":"Robot. Auton. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1476","DOI":"10.1016\/j.ifacol.2015.06.295","article-title":"An approach for peg-in-hole assembling using intuitive search algorithm based on human behavior and carried by sensors guided industrial robot","volume":"48","author":"Abdullah","year":"2015","journal-title":"IFAc-PapersOnLine"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1687814017748078","DOI":"10.1177\/1687814017748078","article-title":"Vision-guided peg-in-hole assembly by Baxter robot","volume":"9","author":"Huang","year":"2017","journal-title":"Adv. Mech. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Almaghout, K., Boby, R.A., Othman, M., Shaarawy, A., and Klimchik, A. (2021, January 26\u201329). Robotic pick and assembly using deep learning and hybrid vision\/force control. Proceedings of the 2021 International Conference \u201cNonlinearity, Information and Robotics\u201d (NIR), Innopolis, Russia.","DOI":"10.1109\/NIR52917.2021.9666138"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Schoettler, G., Nair, A., Luo, J., Bahl, S., Ojea, J.A., Solowjow, E., and Levine, S. (2020, January 25\u201329). Deep reinforcement learning for industrial insertion tasks with visual inputs and natural rewards. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341714"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1834","DOI":"10.1109\/LRA.2023.3243526","article-title":"Visual Spatial Attention and Proprioceptive Data-Driven Reinforcement Learning for Robust Peg-in-Hole Task Under Variable Conditions","volume":"8","author":"Yasutomi","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1340","DOI":"10.1016\/j.robot.2012.07.005","article-title":"Dual arm manipulation\u2014A survey","volume":"60","author":"Smith","year":"2012","journal-title":"Robot. Auton. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Enebuse, I., Ibrahim, B.K.K., Foo, M., Matharu, R.S., and Ahmed, H. (2022). Accuracy evaluation of hand-eye calibration techniques for vision-guided robots. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0273261"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"6490","DOI":"10.1109\/TIM.2020.2967958","article-title":"A new formulation for hand\u2013eye calibrations as point-set matching","volume":"69","author":"Qiu","year":"2020","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1109\/70.34770","article-title":"A new technique for fully autonomous and efficient 3 d robotics hand\/eye calibration","volume":"5","author":"Tsai","year":"1989","journal-title":"IEEE Trans. Robot. Autom."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1177\/027836499101000305","article-title":"Finding the position and orientation of a sensor on a robot manipulator using quaternions","volume":"10","author":"Chou","year":"1991","journal-title":"Int. J. Robot. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"717","DOI":"10.1109\/70.326576","article-title":"Robot sensor calibration: Solving AX=XB on the Euclidean group","volume":"10","author":"Park","year":"1994","journal-title":"IEEE Trans. Robot. Autom."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1109\/ICPR.1996.546041","article-title":"The dual quaternion approach to hand-eye calibration","volume":"Volume 1","author":"Daniilidis","year":"1996","journal-title":"Proceedings of the 13th International Conference on Pattern Recognition"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3316","DOI":"10.1109\/ICSMC.1995.538297","article-title":"Eight-space quaternion approach for robotic hand-eye calibration","volume":"Volume 4","author":"Lu","year":"1995","journal-title":"Proceedings of the 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, W., Dong, M., Lu, N., Lou, X., and Sun, P. (2018). Simultaneous robot\u2013world and hand\u2013eye calibration without a calibration object. Sensors, 18.","DOI":"10.3390\/s18113949"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1109\/TRO.2013.2289018","article-title":"Data-driven grasp synthesis\u2014A survey","volume":"30","author":"Bohg","year":"2013","journal-title":"IEEE Trans. Robot."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1109\/MRA.2004.1371616","article-title":"Graspit! a versatile simulator for robotic grasping","volume":"11","author":"Miller","year":"2004","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Haugaard, R.L., Buch, A.G., and Iversen, T.M. (2022, January 20\u201324). Self-supervised deep visual servoing for high precision peg-in-hole insertion. Proceedings of the 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), Mexico City, Mexico.","DOI":"10.1109\/CASE49997.2022.9926468"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3639","DOI":"10.1109\/TASE.2021.3128639","article-title":"A self-supervised learning-based 6-DOF grasp planning method for manipulator","volume":"19","author":"Peng","year":"2021","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1007\/s10462-024-10726-1","article-title":"Automated machine learning: Past, present and future","volume":"57","author":"Baratchi","year":"2024","journal-title":"Artif. Intell. Rev."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chauhan, K., Jani, S., Thakkar, D., Dave, R., Bhatia, J., Tanwar, S., and Obaidat, M.S. (2020, January 5\u20137). Automated Machine Learning: The New Wave of Machine Learning. Proceedings of the 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), Bangalore, India.","DOI":"10.1109\/ICIMIA48430.2020.9074859"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","unstructured":"Hough, P.V. (1962). Method and Means for Recognizing Complex Patterns. (3,069,654), US Patent."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1080\/00031305.1978.10479236","article-title":"Variations of box plots","volume":"32","author":"McGill","year":"1978","journal-title":"Am. Stat."},{"key":"ref_32","unstructured":"Khan, M.A., Zia, M.F., and Faisal, A. (2024, October 28). PyCaret: An Open-Source, Low-Code Machine Learning Library in Python. Available online: https:\/\/www.pycaret.org\/."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/14\/1\/4\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:56:47Z","timestamp":1760115407000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/14\/1\/4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,30]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1]]}},"alternative-id":["robotics14010004"],"URL":"https:\/\/doi.org\/10.3390\/robotics14010004","relation":{},"ISSN":["2218-6581"],"issn-type":[{"type":"electronic","value":"2218-6581"}],"subject":[],"published":{"date-parts":[[2024,12,30]]}}}