{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T15:19:09Z","timestamp":1772119149705,"version":"3.50.1"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,7,1]],"date-time":"2024-07-01T00:00:00Z","timestamp":1719792000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,10]],"date-time":"2024-07-10T00:00:00Z","timestamp":1720569600000},"content-version":"vor","delay-in-days":9,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Turkish Ministry of National Education"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Machine Vision and Applications"],"published-print":{"date-parts":[[2024,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>This study addresses the problem of hand\u2013eye calibration in robotic systems by developing Continual Learning (CL)-based approaches. Traditionally, robots require explicit models to transfer knowledge from camera observations to their hands or base. However, this poses limitations, as the hand\u2013eye calibration parameters are typically valid only for the current camera configuration. We, therefore, propose a flexible and autonomous hand\u2013eye calibration system that can adapt to changes in camera pose over time. Three CL-based approaches are introduced: the naive CL approach, the reservoir rehearsal approach, and the hybrid approach combining reservoir sampling with new data evaluation. The naive CL approach suffers from catastrophic forgetting, while the reservoir rehearsal approach mitigates this issue by sampling uniformly from past data. The hybrid approach further enhances performance by incorporating reservoir sampling and assessing new data for novelty. Experiments conducted in simulated and real-world environments demonstrate that the CL-based approaches, except for the naive approach, achieve competitive performance compared to traditional batch learning-based methods. This suggests that treating hand\u2013eye calibration as a time sequence problem enables the extension of the learned space without complete retraining. The adaptability of the CL-based approaches facilitates accommodating changes in camera pose, leading to an improved hand\u2013eye calibration system.<\/jats:p>","DOI":"10.1007\/s00138-024-01572-w","type":"journal-article","created":{"date-parts":[[2024,7,10]],"date-time":"2024-07-10T14:02:10Z","timestamp":1720620130000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Continual learning approaches to hand\u2013eye calibration in robots"],"prefix":"10.1007","volume":"35","author":[{"given":"Ozan","family":"Bahadir","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jan Paul","family":"Siebert","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gerardo","family":"Aragon-Camarasa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,7,10]]},"reference":[{"issue":"3","key":"1572_CR1","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1109\/70.34770","volume":"5","author":"RY Tsai","year":"1989","unstructured":"Tsai, R.Y., Lenz, R.K., et al.: A new technique for fully autonomous and efficient 3d robotics hand\/eye calibration. IEEE Trans. Robot. Autom. 5(3), 345\u2013358 (1989)","journal-title":"IEEE Trans. Robot. Autom."},{"issue":"10","key":"1572_CR2","first-page":"1530","volume":"5","author":"A Li","year":"2010","unstructured":"Li, A., Wang, L., Wu, D.: Simultaneous robot-world and hand\u2013eye calibration using dual-quaternions and Kronecker product. Int. J. Phys. Sci. 5(10), 1530\u20131536 (2010)","journal-title":"Int. J. Phys. Sci."},{"issue":"4","key":"1572_CR3","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1109\/70.313105","volume":"10","author":"H Zhuang","year":"1994","unstructured":"Zhuang, H., Roth, Z.S., Sudhakar, R.: Simultaneous robot\/world and tool\/flange calibration by solving homogeneous transformation equations of the form ax= yb. IEEE Trans. Robot. Autom. 10(4), 549\u2013554 (1994)","journal-title":"IEEE Trans. Robot. Autom."},{"issue":"3","key":"1572_CR4","doi-asserted-by":"publisher","first-page":"240","DOI":"10.1177\/027836499101000305","volume":"10","author":"JC Chou","year":"1991","unstructured":"Chou, J.C., Kamel, M.: Finding the position and orientation of a sensor on a robot manipulator using quaternions. Int. J. Robot. Res. 10(3), 240\u2013254 (1991)","journal-title":"Int. J. Robot. Res."},{"key":"1572_CR5","doi-asserted-by":"crossref","unstructured":"Bahadir, O., Siebert, J.P., Aragon-Camarasa, G.: A deep learning-based hand-eye calibration approach using a single reference point on a robot manipulator. In: 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1109\u20131114 (2022). IEEE","DOI":"10.1109\/ROBIO55434.2022.10011774"},{"key":"1572_CR6","unstructured":"Valassakis, E., Dreczkowski, K., Johns, E.: Learning eye-in-hand camera calibration from a single image. In: Conference on Robot Learning, pp. 1336\u20131346 (2022)"},{"key":"1572_CR7","doi-asserted-by":"crossref","unstructured":"Lee, T.E., Tremblay, J., To, T., Cheng, J., Mosier, T., Kroemer, O., Fox, D., Birchfield, S.: Camera-to-robot pose estimation from a single image. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 9426\u20139432 (2020). IEEE","DOI":"10.1109\/ICRA40945.2020.9196596"},{"issue":"8","key":"1572_CR8","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735\u20131780 (1997)","journal-title":"Neural Comput."},{"key":"1572_CR9","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1016\/j.neunet.2019.01.012","volume":"113","author":"GI Parisi","year":"2019","unstructured":"Parisi, G.I., Kemker, R., Part, J.L., Kanan, C., Wermter, S.: Continual lifelong learning with neural networks: a review. Neural Netw. 113, 54\u201371 (2019)","journal-title":"Neural Netw."},{"issue":"13","key":"1572_CR10","doi-asserted-by":"publisher","first-page":"3521","DOI":"10.1073\/pnas.1611835114","volume":"114","author":"J Kirkpatrick","year":"2017","unstructured":"Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. 114(13), 3521\u20133526 (2017)","journal-title":"Proc. Natl. Acad. Sci."},{"key":"1572_CR11","doi-asserted-by":"crossref","unstructured":"Liu, X., Masana, M., Herranz, L., Weijer, J., Lopez, A.M., Bagdanov, A.D.: Rotate your networks: better weight consolidation and less catastrophic forgetting. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 2262\u20132268 (2018). IEEE","DOI":"10.1109\/ICPR.2018.8545895"},{"key":"1572_CR12","doi-asserted-by":"crossref","unstructured":"Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, pp. 1131\u20131140 (2020)","DOI":"10.1109\/WACV45572.2020.9093365"},{"key":"1572_CR13","unstructured":"Rusu, A.A., Rabinowitz, N.C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., Hadsell, R.: Progressive neural networks. arXiv preprint arXiv:1606.04671 (2016)"},{"key":"1572_CR14","doi-asserted-by":"crossref","unstructured":"Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: Lifelong learning with a network of experts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3366\u20133375 (2017)","DOI":"10.1109\/CVPR.2017.753"},{"key":"1572_CR15","doi-asserted-by":"crossref","unstructured":"Hayes, T.L., Cahill, N.D., Kanan, C.: Memory efficient experience replay for streaming learning. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 9769\u20139776 (2019). IEEE","DOI":"10.1109\/ICRA.2019.8793982"},{"key":"1572_CR16","doi-asserted-by":"crossref","unstructured":"Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001\u20132010 (2017)","DOI":"10.1109\/CVPR.2017.587"},{"key":"1572_CR17","unstructured":"Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., Wayne, G.: Experience replay for continual learning. In: Advances in Neural Information Processing Systems, 32 (2019)"},{"issue":"3","key":"1572_CR18","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1177\/027836499501400301","volume":"14","author":"R Horaud","year":"1995","unstructured":"Horaud, R., Dornaika, F.: Hand\u2013eye calibration. Int. J. Robot. Res. 14(3), 195\u2013210 (1995)","journal-title":"Int. J. Robot. Res."},{"issue":"3","key":"1572_CR19","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1177\/02783649922066213","volume":"18","author":"K Daniilidis","year":"1999","unstructured":"Daniilidis, K.: Hand\u2013eye calibration using dual quaternions. Int. J. Robot. Res. 18(3), 286\u2013298 (1999)","journal-title":"Int. J. Robot. Res."},{"key":"1572_CR20","doi-asserted-by":"crossref","unstructured":"Zhao, Z.: Hand\u2013eye calibration using convex optimization. In: 2011 IEEE International Conference on Robotics and Automation, pp. 2947\u20132952 (2011). IEEE","DOI":"10.1109\/ICRA.2011.5979569"},{"issue":"4","key":"1572_CR21","doi-asserted-by":"publisher","first-page":"617","DOI":"10.1109\/70.704233","volume":"14","author":"F Dornaika","year":"1998","unstructured":"Dornaika, F., Horaud, R.: Simultaneous robot-world and hand\u2013eye calibration. IEEE Trans. Robot. Autom. 14(4), 617\u2013622 (1998)","journal-title":"IEEE Trans. Robot. Autom."},{"issue":"3","key":"1572_CR22","doi-asserted-by":"publisher","DOI":"10.1115\/1.4024473","volume":"5","author":"M Shah","year":"2013","unstructured":"Shah, M.: Solving the robot-world\/hand\u2013eye calibration problem using the Kronecker product. J. Mech. Robot. 5(3), 031007 (2013)","journal-title":"J. Mech. Robot."},{"key":"1572_CR23","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1016\/j.patrec.2018.08.023","volume":"127","author":"Z Zhao","year":"2019","unstructured":"Zhao, Z.: Simultaneous robot-world and hand\u2013eye calibration by the alternative linear programming. Pattern Recognit. Lett. 127, 174\u2013180 (2019)","journal-title":"Pattern Recognit. Lett."},{"key":"1572_CR24","doi-asserted-by":"crossref","unstructured":"Tabb, A., Yousef, K.M.A.: Parameterizations for reducing camera reprojection error for robot-world hand\u2013eye calibration. In: 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3030\u20133037 (2015). IEEE","DOI":"10.1109\/IROS.2015.7353795"},{"key":"1572_CR25","doi-asserted-by":"crossref","unstructured":"Zhi, X., Schwertfeger, S.: Simultaneous hand-eye calibration and reconstruction. In: 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1470\u20131477 (2017). IEEE","DOI":"10.1109\/IROS.2017.8205949"},{"issue":"12","key":"1572_CR26","doi-asserted-by":"publisher","first-page":"2837","DOI":"10.3390\/s19122837","volume":"19","author":"I Ali","year":"2019","unstructured":"Ali, I., Suominen, O., Gotchev, A., Morales, E.R.: Methods for simultaneous robot-world-hand\u2013eye calibration: a comparative study. Sensors 19(12), 2837 (2019)","journal-title":"Sensors"},{"key":"1572_CR27","doi-asserted-by":"crossref","unstructured":"Lambrecht, J.: Robust few-shot pose estimation of articulated robots using monocular cameras and deep-learning-based keypoint detection. In: 2019 7th International Conference on Robot Intelligence Technology and Applications (RiTA), pp. 136\u2013141 (2019). IEEE","DOI":"10.1109\/RITAPP.2019.8932886"},{"issue":"2","key":"1572_CR28","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1007\/s11263-008-0152-6","volume":"81","author":"V Lepetit","year":"2009","unstructured":"Lepetit, V., Moreno-Noguer, F., Fua, P.: Epnp: an accurate o(n) solution to the pnp problem. Int. J. Comput. Vis. 81(2), 155 (2009)","journal-title":"Int. J. Comput. Vis."},{"key":"1572_CR29","unstructured":"Peretroukhin, V., Giamou, M., Rosen, D.M., Greene, W.N., Roy, N., Kelly, J.: A smooth representation of belief over so (3) for deep rotation learning with uncertainty"},{"key":"1572_CR30","unstructured":"Ven, G.M., Tuytelaars, T., Tolias, A.S.: Three types of incremental learning. Nat. Mach. Intell., pp. 1\u201313 (2022)"},{"key":"1572_CR31","doi-asserted-by":"crossref","unstructured":"Pore, A., Aragon-Camarasa, G.: On simple reactive neural networks for behaviour-based reinforcement learning. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 7477\u20137483 (2020). IEEE","DOI":"10.1109\/ICRA40945.2020.9197262"},{"key":"1572_CR32","unstructured":"Mirza, M.J., Masana, M., Possegger, H., Bischof, H.: An efficient domain-incremental learning approach to drive in all weather conditions. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 3001\u20133011 (2022)"},{"key":"1572_CR33","doi-asserted-by":"crossref","unstructured":"Li, D., Tasci, S., Ghosh, S., Zhu, J., Zhang, J., Heck, L.: Rilod: near real-time incremental learning for object detection at the edge. In: Proceedings of the 4th ACM\/IEEE Symposium on Edge Computing, pp. 113\u2013126 (2019)","DOI":"10.1145\/3318216.3363317"},{"key":"1572_CR34","unstructured":"Fernando, C., Banarse, D., Blundell, C., Zwols, Y., Ha, D., Rusu, A.A., Pritzel, A., Wierstra, D.: Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734 (2017)"},{"key":"1572_CR35","doi-asserted-by":"crossref","unstructured":"Mallya, A., Lazebnik, S.: Packnet: adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765\u20137773 (2018)","DOI":"10.1109\/CVPR.2018.00810"},{"key":"1572_CR36","doi-asserted-by":"crossref","unstructured":"Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139\u2013154 (2018)","DOI":"10.1007\/978-3-030-01219-9_9"},{"issue":"12","key":"1572_CR37","doi-asserted-by":"publisher","first-page":"2935","DOI":"10.1109\/TPAMI.2017.2773081","volume":"40","author":"Z Li","year":"2017","unstructured":"Li, Z., Hoiem, D.: Learning without forgetting. IEEE Trans. Pattern Anal. Mach. Intell. 40(12), 2935\u20132947 (2017)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"1572_CR38","unstructured":"Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)"},{"key":"1572_CR39","doi-asserted-by":"crossref","unstructured":"Wang, S., Laskar, Z., Melekhov, I., Li, X., Kannala, J.: Continual learning for image-based camera localization. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp. 3252\u20133262 (2021)","DOI":"10.1109\/ICCV48922.2021.00324"},{"issue":"11","key":"1572_CR40","doi-asserted-by":"publisher","first-page":"5008","DOI":"10.1109\/TNNLS.2020.3026621","volume":"32","author":"W Zhang","year":"2020","unstructured":"Zhang, W., Wu, Q.J., Yang, Y., Akilan, T.: Multimodel feature reinforcement framework using Moore\u2013Penrose inverse for big data analysis. IEEE Trans. Neural Netw. Learn. Syst. 32(11), 5008\u20135021 (2020)","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"1572_CR41","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/j.neucom.2020.07.018","volume":"414","author":"W Zhang","year":"2020","unstructured":"Zhang, W., Wu, J., Yang, Y.: Wi-hsnn: a subnetwork-based encoding structure for dimension reduction and food classification via harnessing multi-cnn model high-level features. Neurocomputing 414, 57\u201366 (2020)","journal-title":"Neurocomputing"},{"key":"1572_CR42","unstructured":"Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)"}],"container-title":["Machine Vision and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00138-024-01572-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00138-024-01572-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00138-024-01572-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,31]],"date-time":"2024-07-31T15:38:04Z","timestamp":1722440284000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00138-024-01572-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7]]},"references-count":42,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,7]]}},"alternative-id":["1572"],"URL":"https:\/\/doi.org\/10.1007\/s00138-024-01572-w","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-3906009\/v1","asserted-by":"object"}]},"ISSN":["0932-8092","1432-1769"],"issn-type":[{"value":"0932-8092","type":"print"},{"value":"1432-1769","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7]]},"assertion":[{"value":"28 January 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 June 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 June 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 July 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"97"}}