{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T18:59:20Z","timestamp":1776106760361,"version":"3.50.1"},"reference-count":45,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,12,29]],"date-time":"2023-12-29T00:00:00Z","timestamp":1703808000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004193","name":"Nanjing University of Aeronautics and Astronautics","doi-asserted-by":"publisher","award":["xcxjh20221509"],"award-info":[{"award-number":["xcxjh20221509"]}],"id":[{"id":"10.13039\/501100004193","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>This paper addresses the image-based visual servoing (IBVS) control problem with an uncalibrated camera, unknown dynamics, and constraints. A novel data-driven uncalibrated IBVS (UIBVS) strategy is proposed, incorporated with the Koopman-based model predictive control (KMPC) algorithm and the adaptive robust Kalman filter (ARKF). First, to alleviate the need for calibration of the camera\u2019s intrinsic and extrinsic parameters, the ARKF with an adaptive factor is utilized to estimate the image Jacobian matrix online, thereby eliminating the laborious camera calibration procedures and improving robustness against camera disturbances. Then, a data-driven MPC strategy is proposed, wherein the unknown nonlinear dynamic model is learned using the Koopman operator theory, resulting in a linear Koopman prediction model. Only input\u2013output data are used to construct the prediction model, and hence, the proposed approach is robust against model uncertainties. Furthermore, with a symmetric quadratic cost function, the proposed approach solves the quadratic programming problem online, and visibility constraints as well as joint torque constraints are taken into account. As a result, the proposed KMPC scheme can be implemented in real time, and the UIBVS performance degradation which arises from the control torque constraints can be avoided. Simulations and comparisons for a 2-DOF robotic manipulator demonstrate the feasibility of the proposed approach. Simulation results further validate that the computation time of the proposed approach is comparable to the one of kinematic-based methods.<\/jats:p>","DOI":"10.3390\/sym16010048","type":"journal-article","created":{"date-parts":[[2023,12,29]],"date-time":"2023-12-29T06:18:13Z","timestamp":1703830693000},"page":"48","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Data-Driven Model Predictive Control for Uncalibrated Visual Servoing"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3845-2534","authenticated-orcid":false,"given":"Tianjiao","family":"Han","sequence":"first","affiliation":[{"name":"College of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongyu","family":"Zhu","sequence":"additional","affiliation":[{"name":"College of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2656-4731","authenticated-orcid":false,"given":"Dan","family":"Yu","sequence":"additional","affiliation":[{"name":"College of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"73540","DOI":"10.1109\/ACCESS.2019.2920389","article-title":"Model Predictive Control for Uncalibrated and Constrained Image-Based Visual Servoing without Joint Velocity Measurements","volume":"7","author":"Qiu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Adjigble, M., Tamadazte, B., de Farias, C., Stolkin, R., and Marturi, N. (June, January 29). 3D Spectral Domain Registration-Based Visual Servoing. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10160430"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"828","DOI":"10.1109\/TRO.2011.2112593","article-title":"Photometric Visual Servoing","volume":"27","author":"Collewet","year":"2011","journal-title":"IEEE Trans. Robot."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1109\/LRA.2016.2535961","article-title":"Histograms-Based Visual Servoing","volume":"2","author":"Bateux","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1109\/TRO.2018.2876765","article-title":"Visual Servoing with Photometric Gaussian Mixtures as Dense Features","volume":"35","author":"Crombez","year":"2019","journal-title":"IEEE Trans. Robot."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"10895","DOI":"10.1109\/TCYB.2021.3070598","article-title":"Fully Uncalibrated Image-Based Visual Servoing of 2DOFs Planar Manipulators with a Fixed Camera","volume":"52","author":"Liang","year":"2022","journal-title":"IEEE Trans. Cybern."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.automatica.2015.06.018","article-title":"A unified design method for adaptive visual tracking control of robots with eye-in-hand\/fixed camera configuration","volume":"59","author":"Liang","year":"2015","journal-title":"Automatica"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"102381","DOI":"10.1016\/j.mechatronics.2020.102381","article-title":"Improved noise covariance estimation in visual servoing using an autocovariance least-squares approach","volume":"68","author":"Brown","year":"2020","journal-title":"Mechatronics"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"76891","DOI":"10.1109\/ACCESS.2019.2920941","article-title":"Adaptive Neuro-Filtering Based Visual Servo Control of a Robotic Manipulator","volume":"7","author":"Zhong","year":"2019","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.robot.2018.10.011","article-title":"Kinematics-based incremental visual servo for robotic capture of non-cooperative target","volume":"112","author":"Dong","year":"2019","journal-title":"Robot. Auton. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MRA.2006.250573","article-title":"Visual servo control. I. Basic approaches","volume":"13","author":"Chaumette","year":"2006","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1109\/TMECH.2013.2245337","article-title":"Image-Based Visual Servoing of a 7-DOF Robot Manipulator Using an Adaptive Distributed Fuzzy PD Controller","volume":"19","author":"Siradjuddin","year":"2014","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"8214","DOI":"10.1109\/TIE.2018.2881948","article-title":"Quality Inspection of Remote Radio Units Using Depth-Free Image-Based Visual Servo with Acceleration Command","volume":"66","author":"Anwar","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1016\/j.ins.2018.12.082","article-title":"A learning approach to image-based visual servoing with a bagging method of velocity calculations","volume":"481","author":"Shi","year":"2019","journal-title":"Inf. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"103712","DOI":"10.1016\/j.robot.2020.103712","article-title":"Gaussian process-based nonlinear predictive control for visual servoing of constrained mobile robots with unknown dynamics","volume":"136","author":"Jin","year":"2021","journal-title":"Robot. Auton. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"170","DOI":"10.37394\/23202.2021.20.19","article-title":"Dynamic Programming in Data Driven Model Predictive Control","volume":"20","author":"Jianhong","year":"2021","journal-title":"WSEAS Trans. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, Y., Li, L., and Zhang, C. (2021). AMT Starting Control as a Soft Starter for Belt Conveyors Using a Data-Driven Method. Symmetry, 13.","DOI":"10.3390\/sym13101808"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.isatra.2020.06.013","article-title":"Disturbance observer based adaptive model predictive control for uncalibrated visual servoing in constrained environments","volume":"106","author":"Qiu","year":"2020","journal-title":"ISA Trans."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"9270","DOI":"10.1109\/TIE.2022.3212411","article-title":"Synthetic Robust Model Predictive Control with Input Mapping for Constrained Visual Servoing","volume":"70","author":"He","year":"2022","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"562","DOI":"10.1109\/ROBOT.2002.1013418","article-title":"Online estimation of image Jacobian matrix by Kalman-Bucy filter for uncalibrated stereo vision feedback","volume":"Volume 1","author":"Qian","year":"2002","journal-title":"Proceedings of the 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292)"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"13464","DOI":"10.3390\/s131013464","article-title":"Robust Kalman filtering cooperated Elman neural network learning for vision-sensing-based robotic manipulation with global stability","volume":"13","author":"Zhong","year":"2013","journal-title":"Sensors"},{"key":"ref_22","first-page":"347410","article-title":"Real-Time Inverse Optimal Neural Control for Image Based Visual Servoing with Nonholonomic Mobile Robots","volume":"2015","author":"Alanis","year":"2015","journal-title":"Math. Probl. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"755","DOI":"10.1007\/s11431-020-1688-2","article-title":"Tracking and grasping of moving target based on accelerated geometric particle filter on colored image","volume":"64","author":"Gong","year":"2021","journal-title":"Sci. China Technol. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"12586","DOI":"10.1109\/TIE.2023.3237881","article-title":"Visual servoing control of robotics with a neural network estimator based on spectral adaptive law","volume":"70","author":"Han","year":"2023","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"92020","DOI":"10.1109\/ACCESS.2019.2927413","article-title":"Unscented particle filter for online total image Jacobian matrix estimation in robot visual servoing","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"402","DOI":"10.1109\/TCST.2022.3172571","article-title":"Eye-in-hand visual servoing control of robot manipulators based on an input mapping method","volume":"31","author":"He","year":"2022","journal-title":"IEEE Trans. Control. Syst. Technol."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhang, S., and Zhuan, X. (2022). Two-Dimensional Car-Following Control Strategy for Electric Vehicle Based on MPC and DQN. Symmetry, 14.","DOI":"10.3390\/sym14081718"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, S., and Zhuan, X. (2022). Distributed Model Predictive Control for Two-Dimensional Electric Vehicle Platoon Based on QMIX Algorithm. Symmetry, 14.","DOI":"10.3390\/sym14102069"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Urrea, C., and Saa, D. (2023). Design, Simulation, Implementation, and Comparison of Advanced Control Strategies Applied to a 6-DoF Planar Robot. Symmetry, 15.","DOI":"10.3390\/sym15051070"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/j.isatra.2020.10.057","article-title":"Vision-based neural predictive tracking control for multi-manipulator systems with parametric uncertainty","volume":"110","author":"Wu","year":"2021","journal-title":"ISA Trans."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1109\/TCST.2015.2420629","article-title":"Gaussian process-based predictive control for periodic error correction","volume":"24","author":"Klenske","year":"2015","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Thuruthel, T.G., Falotico, E., Renda, F., and Laschi, C. (2017). Learning dynamic models for open loop predictive control of soft robotic manipulators. Bioinspir. Biomim., 12.","DOI":"10.1088\/1748-3190\/aa839f"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Abraham, I., De La Torre, G., and Murphey, T.D. (2017). Model-based control using Koopman operators. arXiv.","DOI":"10.15607\/RSS.2017.XIII.052"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1177\/00202940221095559","article-title":"Koopman operator based model predictive control for trajectory tracking of an omnidirectional mobile manipulator","volume":"55","author":"Zhu","year":"2022","journal-title":"Meas. Control"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"948","DOI":"10.1109\/TRO.2020.3038693","article-title":"Data-Driven Control of Soft Robots Using Koopman Operator Theory","volume":"37","author":"Bruder","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Tan, N., Yu, P., and Zheng, W. (2022). Uncalibrated and Unmodeled Image-Based Visual Servoing of Robot Manipulators Using Zeroing Neural Networks. IEEE Trans. Cybern., 1\u201314.","DOI":"10.1109\/TCYB.2022.3227470"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zhang, T., and Yan, P. (2023). Symmetric Time-Variant IBLF-Based Tracking Control with Prescribed Performance for a Robot. Symmetry, 15.","DOI":"10.3390\/sym15101919"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zheng, Y.X., and Liao, Y. (2016, January 27\u201328). Missile Control Parameters Estimation That Uses Robust Adaptive Kalman Filter Algorithm. Proceedings of the 2016 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), Hangzhou, China.","DOI":"10.1109\/IHMSC.2016.262"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.automatica.2018.03.046","article-title":"Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control","volume":"93","author":"Korda","year":"2018","journal-title":"Automatica"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1109\/LRA.2022.3229235","article-title":"An Improved Koopman-MPC Framework for Data-Driven Modeling and Control of Soft Actuators","volume":"8","author":"Wang","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"89","DOI":"10.37394\/232014.2022.18.12","article-title":"A Computer Graphic Image Technology with Visual Communication Based on Data Mining","volume":"18","author":"Yan","year":"2022","journal-title":"Wseas Trans. Signal Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"3102","DOI":"10.1109\/LRA.2023.3264816","article-title":"Online Model Predictive Control of Robot Manipulator With Structured Deep Koopman Model","volume":"8","author":"Zhang","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2255","DOI":"10.1109\/TRO.2022.3228130","article-title":"Learning Stable Models for Prediction and Control","volume":"39","author":"Mamakoukas","year":"2023","journal-title":"IEEE Trans. Robot."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"110114","DOI":"10.1016\/j.automatica.2021.110114","article-title":"Robust tube-based model predictive control with Koopman operators","volume":"137","author":"Zhang","year":"2022","journal-title":"Automatica"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1049\/cth2.12238","article-title":"Adaptive neural network control for image-based visual servoing of robot manipulators","volume":"16","author":"Qiu","year":"2022","journal-title":"IET Control Theory Appl."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/1\/48\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:44:13Z","timestamp":1760132653000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/1\/48"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,29]]},"references-count":45,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["sym16010048"],"URL":"https:\/\/doi.org\/10.3390\/sym16010048","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,29]]}}}