{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T11:04:27Z","timestamp":1767956667905,"version":"3.49.0"},"reference-count":40,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T00:00:00Z","timestamp":1767916800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Pioneer Center for Accelerating P2X Materials Discovery, CAPeX"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Autonomous experimental platforms increasingly rely on robust, vision-guided robotic manipulation to support reliable and repeatable laboratory operations. This work presents a modular motion-execution subsystem designed for integration into self-driving laboratory (SDL) workflows, focusing on the coupling of real-time visual perception with smooth and stable manipulator control. The framework enables autonomous detection, tracking, and interaction with textured objects through a hybrid scheme that couples advanced motion planning algorithms with real-time visual feedback. Kinematic analysis of the manipulator is performed using the screw theory formulations, which provide a rigorous foundation for deriving forward kinematics and the space Jacobian. These formulations are further employed to compute inverse kinematic solutions via the Damped Least Squares (DLS) method, ensuring stable and continuous joint trajectories even in the presence of redundancy and singularities. Motion trajectories toward target objects are generated using the RRT* algorithm, offering optimal path planning under dynamic constraints. Object pose estimation is achieved through a a vision workflow integrating feature-driven detection and homography-guided depth analysis, enabling adaptive tracking and dynamic grasping of textured objects. The manipulator\u2019s performance is quantitatively evaluated using smoothness metrics, RMSE pose errors, and joint motion profiles, including velocity continuity, acceleration, jerk, and snap. Simulation results demonstrate that the proposed subsystem delivers stable, smooth, and reproducible motion execution, establishing a validated baseline for the manipulation layer of next-generation SDL architectures.<\/jats:p>","DOI":"10.3390\/robotics15010017","type":"journal-article","created":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T09:25:02Z","timestamp":1767950702000},"page":"17","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Multimodal Control of Manipulators: Coupling Kinematics and Vision for Self-Driving Laboratory Operations"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5330-0053","authenticated-orcid":false,"given":"Shifa","family":"Sulaiman","sequence":"first","affiliation":[{"name":"Department of Electronics Systems, Aalborg University, 9220 Aalborg, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3367-5755","authenticated-orcid":false,"given":"Amarnath","family":"Harikumar","sequence":"additional","affiliation":[{"name":"Nawe Robotics, Calicut 673601, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5960-4365","authenticated-orcid":false,"given":"Simon","family":"B\u00f8gh","sequence":"additional","affiliation":[{"name":"Department of Electronics Systems, Aalborg University, 9220 Aalborg, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0159-167X","authenticated-orcid":false,"given":"Naresh","family":"Marturi","sequence":"additional","affiliation":[{"name":"Extreme Robotics Laboratory, School of Metallurgy & Materials, University of Birmingham, Birmingham B15 2TT, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,1,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Dobrokvashina, A., Sulaiman, S., Gamberov, T., Hsia, K.H., and Magid, E. (2023, January 9\u201312). New Features Implementation for Servosila Engineer Model in Gazebo Simulator for ROS Noetic. Proceedings of the International Conference on Artificial Life and Robotics, Oita, Japan.","DOI":"10.5954\/ICAROB.2023.OS6-3"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2454","DOI":"10.1021\/acs.accounts.2c00220","article-title":"Autonomous chemical experiments: Challenges and perspectives on establishing a self-driving lab","volume":"55","author":"Seifrid","year":"2022","journal-title":"Acc. Chem. Res."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sultanov, R., Sulaiman, S., Tsoy, T., and Chebotareva, E. (2023, January 9\u201312). Virtual collaborative cells modeling for UR3 and UR5 robots in Gazebo simulator. Proceedings of the 2023 International Conference on Artificial Life and Robotics, Oita, Japan.","DOI":"10.5954\/ICAROB.2023.OS6-2"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Sulaiman, S., Harikumar, A., Marturi, N., and B\u00f8gh, S. (2025, January 17\u201319). Comparative Analysis of Jacobian-Based Motion Planning Methods for Redundant Manipulators. Proceedings of the 2025 IEEE International Conference on Advanced Robotics and its Social Impacts (ARSO), Osaka, Japan.","DOI":"10.1109\/ARSO64737.2025.11124951"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1729881420925645","DOI":"10.1177\/1729881420925645","article-title":"A novel solution of inverse kinematic for 6R robot manipulator with offset joint based on screw theory","volume":"17","author":"Liao","year":"2020","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Gandhi, N., Nagababu, G., and Vistapalli, J. (2021). Modeling and Kinematic Analysis of a Robotic Manipulator for Street Cleaning Applications Using Screw Theory. Advances in Industrial Machines and Mechanisms: Select Proceedings of IPROMM 2020, Springer.","DOI":"10.1007\/978-981-16-1769-0_55"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"6857106","DOI":"10.1155\/2019\/6857106","article-title":"Kinematical research of free-floating space-robot system at position level based on screw theory","volume":"2019","author":"Wang","year":"2019","journal-title":"Int. J. Aerosp. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1186\/s10033-021-00610-2","article-title":"Generalized kinematics analysis of hybrid mechanisms based on screw theory and lie groups lie algebras","volume":"34","author":"Sun","year":"2021","journal-title":"Chin. J. Mech. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1007\/s12541-018-0021-3","article-title":"Trajectory planning with minimum synthesis error for industrial robots using screw theory","volume":"19","author":"Liu","year":"2018","journal-title":"Int. J. Precis. Eng. Manuf."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"012068","DOI":"10.1088\/1742-6596\/2246\/1\/012068","article-title":"Kinematics modeling of redundant manipulator based on screw theory and Newton-Raphson method","volume":"2246","author":"Ge","year":"2022","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"012036","DOI":"10.1088\/1757-899X\/1132\/1\/012036","article-title":"Modelling of torso and dual arms for a humanoid robot with fixed base by using screw theory for dexterous applications","volume":"1132","author":"Shifa","year":"2021","journal-title":"IOP Conf. Ser. Mater. Sci. Eng."},{"key":"ref_12","first-page":"1058","article-title":"Modeling of a wheeled humanoid robot and hybrid algorithm-based path planning of wheel base for the dynamic obstacles avoidance","volume":"49","author":"Shifa","year":"2022","journal-title":"Ind. Robot. Int. J. Robot. Res. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Medrano-Hermosillo, J.A., Lozoya-Ponce, R., Ram\u00edrez-Quintana, J., and Baray-Arana, R. (2022, January 9\u201311). Forward Kinematics Analysis of 6-DoF Articulated Robot using Screw Theory and Geometric Algebra. Proceedings of the 2022 XXIV Robotics Mexican Congress (COMRob), Mineral de la Reforma, Mexico.","DOI":"10.1109\/COMRob57154.2022.9962312"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"0098","DOI":"10.34133\/cbsystems.0098","article-title":"A robotic teleoperation system enhanced by augmented reality for natural human\u2013robot interaction","volume":"5","author":"Wang","year":"2024","journal-title":"Cyborg Bionic Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"0120","DOI":"10.34133\/cbsystems.0120","article-title":"Fast and comfortable robot-to-human handover for mobile cooperation robot system","volume":"5","author":"Meng","year":"2024","journal-title":"Cyborg Bionic Syst."},{"key":"ref_16","unstructured":"H\u00e9l\u00e9non, F., Huber, J., Amar, F.B., and Doncieux, S. (2023). Toward a Plug-and-Play Vision-Based Grasping Module for Robotics. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1214","DOI":"10.1007\/s41315-025-00428-4","article-title":"Vision-based intelligent robot grasping using sparse neural network","volume":"9","author":"Kushwaha","year":"2025","journal-title":"Int. J. Intell. Robot. Appl."},{"key":"ref_18","first-page":"112","article-title":"Polynomial-based smooth trajectory planning for fruit-picking robot manipulator","volume":"9","author":"Wang","year":"2022","journal-title":"Inf. Process. Agric."},{"key":"ref_19","unstructured":"Zhang, H., Tang, J., Sun, S., and Lan, X. (2022). Robotic grasping from classical to modern: A survey. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3994","DOI":"10.1109\/TRO.2023.3280597","article-title":"Deep learning approaches to grasp synthesis: A review","volume":"39","author":"Newbury","year":"2023","journal-title":"IEEE Trans. Robot."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1677","DOI":"10.1007\/s10462-020-09888-5","article-title":"Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: A review","volume":"54","author":"Du","year":"2021","journal-title":"Artif. Intell. Rev."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhang, S., Lyu, B., Li, Y., Sui, L., and Yang, J. (2021, January 28\u201329). Trajectory planning of a 6R manipulator around obstacles by using SRRT* Algorithm. Proceedings of the 2nd International Conference on Computing and Data Science, Online.","DOI":"10.1145\/3448734.3450853"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ortenzi, V., Marturi, N., Rajasekaran, V., Adjigble, M., and Stolkin, R. (2019, January 22\u201326). Singularity-robust inverse kinematics solver for tele-manipulation. Proceedings of the 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE), Vancouver, BC, Canada.","DOI":"10.1109\/COASE.2019.8842871"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"122","DOI":"10.2478\/msr-2013-0021","article-title":"A Comparative Study of SIFT and its Variants","volume":"13","author":"Wu","year":"2013","journal-title":"Meas. Sci. Rev."},{"key":"ref_25","unstructured":"Muja, M., and Lowe, D.G. (2009, January 5\u20138). Fast approximate nearest neighbors with automatic algorithm configuration. Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), Lisboa, Portugal."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Pardos-Gotor, J. (2021). Screw Theory in Robotics: An Illustrated and Practicable Introduction to Modern Mechanics, CRC Press.","DOI":"10.1201\/9781003216858"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Lynch, K.M., and Park, F.C. (2017). Modern Robotics, Cambridge University Press.","DOI":"10.1017\/9781316661239"},{"key":"ref_29","first-page":"16","article-title":"Introduction to inverse kinematics with jacobian transpose, pseudoinverse and damped least squares methods","volume":"17","author":"Buss","year":"2004","journal-title":"IEEE J. Robot. Autom."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Paul, S.K., Chowdhury, M.T., Nicolescu, M., Nicolescu, M., and Feil-Seifer, D. (2021). Object detection and pose estimation from rgb and depth data for real-time, adaptive robotic grasping. Advances in Computer Vision and Computational Biology: Proceedings from IPCV\u201920, HIMS\u201920, BIOCOMP\u201920, and BIOENG\u201920, Springer.","DOI":"10.1007\/978-3-030-71051-4_10"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kato, N., Iuchi, T., Murabayashi, K., and Tanaka, T. (2023). Comparison of Smoothness, Movement Speed and Trajectory during Reaching Movements in Real and Virtual Spaces Using a Head-Mounted Display. Life, 13.","DOI":"10.3390\/life13081618"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"103810","DOI":"10.1016\/j.robot.2021.103810","article-title":"Benchmarking pose estimation for robot manipulation","volume":"143","author":"Hietanen","year":"2021","journal-title":"Robot. Auton. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1252","DOI":"10.1016\/j.patrec.2007.02.009","article-title":"3D free-form object recognition in range images using local surface patches","volume":"28","author":"Chen","year":"2007","journal-title":"Pattern Recognit. Lett."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Tombari, F., and Di Stefano, L. (2010, January 14\u201317). Object recognition in 3d scenes with occlusions and clutter by hough voting. Proceedings of the 2010 Fourth Pacific-Rim Symposium on Image and Video Technology, Singapore.","DOI":"10.1109\/PSIVT.2010.65"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Glent Buch, A., Yang, Y., Kruger, N., and Gordon Petersen, H. (2014, January 23\u201328). In search of inliers: 3d correspondence by local and global voting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.266"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Leordeanu, M., and Hebert, M. (2005, January 17\u201321). A spectral technique for correspondence problems using pairwise constraints. Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV\u201905) Volume 1, Beijing, China.","DOI":"10.1109\/ICCV.2005.20"},{"key":"ref_37","unstructured":"Hough, P.V. (1962). Method and Means for Recognizing Complex Patterns. (US Patent 3,069,654)."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Dong, S., and Rodriguez, A. (2019, January 3\u20138). Tactile-Based Insertion for Dense Box-Packing. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8968204"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Li, R., Platt, R., Yuan, W., ten Pas, A., Roscup, N., Srinivasan, M.A., and Adelson, E. (2014, January 14\u201318). Localization and manipulation of small parts using GelSight tactile sensing. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6943123"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2999","DOI":"10.1007\/s00170-021-07669-0","article-title":"A novel accurate positioning method for object pose estimation in robotic manipulation based on vision and tactile sensors","volume":"116","author":"Zhao","year":"2021","journal-title":"Int. J. Adv. Manuf. Technol."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/15\/1\/17\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T09:37:04Z","timestamp":1767951424000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/15\/1\/17"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,9]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1]]}},"alternative-id":["robotics15010017"],"URL":"https:\/\/doi.org\/10.3390\/robotics15010017","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,9]]}}}