{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T05:03:26Z","timestamp":1789621406099,"version":"build-2803163510"},"reference-count":31,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T00:00:00Z","timestamp":1789084800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42005100"],"award-info":[{"award-number":["42005100"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"award":["42005100"],"award-info":[{"award-number":["42005100"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Reliable estimation of humanoid robot body configuration from monocular RGB frames is useful for external monitoring in traffic, logistics, and service-robotics environments. However, direct transfer of human pose models is challenged by differences in body proportions, rigid surface appearance, joint morphology, and local texture, while robot-specific annotated data are often limited. This study presents a detection-guided framework for full-body 2D keypoint estimation from monocular video frames. A robot-specific detector first localises the target, and the detected box is expanded and normalised into a local region of interest (ROI) for keypoint recovery. Rather than treating the task as direct coordinate regression, the proposed integration uses ResNet18 feature extraction, convolutional block attention module (CBAM) refinement, heatmap-based keypoint representation, differentiable spatial to numerical transform (DSNT)-based continuous coordinate decoding, and skeleton-aware regularisation. A compact 13-keypoint annotation protocol is defined to describe the head, shoulders, elbows, hands, hips, knees, and feet. On the held-out test set, the proposed model reduces mean per-joint position error (MPJPE) from 27.98 px to 15.49 px relative to the ResNet18 direct-regression baseline and improves object keypoint similarity (OKS) from 0.80 to 0.96. Under the same 13-keypoint protocol, the proposed model also achieves a lower MPJPE than fine-tuned YOLOv8-Pose with detector-ROI input (18.85 px) and HRNet-W18 (30.82 px). Direct transfer of COCO-pretrained YOLOv8-Pose performs substantially worse, with an MPJPE of 201.50 px and an OKS of 0.28. These results support the effectiveness of robot-specific local spatial modelling and continuous coordinate recovery for monocular humanoid robot keypoint estimation under the evaluated limited-data setting.<\/jats:p>","DOI":"10.3390\/s26185784","type":"journal-article","created":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T14:39:24Z","timestamp":1789137564000},"page":"5784","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Detection-Guided Keypoint Estimation for Humanoid Robots from Video Frames"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-8026-7827","authenticated-orcid":false,"given":"Xuan","family":"Lou","sequence":"first","affiliation":[{"name":"School of Transportation and Civil Engineering, Nantong University, Nantong 226019, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5645-3610","authenticated-orcid":false,"given":"Zhihuo","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Transportation and Civil Engineering, Nantong University, Nantong 226019, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1183-103X","authenticated-orcid":false,"given":"Yuexia","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Transportation and Civil Engineering, Nantong University, Nantong 226019, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Mehta, V., Sharma, C., and Thiyagarajan, K. (2025). Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy. Sensors, 25.","DOI":"10.3390\/s25206394"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Panasiuk, J. (2025). Controlling an Industrial Robot Using Stereo 3D Vision Systems with AI Elements. Sensors, 25.","DOI":"10.3390\/s25206402"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mansakul, T., Tang, G., Webb, P., Rice, J., Oakley, D., and Fowler, J. (2025). An End-to-End Computationally Lightweight Vision-Based Grasping System for Grocery Items. Sensors, 25.","DOI":"10.3390\/s25175309"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zhang, P., Li, J., Liu, J., He, F., and Jiang, Y. (2025). A Likelihood-Based Pose Estimation Method for Robotic Arm Repeatability Measurement Using Monocular Vision. Sensors, 25.","DOI":"10.3390\/s25227089"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Balasubramanian, B., and Cetin, K. (2025). Vision-Based 6D Pose Analytics Solution for High-Precision Industrial Robot Pick-and-Place Applications. Sensors, 25.","DOI":"10.3390\/s25154824"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Shan, B., Zhao, D., Zhao, R., and Hiroshi, Y. (2025). Real-Time 6D Pose Estimation and Multi-Target Tracking for Low-Cost Multi-Robot System. Sensors, 25.","DOI":"10.3390\/s25237130"},{"key":"ref_7","unstructured":"Suzuki, R., Karim, A., Xia, T., Hedayati, H., and Marquardt, N. (May, January 29). Augmented Reality and Robotics: A Survey and Taxonomy for AR-Enhanced Human\u2013Robot Interaction and Robotic Interfaces. Proceedings of the CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Qian, L., Deguet, A., Wang, Z., Liu, Y.-H., and Kazanzides, P. (2019, January 20\u201324). Augmented Reality Assisted Instrument Insertion and Tool Manipulation for the First Assistant in Robotic Surgery. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794263"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Toshev, A., and Szegedy, C. (2014, January 23\u201328). DeepPose: Human Pose Estimation via Deep Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.214"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cao, Z., Simon, T., Wei, S.-E., and Sheikh, Y. (2017, January 21\u201326). Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.143"},{"key":"ref_11","unstructured":"Artacho, B., and Savakis, A. (2021). OmniPose: A Multi-Scale Framework for Multi-Person Pose Estimation. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"38571","DOI":"10.52202\/068431-2795","article-title":"ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation","volume":"Volume 35","author":"Xu","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P., and Zitnick, C.L. (2014). Microsoft COCO: Common Objects in Context. Computer Vision\u2013ECCV 2014, Springer.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Amini, A., Farazi, H., and Behnke, S. (2022). Real-Time Pose Estimation from Images for Multiple Humanoid Robots. Robot World Cup, Springer.","DOI":"10.1007\/978-3-030-98682-7_8"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cho, Y., Son, W., Bak, J., Lee, Y., Lim, H., and Cha, Y. (2024). Full-Body Pose Estimation of Humanoid Robots Using Head-Worn Cameras for Digital Human-Augmented Robotic Telepresence. Mathematics, 12.","DOI":"10.3390\/math12193039"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ban, S., Fan, J., Ma, X., Zhu, W., Qiao, Y., and Wang, Y. (2024). Real-Time Holistic Robot Pose Estimation with Unknown States. Computer Vision\u2013ECCV 2024, Springer.","DOI":"10.1007\/978-3-031-72967-6_1"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Xiao, B., Wu, H., and Wei, Y. (2018, January 8\u201314). Simple Baselines for Human Pose Estimation and Tracking. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01231-1_29"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 15\u201320). Deep High-Resolution Representation Learning for Human Pose Estimation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00584"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sun, X., Xiao, B., Wei, F., Liang, S., and Wei, Y. (2018, January 8\u201314). Integral Human Pose Regression. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01231-1_33"},{"key":"ref_20","unstructured":"Nibali, A., He, Z., Morgan, S., and Prendergast, L. (2018). Numerical Coordinate Regression with Convolutional Neural Networks. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lu, J., Richter, F., and Yip, M.C. (2023, January 17\u201324). Markerless Camera-to-Robot Pose Estimation via Self-Supervised Sim-to-Real Transfer. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.02040"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tian, Y., Zhang, J., Huang, G., Wang, B., Wang, P., Pang, J., and Dong, H. (2024). RoboKeyGen: Robot Pose and Joint Angles Estimation via Diffusion-Based 3D Keypoint Generation. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan, 13\u201317 May 2024, IEEE.","DOI":"10.1109\/ICRA57147.2024.10611423"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Heo, S., Cho, Y., Park, J., Cho, S., Tsoy, Z., Lim, H., and Cha, Y. (2024). Diverse Humanoid Robot Pose Estimation from Images Using Only Sparse Datasets. Appl. Sci., 14.","DOI":"10.3390\/app14199042"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1108\/IR-04-2024-0134","article-title":"Research on 3D Humanoid Robot Pose Estimation Based on HRNet-Epipolar and CRF Robot Model by Multiple View","volume":"52","author":"Wang","year":"2025","journal-title":"Ind. Robot"},{"key":"ref_25","unstructured":"Young, J., and Roberts, J. (2025, January 1\u20133). Robots Watching Robots: Human Pose Estimation Applied to Humanoid Robots. Proceedings of the Australasian Conference on Robotics and Automation (ACRA), Perth, Australia."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Bultmann, S., Memmesheimer, R., and Behnke, S. (2023). External Camera-Based Mobile Robot Pose Estimation for Collaborative Perception with Smart Edge Sensors. arXiv.","DOI":"10.1109\/ICRA48891.2023.10160892"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Labbe, Y., Carpentier, J., Aubry, M., and Sivic, J. (2021, January 13\u201319). Single-View Robot Pose and Joint Angle Estimation via Render and Compare. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR46437.2021.00170"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1007\/s11263-007-0090-8","article-title":"LabelMe: A Database and Web-Based Tool for Image Annotation","volume":"77","author":"Russell","year":"2008","journal-title":"Int. J. Comput. Vis."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8\u201314). CBAM: Convolutional Block Attention Module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_31","unstructured":"Jocher, G., Chaurasia, A., and Qiu, J. (2026, April 25). Ultralytics YOLOv8: A State-of-the-Art Object Detection and Computer Vision Model. Available online: https:\/\/github.com\/ultralytics\/ultralytics."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/18\/5784\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T04:25:08Z","timestamp":1789619108000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/18\/5784"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,11]]},"references-count":31,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["s26185784"],"URL":"https:\/\/doi.org\/10.3390\/s26185784","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,11]]}}}