{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T01:57:50Z","timestamp":1778551070740,"version":"3.51.4"},"reference-count":47,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T00:00:00Z","timestamp":1761609600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>We propose a point cloud-based framework for open-vocabulary object pose estimation, called Pov9D. Existing approaches are predominantly RGB-based and often rely on texture or appearance cues, making them susceptible to pose ambiguities when objects are textureless or lack distinctive visual features. In contrast, Pov9D takes 3D point clouds as input, enabling direct access to geometric structures that are essential for accurate and robust pose estimation, especially in open-vocabulary settings. To bridge the gap between geometric observations and semantic understanding, Pov9D integrates category-level textual descriptions to guide the estimation process. To this end, we introduce a text-conditioned shape prior generator that predicts a normalized object shape from both the observed point cloud and the textual category description. This shape prior provides a consistent geometric reference, facilitating precise prediction of object translation, rotation, and size, even for unseen categories. Extensive experiments on the OO3D-9D benchmark demonstrate that Pov9D achieves state-of-the-art performance, improving Abs IoU@50 by 7.2% and Rel 10\u00b0 10 cm by 27.2% over OV9D.<\/jats:p>","DOI":"10.3390\/jimaging11110380","type":"journal-article","created":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T02:23:28Z","timestamp":1761704608000},"page":"380","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Pov9D: Point Cloud-Based Open-Vocabulary 9D Object Pose Estimation"],"prefix":"10.3390","volume":"11","author":[{"given":"Tianfu","family":"Wang","sequence":"first","affiliation":[{"name":"State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Chuangxin Road 135, Shenyang 110016, China"},{"name":"University of Chinese Academy of Sciences, Yuquan Road 19, Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongguang","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Chuangxin Road 135, Shenyang 110016, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Lu, Y., Xu, C., Wei, X., Xie, X., Tomizuka, M., Keutzer, K., and Zhang, S. (2023, January 17\u201324). Open-Vocabulary Point-Cloud Object Detection Without 3D Annotation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00121"},{"key":"ref_2","unstructured":"Zhang, H., Xu, J., Tang, T., Sun, H., Yu, X., Huang, Z., and Yu, K. (October, January 29). Opensight: A Simple Open-Vocabulary Framework for Lidar-Based Object detection. Proceedings of the European Conference on Computer Vision, Milan, Italy."},{"key":"ref_3","unstructured":"Wang, Z., Li, Y., Liu, T., Zhao, H., and Wang, S. (October, January 29). OV-Uni3DETR: Towards Unified Open-Vocabulary 3D Object Detection via Cycle-Modality Propagation. Proceedings of the European Conference on Computer Vision, Milan, Italy."},{"key":"ref_4","unstructured":"Takmaz, A., Fedele, E., Sumner, R.W., Pollefeys, M., Tombari, F., and Engelmann, F. (2023). Openmask3d: Open-vocabulary 3d instance segmentation. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Peng, S., Genova, K., Jiang, C., Tagliasacchi, A., Pollefeys, M., and Funkhouser, T. (2023, January 17\u201324). Openscene: 3D Scene Understanding with Open Vocabularies. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00085"},{"key":"ref_6","unstructured":"He, Q., Peng, J., Jiang, Z., Wu, K., Ji, X., Zhang, J., Wang, Y., Wang, C., Chen, M., and Wu, Y. (2024). UniM-OV3D: Uni-modality open-vocabulary 3D scene understanding with fine-grained feature representation. arXiv."},{"key":"ref_7","unstructured":"Mousavian, A., Eppner, C., and Fox, D. (November, January 27). 6-dof Graspnet: Variational Grasp Generation for Object Manipulation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wen, B., Lian, W., Bekris, K., and Schaal, S. (2022). You only demonstrate once: Category-level manipulation from single visual demonstration. arXiv.","DOI":"10.15607\/RSS.2022.XVIII.044"},{"key":"ref_9","first-page":"13174","article-title":"Grasp proposal networks: An end-to-end solution for visual learning of robotic grasps","volume":"33","author":"Wu","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1162\/pres.1997.6.4.355","article-title":"A survey of augmented reality","volume":"6","author":"Azuma","year":"1997","journal-title":"Presence Teleoperators Virtual Environ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2633","DOI":"10.1109\/TVCG.2015.2513408","article-title":"Pose estimation for augmented reality: A hands-on survey","volume":"22","author":"Marchand","year":"2015","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kehl, W., Manhardt, F., Tombari, F., Ilic, S., and Navab, N. (2017, January 22\u201329). Ssd-6d: Making RGB-Based 3D Detection and 6D Pose Estimation Great again. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.169"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., and Rother, C. (2014, January 6\u201312). Learning 6D Object Pose Estimation Using 3D Object Coordinates. Proceedings of the Computer Vision\u2014ECCV 2014: 13th European Conference, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10605-2_35"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Balntas, V., Doumanoglou, A., Sahin, C., Sock, J., Kouskouridas, R., and Kim, T.K. (2017, January 21\u201326). Pose Guided RGBD Feature Learning for 3D Object Pose Estimation. Proceedings of the IEEE international Conference on Computer Vision, Honolulu, HI, USA.","DOI":"10.1109\/ICCV.2017.416"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Haugaard, R.L., and Buch, A.G. (2022, January 18\u201324). Surfemb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddings. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00663"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wang, H., Sridhar, S., Huang, J., Valentin, J., Song, S., and Guibas, L.J. (2019, January 15\u201320). Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00275"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Tian, M., Ang, M.H., and Lee, G.H. (2020, January 23\u201328). Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation. Proceedings of the Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK.","DOI":"10.1007\/978-3-030-58589-1_32"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Nguyen, V.N., Hu, Y., Xiao, Y., Salzmann, M., and Lepetit, V. (2022, January 18\u201324). Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00665"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wang, T., Hu, G., and Wang, H. (2024, January 16\u201322). Object Pose Estimation via the Aggregation of Diffusion Features. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.00975"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Sundermeyer, M., Durner, M., Puang, E.Y., Marton, Z.C., Vaskevicius, N., Arras, K.O., and Triebel, R. (2020, January 13\u201319). Multi-Path Learning for Object Pose Estimation Across Domains. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01393"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhao, C., Hu, Y., and Salzmann, M. (2022, January 23\u201327). Fusing Local Similarities for Retrieval-Based 3D Orientation Estimation of Unseen Objects. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-19769-7_7"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Cai, J., He, Y., Yuan, W., Zhu, S., Dong, Z., Bo, L., and Chen, Q. (2024). Ov9d: Open-vocabulary category-level 9d object pose and size estimation. arXiv.","DOI":"10.1109\/LRA.2024.3430156"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Schmidt, T., Narayanan, V., and Fox, D. (2017). Posecnn: A convolutional neural network for 6D object pose estimation in cluttered scenes. arXiv.","DOI":"10.15607\/RSS.2018.XIV.019"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, K., and Dou, Q. (2021, January 11\u201317). Sgpa: Structure-Guided Prior Adaptation for Category-Level 6D Object Pose Estimation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00277"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zheng, L., Wang, C., Sun, Y., Dasgupta, E., Chen, H., Leonardis, A., Zhang, W., and Chang, H.J. (2023, January 17\u201324). Hs-pose: Hybrid Scope Feature Extraction for Category-Level Object Pose Estimation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01646"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liu, J., Chen, Y., Ye, X., and Qi, X. (2023, January 2\u20136). Ist-Net: Prior-Free Category-Level Pose Estimation with Implicit Space Transformation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Paris, France.","DOI":"10.1109\/ICCV51070.2023.01285"},{"key":"ref_27","unstructured":"Labb\u00e9, Y., Manuelli, L., Mousavian, A., Tyree, S., Birchfield, S., Tremblay, J., Carpentier, J., Aubry, M., Fox, D., and Sivic, J. (2022). Megapose: 6D pose estimation of novel objects via render & compare. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wen, B., Yang, W., Kautz, J., and Birchfield, S. (2024, January 16\u201322). Foundationpose: Unified 6D Pose Estimation and Tracking of Novel Objects. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.01692"},{"key":"ref_29","first-page":"44860","article-title":"Openshape: Scaling up 3D shape representation towards open-world understanding","volume":"36","author":"Liu","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Xue, L., Gao, M., Xing, C., Mart\u00edn-Mart\u00edn, R., Wu, J., Xiong, C., Xu, R., Niebles, J.C., and Savarese, S. (2023, January 17\u201324). Ulip: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00120"},{"key":"ref_31","unstructured":"Xu, R., Wang, X., Wang, T., Chen, Y., Pang, J., and Lin, D. (October, January 29). Pointllm: Empowering Large Language Models to Understand Point Clouds. Proceedings of the European Conference on Computer Vision, Milan, Italy."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"105892","DOI":"10.1016\/j.euromechsol.2025.105892","article-title":"Prediction model of dynamic fracture toughness of nickel-based alloys: Combination of data-driven and multi-scale modelling","volume":"116","author":"Wang","year":"2025","journal-title":"Eur. J. Mech. A\/Solids"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"94363","DOI":"10.1109\/ACCESS.2022.3204652","article-title":"Modeling fabric-type actuator using point clouds by deep learning","volume":"10","author":"Peng","year":"2022","journal-title":"IEEE Access"},{"key":"ref_34","unstructured":"Nichol, A., Jun, H., Dhariwal, P., Mishkin, P., and Chen, M. (2022). Point-e: A system for generating 3D point clouds from complex prompts. arXiv."},{"key":"ref_35","unstructured":"Zhou, J., Wang, J., Ma, B., Liu, Y.S., Huang, T., and Wang, X. (2023). Uni3d: Exploring unified 3D representation at scale. arXiv."},{"key":"ref_36","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16 \u00d7 16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_37","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (July, January Online). Learning Transferable Visual Models from Natural Language Supervision. Proceedings of the International Conference on Machine Learning."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Fan, H., Su, H., and Guibas, L.J. (2017, January 21\u201326). A Point Set Generation Network for 3D Object Reconstruction From a Single Image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.264"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1109\/34.88573","article-title":"Least-squares estimation of transformation parameters between two point patterns","volume":"13","author":"Umeyama","year":"1991","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","first-page":"5105\u2013","article-title":"Pointnet++: Deep hierarchical feature learning on point sets in a metric space","volume":"10","author":"Qi","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_41","unstructured":"Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16\u201321). Rectifier Nonlinearities Improve Neural Network Acoustic Models. Proceedings of the 30th International Conference on International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_42","unstructured":"Loshchilov, I., and Hutter, F. (2017). Decoupled weight decay regularization. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Lee, J., Park, C., Choe, J., Wang, Y.C.F., Kautz, J., Cho, M., and Choy, C. (2025). Mosaic3D: Foundation Dataset and Model for Open-Vocabulary 3D Segmentation. arXiv.","DOI":"10.1109\/CVPR52734.2025.01315"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Jiang, L., Shi, S., and Schiele, B. (2024, January 16\u201322). Open-Vocabulary 3D Semantic Segmentation with Foundation Models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02011"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The kitti dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Li, Y., Wang, G., Ji, X., Xiang, Y., and Fox, D. (2018, January 8\u201314). Deepim: Deep Iterative Matching for 6D Pose Estimation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01231-1_42"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Sajnani, R., Poulenard, A., Jain, J., Dua, R., Guibas, L.J., and Sridhar, S. (2022, January 18\u201324). Condor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01646"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/11\/380\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T03:32:04Z","timestamp":1761708724000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/11\/380"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,28]]},"references-count":47,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["jimaging11110380"],"URL":"https:\/\/doi.org\/10.3390\/jimaging11110380","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,28]]}}}