{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T19:11:18Z","timestamp":1772997078038,"version":"3.50.1"},"reference-count":53,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,12]],"date-time":"2021-02-12T00:00:00Z","timestamp":1613088000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the National Science Fund for Distinguished Young Scholars of China","award":["No. 51625501"],"award-info":[{"award-number":["No. 51625501"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>This article addresses the challenge of 6D aircraft pose estimation from a single RGB image during the flight. Many recent works have shown that keypoints-based approaches, which first detect keypoints and then estimate the 6D pose, achieve remarkable performance. However, it is hard to locate the keypoints precisely in complex weather scenes. In this article, we propose a novel approach, called Pose Estimation with Keypoints and Structures (PEKS), which leverages multiple intermediate representations to estimate the 6D pose. Unlike previous works, our approach simultaneously locates keypoints and structures to recover the pose parameter of aircraft through a Perspective-n-Point Structure (PnPS) algorithm. These representations integrate the local geometric information of the object and the topological relationship between components of the target, which effectively improve the accuracy and robustness of 6D pose estimation. In addition, we contribute a dataset for aircraft pose estimation which consists of 3681 real images and 216,000 rendered images. Extensive experiments on our own aircraft pose dataset and multiple open-access pose datasets (e.g., ObjectNet3D, LineMOD) demonstrate that our proposed method can accurately estimate 6D aircraft pose in various complex weather scenes while achieving the comparative performance with the state-of-the-art pose estimation methods.<\/jats:p>","DOI":"10.3390\/rs13040663","type":"journal-article","created":{"date-parts":[[2021,2,12]],"date-time":"2021-02-12T18:45:00Z","timestamp":1613155500000},"page":"663","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Estimating 6D Aircraft Pose from Keypoints and Structures"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4694-7086","authenticated-orcid":false,"given":"Runze","family":"Fan","sequence":"first","affiliation":[{"name":"Key Laboratory of Precision Opto-Mechatronics Technology, Ministry of Education, School of Instrumentation Science and Opto-Electronics Engineering, Beihang University, Beijing 100191, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0835-5792","authenticated-orcid":false,"given":"Ting-Bing","family":"Xu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Precision Opto-Mechatronics Technology, Ministry of Education, School of Instrumentation Science and Opto-Electronics Engineering, Beihang University, Beijing 100191, China"}]},{"given":"Zhenzhong","family":"Wei","sequence":"additional","affiliation":[{"name":"Key Laboratory of Precision Opto-Mechatronics Technology, Ministry of Education, School of Instrumentation Science and Opto-Electronics Engineering, Beihang University, Beijing 100191, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Lowe, D.G. (1999, January 20\u201327). Object recognition from local scale-invariant features. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece.","DOI":"10.1109\/ICCV.1999.790410"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/0600000001","article-title":"Monocular Model-Based 3D Tracking of Rigid Objects: A Survey","volume":"1","author":"Lepetit","year":"2005","journal-title":"Found. Trends Comput. Graph. Vis."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1284","DOI":"10.1177\/0278364911401765","article-title":"The MOPED framework: Object recognition and pose estimation for manipulation","volume":"30","author":"Collet","year":"2011","journal-title":"Int. J. Robot. Res."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, M.Y., Tuzel, O., Veeraraghavan, A., and Chellappa, R. (2010, January 13\u201318). Fast directional chamfer matching. Proceedings of the Seventh IEEE International Conference on Computer Vision, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539837"},{"key":"ref_5","unstructured":"Jurie, F., and Dhome, M. (2001, January 8\u201314). Real Time 3D Template Matching. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Kauai, HI, USA."},{"key":"ref_6","unstructured":"Gu, C., and Ren, X. (2020, January 5\u201311). Discriminative Mixture-of-Templates for Viewpoint Classification. Proceedings of the European Conference on Computer Vision (ECCV), Crete, Greece."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhu, M., Derpanis, K.G., Yang, Y., Brahmbhatt, S., Zhang, M., Phillips, C., Lecce, M., and Daniilidis, K. (June, January 31). Single image 3D object detection and pose estimation for grasping. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.","DOI":"10.1109\/ICRA.2014.6907430"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"876","DOI":"10.1109\/TPAMI.2011.206","article-title":"Gradient Response Maps for Real-Time Detection of Textureless Objects","volume":"34","author":"Hinterstoisser","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Su, H., Qi, C.R., Li, Y., and Guibas, L.J. (2015, January 7\u201313). Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.308"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Schmidt, T., Narayanan, V., and Fox, D. (2018, January 26\u201330). PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes. Proceedings of the Robotics: Science and Systems, Pittsburgh, PA, USA.","DOI":"10.15607\/RSS.2018.XIV.019"},{"key":"ref_11","unstructured":"Wilson, R.C., Hancock, E.R., and Smith, W.A.P. (2016, January 19\u201322). Crafting a multi-task CNN for viewpoint estimation. Proceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1007\/s11263-019-01250-9","article-title":"DeepIM: Deep Iterative Matching for 6D Pose Estimation","volume":"128","author":"Li","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, G., Manhardt, F., Shao, J., Ji, X., Navab, N., and Tombari, F. (2020, January 23\u201328). Self6D: Self-Supervised Monocular 6D Object Pose Estimation. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58452-8_7"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Hu, Y., Fua, P., Wang, W., and Salzmann, M. (2020, January 14\u201319). Single-Stage 6D Object Pose Estimation. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00300"},{"key":"ref_15","unstructured":"Pitteri, G., Bugeau, A., Ilic, S., and Lepetit, V. (December, January 30). 3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings. Proceedings of the Asian Conference on Computer Vision, Perth, Australia, Kyoto, Japan."},{"key":"ref_16","unstructured":"Busam, B., Jung, H.J., and Navab, N. (2020). I Like to Move It: 6D Pose Estimation as an Action Decision Process. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Poirson, P., Ammirato, P., Fu, C.Y., Liu, W., and Berg, A.C. (2016, January 25\u201328). Fast Single Shot Detection and Pose Estimation. Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.","DOI":"10.1109\/3DV.2016.78"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Michel, F., Kirillov, A., Brachmann, E., Krull, A., Gumhold, S., Savchynskyy, B., and Rother, C. (2017, January 21\u201326). Global Hypothesis Generation for 6D Object Pose Estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.20"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kehl, W., Manhardt, F., Tombari, F., Ilic, S., and Navab, N. (2017, January 22\u201329). SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.169"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tekin, B., Sinha, S.N., and Fua, P. (2018, January 18\u201323). Real-Time Seamless Single Shot 6D Object Pose Prediction. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00038"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Peng, S., Liu, Y., Huang, Q., Zhou, X., and Bao, H. (2019, January 16\u201320). PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00469"},{"key":"ref_22","unstructured":"Tremblay, J., To, T., Sundaralingam, B., Xiang, Y., Fox, D., and Birchfield, S. (2018, January 29\u201331). Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects. Proceedings of the 2nd Annual Conference on Robot Learning, CoRL 2018, Z\u00fcrich, Switzerland."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Pavlakos, G., Zhou, X., Chan, A., Derpanis, K.G., and Daniilidis, K. (June, January 29). 6-DoF object pose from semantic keypoints. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Marina Bay Sands, Singapore.","DOI":"10.1109\/ICRA.2017.7989233"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Brachmann, E., Michel, F., Krull, A., Yang, M.Y., Gumhold, S., and Rother, C. (2016, January 27\u201330). Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.366"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Rad, M., and Lepetit, V. (2017, January 22\u201329). BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.413"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hodan, T., Barath, D., and Matas, J. (2020, January 14\u201319). EPOS: Estimating 6D Pose of Objects With Symmetries. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01172"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"He, Y., Sun, W., Huang, H., Liu, J., Fan, H., and Sun, J. (2020, January 14\u201319). PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01165"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chen, X., Dong, Z., Song, J., Geiger, A., and Hilliges, O. (2020, January 23\u201328). Category Level Object Pose Estimation via Neural Analysis-by-Synthesis. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.","DOI":"10.1007\/978-3-030-58574-7_9"},{"key":"ref_29","unstructured":"Manhardt, F., Wang, G., Busam, B., Nickel, M., Meier, S., Minciullo, L., Ji, X., and Navab, N. (2020). CPS++: Improving Class-level 6D Pose and Shape Estimation From Monocular Images With Self-Supervised Learning. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Song, C., Song, J., and Huang, Q. (2020, January 13\u201319). HybridPose: 6D Object Pose Estimation Under Hybrid Representations. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00051"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1007\/s11263-008-0152-6","article-title":"EPnP: An Accurate O(n) Solution to the PnP Problem","volume":"81","author":"Lepetit","year":"2009","journal-title":"Int. J. Comput. Vis."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Kuang, Y., Sugimoto, S., Strm, K., and Okutomi, M. (2013, January 3\u20136). Revisiting the PnP Problem: A Fast, General and Optimal Solution. Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.291"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"103","DOI":"10.14358\/PERS.81.2.103","article-title":"Direct Linear Transformation from Comparator Coordinates into Object Space Coordinates in Close-Range Photogrammetry","volume":"81","author":"Karara","year":"2015","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"012183","DOI":"10.1088\/1755-1315\/17\/1\/012183","article-title":"Aerial Image Series Quality Assessment","volume":"17","author":"Zhang","year":"2014","journal-title":"IOP Conf. Ser. Earth Environ. Sci."},{"key":"ref_35","first-page":"404","article-title":"SURF: Speeded Up Robust Features","volume":"110","author":"Bay","year":"2006","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kendall, A., Grimes, M., and Cipolla, R. (2015, January 7\u201313). PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.336"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Scovanner, P., Ali, S., and Shah, M. (2007). A 3-Dimensional SIFT Descriptor and its Application to Action Recognition. Acm Int. Conf. Multimed., 357.","DOI":"10.1145\/1291233.1291311"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Bloesch, M., Czarnowski, J., Clark, R., Leutenegger, S., and Davison, A.J. (2018, January 18\u201323). CodeSLAM\u2014 Learning a Compact, Optimisable Representation for Dense Visual SLAM. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00271"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Hsiao, M., Westman, E., Zhang, G., and Kaess, M. (June, January 29). Keyframe-based dense planar SLAM. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Marina Bay Sands, Singapore.","DOI":"10.1109\/ICRA.2017.7989597"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.robot.2018.02.018","article-title":"Probabilistic RGB-D Odometry based on Points, Lines and Planes Under Depth Uncertainty","volume":"104","author":"Proenca","year":"2017","journal-title":"Robot. Auton. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Newell, A., Yang, K., and Deng, J. (2016, January 8\u201316). Stacked Hourglass Networks for Human Pose Estimation. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46484-8_29"},{"key":"ref_43","unstructured":"Busto, P.P., and Gall, J. (2019, January 9\u201312). Joint Viewpoint and Keypoint Estimation with Real and Synthetic Data. Proceedings of the German Conference on Pattern Recognition, Stuttgart, Germany."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wei, S., Ramakrishna, V., Kanade, T., and Sheikh, Y. (2016, January 27\u201330). Convolutional Pose Machines. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.511"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Mousavian, A., Anguelov, D., Flynn, J., and Kosecka, J. (2017, January 21\u201326). 3D Bounding Box Estimation Using Deep Learning and Geometry. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.597"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Teng, X., Yu, Q., Luo, J., Zhang, X., and Wang, G. (2019). Pose Estimation for Straight Wing Aircraft Based on Consistent Line Clustering and Planes Intersection. Sensors, 19.","DOI":"10.3390\/s19020342"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1007\/978-3-319-46478-7_36","article-title":"Accurate and Linear Time Pose Estimation from Points and Lines","volume":"Volume 9911","author":"Leibe","year":"2016","journal-title":"Proceedings of the Computer Vision\u2014ECCV 2016\u201414th European Conference"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The KITTI dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1007\/978-3-319-46484-8_10","article-title":"ObjectNet3D: A Large Scale Database for 3D Object Recognition","volume":"Volume 9912","author":"Leibe","year":"2016","journal-title":"Proceedings of the Computer Vision\u2014ECCV 2016\u201414th European Conference"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., and Navab, N. (2012, January 5\u20139). Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes. Proceedings of the Asian Conference on Computer Vision, Daejeon, Korea.","DOI":"10.1007\/978-3-642-33885-4_60"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Tulsiani, S., and Malik, J. (2015, January 7\u201312). Viewpoints and keypoints. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298758"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Zakharov, S., Shugurov, I., and Ilic, S. (2019, January 27\u201328). DPOD: 6D Pose Object Detector and Refiner. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00203"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Li, Z., Wang, G., and Ji, X. (2019, January 27\u201328). CDPN: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6-DoF Object Pose Estimation. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00777"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/4\/663\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:23:19Z","timestamp":1760160199000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/4\/663"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,12]]},"references-count":53,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["rs13040663"],"URL":"https:\/\/doi.org\/10.3390\/rs13040663","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,12]]}}}