{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T14:41:20Z","timestamp":1775832080295,"version":"3.50.1"},"reference-count":58,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2021,9,20]],"date-time":"2021-09-20T00:00:00Z","timestamp":1632096000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The Qian Xuesen Youth Innovation Foundation from China Aerospace Science and Technology Corporation","award":["2019JY39"],"award-info":[{"award-number":["2019JY39"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Altitude estimation is one of the fundamental tasks of unmanned aerial vehicle (UAV) automatic navigation, where it aims to accurately and robustly estimate the relative altitude between the UAV and specific areas. However, most methods rely on auxiliary signal reception or expensive equipment, which are not always available, or applicable owing to signal interference, cost or power-consuming limitations in real application scenarios. In addition, fixed-wing UAVs have more complex kinematic models than vertical take-off and landing UAVs. Therefore, an altitude estimation method which can be robustly applied in a GPS denied environment for fixed-wing UAVs must be considered. In this paper, we present a method for high-precision altitude estimation that combines the vision information from a monocular camera and poses information from the inertial measurement unit (IMU) through a novel end-to-end deep neural network architecture. Our method has numerous advantages over existing approaches. First, we utilize the visual-inertial information and physics-based reasoning to build an ideal altitude model that provides general applicability and data efficiency for neural network learning. A further advantage is that we have designed a novel feature fusion module to simplify the tedious manual calibration and synchronization of the camera and IMU, which are required for the standard visual or visual-inertial methods to obtain the data association for altitude estimation modeling. Finally, the proposed method was evaluated, and validated using real flight data obtained during a fixed-wing UAV landing phase. The results show the average estimation error of our method is less than 3% of the actual altitude, which vastly improves the altitude estimation accuracy compared to other visual and visual-inertial based methods.<\/jats:p>","DOI":"10.3390\/s21186302","type":"journal-article","created":{"date-parts":[[2021,9,21]],"date-time":"2021-09-21T22:35:20Z","timestamp":1632263720000},"page":"6302","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["VIAE-Net: An End-to-End Altitude Estimation through Monocular Vision and Inertial Feature Fusion Neural Networks for UAV Autonomous Landing"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0080-6306","authenticated-orcid":false,"given":"Xupei","family":"Zhang","sequence":"first","affiliation":[{"name":"Xi\u2019an Microelectronics Technology Institute, Xi\u2019an 710065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhanzhuang","family":"He","sequence":"additional","affiliation":[{"name":"Xi\u2019an Microelectronics Technology Institute, Xi\u2019an 710065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhong","family":"Ma","sequence":"additional","affiliation":[{"name":"Xi\u2019an Microelectronics Technology Institute, Xi\u2019an 710065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peng","family":"Jun","sequence":"additional","affiliation":[{"name":"Sichuan Tengden Technology Co., Ltd., Chengdu 610037, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kun","family":"Yang","sequence":"additional","affiliation":[{"name":"Sichuan Tengden Technology Co., Ltd., Chengdu 610037, China"},{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Templeton, T., Shim, D.H., Geyer, C., and Sastry, S.S. (2007, January 10\u201314). Autonomous vision-based landing and terrain mapping using an MPC-controlled unmanned rotorcraft. Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Rome, Italy.","DOI":"10.1109\/ROBOT.2007.363172"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mohammadi, A., Feng, Y., Zhang, C., Rawashdeh, S., and Baek, S. (2020, January 1\u20134). Vision-based Autonomous Landing Using an MPC-controlled Micro UAV on a Moving Platform. Proceedings of the 2020 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece.","DOI":"10.1109\/ICUAS48674.2020.9214043"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Nilsson, J.O., Gupta, A.K., and H\u00e4ndel, P. (2014, January 27\u201330). Foot-mounted inertial navigation made easy. Proceedings of the 2014 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Busan, Korea.","DOI":"10.1109\/IPIN.2014.7275464"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Siciliano, B., and Khatib, O. (2016). Springer Handbook of Robotics, Springer.","DOI":"10.1007\/978-3-319-32552-1"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gautam, A., Sujit, P., and Saripalli, S. (2014, January 27\u201330). A survey of autonomous landing techniques for UAVs. Proceedings of the 2014 International Conference on Unmanned Aircraft Systems (ICUAS), Orlando, FL, USA.","DOI":"10.1109\/ICUAS.2014.6842377"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/s10846-016-0399-z","article-title":"Vision based autonomous landing of multirotor UAV on moving platform","volume":"85","author":"Araar","year":"2017","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_7","first-page":"7","article-title":"Scale drift-aware large scale monocular SLAM","volume":"2","author":"Strasdat","year":"2010","journal-title":"Robot. Sci. Syst. VI"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Choi, S., Park, J., and Yu, W. (November, January 30). Resolving scale ambiguity for monocular visual odometry. Proceedings of the 2013 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Jeju, Korea.","DOI":"10.1109\/URAI.2013.6677403"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yang, Y., Shen, Q., Li, J., Deng, Z., Wang, H., and Gao, X. (2020). Position and attitude estimation method integrating visual odometer and GPS. Sensors, 20.","DOI":"10.3390\/s20072121"},{"key":"ref_10","unstructured":"Alvertos, N. (1988, January 10\u201313). Resolution limitations and error analysis for stereo camera models. Proceedings of the Conference Proceedings \u201988. IEEE Southeastcon, Knoxville, TN, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gallup, D., Frahm, J.M., Mordohai, P., and Pollefeys, M. (2008, January 23\u201328). Variable baseline\/resolution stereo. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.","DOI":"10.1109\/CVPR.2008.4587671"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"85","DOI":"10.5875\/ausmt.v5i2.460","article-title":"An improvement stereo vision images processing for object distance measurement","volume":"5","author":"Hsu","year":"2015","journal-title":"Int. J. Autom. Smart Technol."},{"key":"ref_13","unstructured":"Luo, X.l., Lv, J.H., and Sun, G. (2020). A visual-inertial navigation method for high-speed unmanned aerial vehicles. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Jung, Y., Lee, D., and Bang, H. (2015, January 24\u201328). Close-range vision navigation and guidance for rotary UAV autonomous landing. Proceedings of the 2015 IEEE International Conference on Automation Science and Engineering (CASE), Gothenburg, Sweden.","DOI":"10.1109\/CoASE.2015.7294102"},{"key":"ref_15","unstructured":"Dagan, E., Mano, O., Stein, G.P., and Shashua, A. (2004, January 14\u201317). Forward collision warning with a single camera. Proceedings of the IEEE Intelligent Vehicles Symposium 2004, Parma, Italy."},{"key":"ref_16","unstructured":"Rosenbaum, D., Gurman, A., Samet, Y., Stein, G.P., and Aloni, D. (2016). Pedestrian Collision Warning System. (9,233,659), U.S. Patent."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1049\/iet-its.2013.0098","article-title":"Monovision-based vehicle detection, distance and relative speed measurement in urban traffic","volume":"8","author":"Arenado","year":"2014","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_18","unstructured":"Saripalli, S., Montgomery, J.F., and Sukhatme, G.S. (2002, January 11\u201315). Vision-based autonomous landing of an unmanned aerial vehicle. Proceedings of the 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), Washington, DC, USA."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Schmidt, T., Hertkorn, K., Newcombe, R., Marton, Z., Suppa, M., and Fox, D. (2015, January 26\u201330). Depth-based tracking with physical constraints for robot manipulation. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7138989"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kolev, S., and Todorov, E. (2015, January 3\u20135). Physically consistent state estimation and system identification for contacts. Proceedings of the 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, Korea.","DOI":"10.1109\/HUMANOIDS.2015.7363481"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Weiss, S., Achtelik, M.W., Lynen, S., Chli, M., and Siegwart, R. (2012, January 14\u201318). Real-time onboard visual-inertial state estimation and self-calibration of mavs in unknown environments. Proceedings of the 2012 IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6225147"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jarry, G., Delahaye, D., and Feron, E. (2020, January 3\u20134). Approach and landing aircraft on-board parameters estimation with lstm networks. Proceedings of the 2020 International Conference on Artificial Intelligence and Data Analytics for Air Transportation (AIDA-AT), Singapore.","DOI":"10.1109\/AIDA-AT48540.2020.9049199"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6358","DOI":"10.1109\/ACCESS.2020.3048342","article-title":"UAV Pose Estimation in GNSS-Denied Environment Assisted by Satellite Imagery Deep Learning Features","volume":"9","author":"Hou","year":"2020","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4910","DOI":"10.1109\/TPWRS.2019.2919157","article-title":"Bayesian state estimation for unobservable distribution systems via deep learning","volume":"34","author":"Mestav","year":"2019","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_25","first-page":"127","article-title":"Galileo: Perceiving physical object properties by integrating a physics engine with deep learning","volume":"28","author":"Wu","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/s10514-015-9459-7","article-title":"Learning state representations with robotic priors","volume":"39","author":"Jonschkowski","year":"2015","journal-title":"Auton. Robot."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"010508","DOI":"10.1103\/PhysRevLett.124.010508","article-title":"Discovering physical concepts with neural networks","volume":"124","author":"Iten","year":"2020","journal-title":"Phys. Rev. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1177\/0278364918770733","article-title":"The limits and potentials of deep learning for robotics","volume":"37","author":"Brock","year":"2018","journal-title":"Int. J. Robot. Res."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chen, C., Rosa, S., Miao, Y., Lu, C.X., Wu, W., Markham, A., and Trigoni, N. (2019, January 15\u201320). Selective sensor fusion for neural visual-inertial odometry. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01079"},{"key":"ref_30","unstructured":"Dai, S., and Wu, Y. (2008, January 23\u201328). Motion from blur. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA."},{"key":"ref_31","first-page":"857","article-title":"Maximizing autonomous performance of fixed-wing unmanned aerial vehicle to reduce motion blur in taken images","volume":"232","author":"Oktay","year":"2018","journal-title":"Proc. Inst. Mech. Eng. Part I J. Syst. Control Eng."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kim, J., Jung, Y., Lee, D., and Shim, D.H. (2014, January 27\u201330). Outdoor autonomous landing on a moving platform for quadrotors using an omnidirectional camera. Proceedings of the 2014 International Conference on Unmanned Aircraft Systems (ICUAS), Orlando, FL, USA.","DOI":"10.1109\/ICUAS.2014.6842381"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Wubben, J., Fabra, F., Calafate, C.T., Krzeszowski, T., Marquez-Barja, J.M., Cano, J.C., and Manzoni, P. (2019). Accurate landing of unmanned aerial vehicles using ground pattern recognition. Electronics, 8.","DOI":"10.3390\/electronics8121532"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Rambach, J.R., Tewari, A., Pagani, A., and Stricker, D. (2016, January 19\u201323). Learning to fuse: A deep learning approach to visual-inertial camera pose estimation. Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Merida, Mexico.","DOI":"10.1109\/ISMAR.2016.19"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Delmerico, J., and Scaramuzza, D. (2018, January 21\u201325). A benchmark comparison of monocular visual-inertial odometry algorithms for flying robots. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8460664"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1177\/0278364914554813","article-title":"Keyframe-based visual\u2013inertial odometry using nonlinear optimization","volume":"34","author":"Leutenegger","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TRO.2018.2853729","article-title":"Vins-mono: A robust and versatile monocular visual-inertial state estimator","volume":"34","author":"Qin","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Mourikis, A.I., and Roumeliotis, S.I. (2007, January 10\u201314). A multi-state constraint Kalman filter for vision-aided inertial navigation. Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Rome, Italy.","DOI":"10.1109\/ROBOT.2007.364024"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1177\/0278364911430419","article-title":"iSAM2: Incremental smoothing and mapping using the Bayes tree","volume":"31","author":"Kaess","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Forster, C., Carlone, L., Dellaert, F., and Scaramuzza, D. (2015). IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation, Georgia Institute of Technology.","DOI":"10.15607\/RSS.2015.XI.006"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Clark, R., Wang, S., Wen, H., Markham, A., and Trigoni, N. (2017, January 4\u20139). Vinet: Visual-inertial odometry as a sequence-to-sequence learning problem. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11215"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhang, H., and Zhu, T. (2018, January 21\u201323). Aircraft Hard Landing Prediction Using LSTM Neural Network. Proceedings of the 2nd International Symposium on Computer Science and Intelligent Control, Stockholm, Sweden.","DOI":"10.1145\/3284557.3284693"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Wang, S., Clark, R., Wen, H., and Trigoni, N. (June, January 29). Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989236"},{"key":"ref_44","unstructured":"Jiao, J., Jiao, J., Mo, Y., Liu, W., and Deng, Z. (2018). MagicVO: End-to-end monocular visual odometry through deep bi-directional recurrent convolutional neural network. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Han, L., Lin, Y., Du, G., and Lian, S. (2019, January 3\u20138). Deepvio: Self-supervised deep learning of monocular visual inertial odometry using 3D geometric constraints. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8968467"},{"key":"ref_46","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_47","unstructured":"Zhang, H., Goodfellow, I., Metaxas, D., and Odena, A. (2019, January 9\u201315). Self-attention generative adversarial networks. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., and Darrell, T. (2015, January 7\u201312). Long-term recurrent convolutional networks for visual recognition and description. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298878"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Khor, H.Q., See, J., Phan, R.C.W., and Lin, W. (2018, January 15\u201319). Enriched long-term recurrent convolutional network for facial micro-expression recognition. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00105"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_51","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Almalioglu, Y., Turan, M., Sari, A.E., Saputra, M.R.U., de Gusm\u00e3o, P.P., Markham, A., and Trigoni, N. (2019). Selfvio: Self-supervised deep monocular visual-inertial odometry and depth estimation. arXiv.","DOI":"10.1109\/ICRA.2019.8793512"},{"key":"ref_54","unstructured":"Eling, C., Klingbeil, L., and Kuhlmann, H. (2014). Development of an RTK-GPS System for Precise Real-Time Positioning of Lightweight UAVs, Herbert Wichmann Verlag."},{"key":"ref_55","first-page":"70","article-title":"RTK GPS","volume":"9","author":"Langley","year":"1998","journal-title":"GPS World"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"2250","DOI":"10.1016\/j.proeng.2012.06.271","article-title":"Vision based autonomous landing of an unmanned aerial vehicle","volume":"38","author":"Anitha","year":"2012","journal-title":"Procedia Eng."},{"key":"ref_57","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Xue, N., Bai, S., Wang, F., Xia, G.S., Wu, T., and Zhang, L. (2019, January 16\u201320). Learning attraction field representation for robust line segment detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00169"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6302\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:02:32Z","timestamp":1760166152000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6302"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,20]]},"references-count":58,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21186302"],"URL":"https:\/\/doi.org\/10.3390\/s21186302","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,20]]}}}