{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T11:56:07Z","timestamp":1773834967353,"version":"3.50.1"},"reference-count":67,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2017,5,5]],"date-time":"2017-05-05T00:00:00Z","timestamp":1493942400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Project","award":["2016YFB1200100"],"award-info":[{"award-number":["2016YFB1200100"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>This study proposes a novel method for image registration and fusion via commonly used visible light and infrared integrated cameras mounted on medium-altitude unmanned aerial vehicles (UAVs).The innovation of image registration lies in three aspects. First, it reveals how complex perspective transformation can be converted to simple scale transformation and translation transformation between two sensor images under long-distance and parallel imaging conditions. Second, with the introduction of metadata, a scale calculation algorithm is designed according to spatial geometry, and a coarse translation estimation algorithm is presented based on coordinate transformation. Third, the problem of non-strictly aligned edges in precise translation estimation is solved via edge\u2013distance field transformation. A searching algorithm based on particle swarm optimization is introduced to improve efficiency. Additionally, a new image fusion algorithm is designed based on a pulse coupled neural network and nonsubsampled contourlet transform to meet the special requirements of preserving color information, adding infrared brightness information, improving spatial resolution, and highlighting target areas for unmanned aerial vehicle (UAV) applications. A medium-altitude UAV is employed to collect datasets. The result is promising, especially in applications that involve other medium-altitude or high-altitude UAVs with similar system structures.<\/jats:p>","DOI":"10.3390\/rs9050441","type":"journal-article","created":{"date-parts":[[2017,5,5]],"date-time":"2017-05-05T10:31:08Z","timestamp":1493980268000},"page":"441","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":61,"title":["Image Registration and Fusion of Visible and Infrared Integrated Camera for Medium-Altitude Unmanned Aerial Vehicle Remote Sensing"],"prefix":"10.3390","volume":"9","author":[{"given":"Hongguang","family":"Li","sequence":"first","affiliation":[{"name":"Unmanned Systems Research Institute, Beihang University, Beijing 100191, China"},{"name":"School of Mechanical Engineering and Automation, Beihang University, Beijing 100191, China"}]},{"given":"Wenrui","family":"Ding","sequence":"additional","affiliation":[{"name":"Unmanned Systems Research Institute, Beihang University, Beijing 100191, China"},{"name":"School of Electronic and Information Engineering, Beihang University, Beijing 100191, China"},{"name":"Collaborative Innovation Centre of Geospatial Technology, Wuhan 430000, China"}]},{"given":"Xianbin","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Beihang University, Beijing 100191, China"}]},{"given":"Chunlei","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Beihang University, Beijing 100191, China"}]}],"member":"1968","published-online":{"date-parts":[[2017,5,5]]},"reference":[{"key":"ref_1","first-page":"419","article-title":"On space-air-ground integrated earth observation network","volume":"14","author":"Deren","year":"2012","journal-title":"J. Geo-Inf. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.infrared.2012.11.003","article-title":"Fusion of visible and infrared images using saliency analysis and detail preserving based image decomposition","volume":"56","author":"Zhao","year":"2013","journal-title":"Infrared Phys. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhou, D., Zhong, Z., Zhang, D., Shen, L., and Yan, C. (2015, January 18\u201322). Autonomous landing of a helicopter UAV with a ground-based multisensory fusion system. Proceedings of the International Conference on Machine Vision, Koto-ku, Japan.","DOI":"10.1117\/12.2183270"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1049\/iet-ipr.2009.0374","article-title":"New method for the fusion of complementary information from infrared and visual images for object detection","volume":"5","author":"Ulusoy","year":"2011","journal-title":"IET Image Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"589","DOI":"10.4028\/www.scientific.net\/AMM.128-129.589","article-title":"Fusion of infrared and visible image based on target regions for environment perception","volume":"128\u2013129","author":"Niu","year":"2011","journal-title":"Appl. Mech. Mater."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Pulpea, B.G. (2015, January 11\u201313). Aspects regarding the development of pyrotechnic obscurant systems for visible and infrared protection of military vehicles. Proceedings of the International Conference Knowledge-Based Organization, Land Forces Academy, Sibiu, Romania.","DOI":"10.1515\/kbo-2015-0123"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.envsoft.2015.11.024","article-title":"Assimilating satellite imagery and visible-near infrared spectroscopy to model and map soil loss by water erosion in australia","volume":"77","author":"Teng","year":"2016","journal-title":"Environ. Model. Softw."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5609","DOI":"10.3390\/s150305609","article-title":"Quantifying efficacy and limits of unmanned aerial vehicle (UAV) technology for weed seedling detection as affected by sensor resolution","volume":"15","year":"2015","journal-title":"Sensors"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1002\/wsb.629","article-title":"Visible and thermal infrared remote sensing for the detection of white-tailed deer using an unmanned aerial system","volume":"40","year":"2016","journal-title":"Wildl. Soc. Bull."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhao, B., Li, Z., Liu, M., Cao, W., and Liu, H. (2013, January 12\u201314). Infrared and visible imagery fusion based on region saliency detection for 24-h-surveillance systems. Proceedings of the IEEE International Conference on Robotics and Biomimetics, Shenzhen, China.","DOI":"10.1109\/ROBIO.2013.6739608"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.patrec.2012.06.002","article-title":"Multi-spectral saliency detection","volume":"34","author":"Wang","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1748","DOI":"10.1016\/j.cviu.2013.07.002","article-title":"Multi-spectral dataset and its application in saliency detection","volume":"117","author":"Wang","year":"2013","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"20845","DOI":"10.3390\/s150820845","article-title":"Distance-dependent multimodal image registration for agriculture tasks","volume":"15","author":"Berenstein","year":"2014","journal-title":"Sensors"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2223","DOI":"10.1016\/S0031-3203(01)00177-7","article-title":"Robust image registration by increment sign correlation","volume":"35","author":"Kaneko","year":"2010","journal-title":"Pattern Recognit."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Tsin, Y., and Kanade, T. (2004, January 11\u201314). A correlation-based approach to robust point set registration. Proceedings of the Computer Vision\u2014ECCV 2004, European Conference on Computer Vision, Prague, Czech Republic.","DOI":"10.1007\/978-3-540-24672-5_44"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1016\/j.ijleo.2015.09.199","article-title":"Infrared and visual image registration based on mutual information with a combined particle swarm optimization\u2014Powell search algorithm","volume":"127","author":"Zhuang","year":"2016","journal-title":"Optik\u2014Int. J. Light Electron Opt."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Yang, G., Chen, D., Li, J., and Yang, W. (2016, January 4\u20137). Registration of infrared and visual images based on phase grouping and mutual information of gradient orientation. Proceedings of the SPIE Photonics Europe, Brussels, Belgium.","DOI":"10.1117\/12.2223359"},{"key":"ref_18","first-page":"145","article-title":"Ir and visible images registration method based on cross cumulative residual entropy","volume":"8704","author":"Li","year":"2013","journal-title":"Proc. SPIE\u2014Int. Soc. Opt. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4514","DOI":"10.1364\/AO.54.004514","article-title":"Image registration under translation and rotation in two-dimensional planes using fourier slice theorem","volume":"54","author":"Pohit","year":"2015","journal-title":"Appl. Opt."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3889","DOI":"10.1016\/j.ijleo.2015.07.172","article-title":"Image registration based on fractional fourier transform","volume":"126","author":"Niu","year":"2015","journal-title":"Optik\u2014Int. J. Light Electron Opt."},{"key":"ref_21","first-page":"1","article-title":"A registration scheme for multispectral systems using phase correlation and scale invariant feature matching","volume":"2016","author":"Li","year":"2016","journal-title":"J. Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.neucom.2012.08.012","article-title":"Image registration by normalized mapping","volume":"101","author":"Wang","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Lowe, D.G. (1999, January 20\u201327). Object recognition from local scale-invariant features. Proceedings of the IEEE International Conference on Computer Vision, Kerkyra, Greece.","DOI":"10.1109\/ICCV.1999.790410"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Huang, Q., Yang, J., Wang, C., Chen, J., and Meng, Y. (2012, January 22\u201327). Improved registration method for infrared and visible remote sensing image using nsct and sift. Proceedings of the Geoscience and Remote Sensing Symposium, Munich, Germany.","DOI":"10.1109\/IGARSS.2012.6351020"},{"key":"ref_26","unstructured":"Li, D. (2017, April 24). An Infrared and Visible Image Registration Based on Surf. 2012, 19\u201325. Available online: https:\/\/datahub.io\/dataset\/an-infrared-and-visible-image-registration-based-on-surf."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1117\/1.602363","article-title":"Segment-based registration technique for visual-infrared images","volume":"39","author":"Coiras","year":"2000","journal-title":"Opt. Eng."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Han, J., Pauwels, E., and Zeeuw, P.D. (2012). Visible and Infrared Image Registration Employing Line-Based Geometric Analysis, Springer.","DOI":"10.1007\/978-3-642-32436-9_10"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1179\/1743131X12Y.0000000046","article-title":"Multi-spectral image registration and evaluation based on edge-enhanced mser","volume":"62","author":"Liu","year":"2014","journal-title":"Imaging Sci. J."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"053017","DOI":"10.1117\/1.JEI.24.5.053017","article-title":"Visible and infrared image registration based on visual salient features","volume":"24","author":"Qin","year":"2015","journal-title":"J. Electron. Imaging"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"10562","DOI":"10.3390\/s140610562","article-title":"Diffusion maps for multimodal registration","volume":"14","author":"Piella","year":"2014","journal-title":"Sensors"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1308","DOI":"10.4028\/www.scientific.net\/AMM.182-183.1308","article-title":"An automatic registration method for multi-modal images based on alignment metric","volume":"182\u2013183","author":"Xia","year":"2012","journal-title":"Appl. Mech. Mater."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.patrec.2012.03.022","article-title":"Visible and infrared image registration in man-made environments employing hybrid visual features","volume":"34","author":"Han","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Huang, Q., Yang, J., Chen, J., Gao, Q., and Song, Z. (2014). Visible and infrared image registration algorithm based on nsct and gradient mirroring. Proc. SPIE\u2014Multispectr. Hyperspectr. Ultraspectr. Remote Sens. Technol. Tech. Appl.","DOI":"10.1117\/12.2068931"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1640","DOI":"10.1080\/01431161.2014.880819","article-title":"Infrared and visible image fusion based on random projection and sparse representation","volume":"35","author":"Wang","year":"2014","journal-title":"Int. J. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1080\/014311698215748","article-title":"Review article multisensor image fusion in remote sensing: Concepts, methods and applications","volume":"19","author":"Pohl","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1166\/jmihi.2016.1811","article-title":"Multi-modal medical image fusion using RGB-principal component analysis","volume":"6","author":"Nawaz","year":"2016","journal-title":"J. Med. Imaging Health Inf."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"650","DOI":"10.1117\/1.600657","article-title":"New false color mapping for image fusion","volume":"35","author":"Toet","year":"1996","journal-title":"Opt. Eng."},{"key":"ref_39","first-page":"302","article-title":"Quick markov random field image fusion","volume":"3374","author":"Kadar","year":"1998","journal-title":"Proc. SPIE\u2014Int. Soc. Opt. Eng."},{"key":"ref_40","first-page":"1364","article-title":"Bayesian sensor image fusion using local linear generative models","volume":"40","author":"Sharma","year":"2002","journal-title":"Opt. Eng."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1325","DOI":"10.1016\/S0165-1684(00)00273-5","article-title":"Image fusion based on median filters and sofm neural networks: A three-step scheme","volume":"81","author":"Zhang","year":"2001","journal-title":"Signal Process."},{"key":"ref_42","first-page":"357","article-title":"A novel pulse coupled neural network based method for multi-focus image fusion","volume":"12","author":"Zhang","year":"2014","journal-title":"Int. J. Signal Process. Image Process. Pattern Recognit."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Guan, W., Li, L., Jin, W., Qiu, S., and Zou, Y. (2015, January 5\u20137). Research on hdr image fusion algorithm based on laplace pyramid weight transform with extreme low-light CMOS. Proceedings of the Applied Optics and Photonics China, Beijing, China.","DOI":"10.1117\/12.2199818"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1006\/gmip.1995.1022","article-title":"Multisensor image fusion using the wavelet transform","volume":"57","author":"Li","year":"1995","journal-title":"Graph. Models Image Process."},{"key":"ref_45","unstructured":"Chen, T., Zhang, J., and Zhang, Y. (2005, January 25\u201329). In Remote sensing image fusion based on ridgelet transform. Proceedings of the 2005 IEEE International Geoscience and Remote Sensing Symposium, Seoul, South Korea."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Lutz, A., Giansiracusa, M., Messer, N., Ezekiel, S., Blasch, E., and Alford, M. (2016, January 17\u201321). Optimal multi-focus contourlet-based image fusion algorithm selection. Proceedings of the SPIE Defense + Security, Baltimore, MD, USA.","DOI":"10.1117\/12.2224325"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1334","DOI":"10.1016\/j.sigpro.2009.01.012","article-title":"Multifocus image fusion using the nonsubsampled contourlet transform","volume":"89","author":"Zhang","year":"2009","journal-title":"Signal Process."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Han, J., Loffeld, O., Hartmann, K., and Wang, R. (2010, January 23\u201325). Multi image fusion based on compressive sensing. Proceedings of the International Conference on Audio Language and Image Processing, Shanghai, China.","DOI":"10.1109\/ICALIP.2010.5684502"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"3658","DOI":"10.1109\/TGRS.2014.2381272","article-title":"Hyperspectral and multispectral image fusion based on a sparse representation","volume":"53","author":"Wei","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_50","unstructured":"Zhang, Q., Liu, Y., Blum, R.S., Han, J., and Tao, D. (2017, May 04). Sparse Representation Based Multi-Sensor Image Fusion: A Review. Available online: https:\/\/arxiv.org\/abs\/1702.03515."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.neucom.2012.12.015","article-title":"Fast saliency-aware multi-modality image fusion","volume":"111","author":"Han","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/S1000-9361(08)60071-0","article-title":"Fusion of infrared and visible light images based on region segmentation","volume":"22","author":"Liu","year":"2009","journal-title":"Chin. J. Aeronaut."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Sturm, P. (2014). Pinhole Camera Model, Springer.","DOI":"10.1007\/978-0-387-31439-6_472"},{"key":"ref_54","first-page":"1865","article-title":"Multiple view geometry in computer vision","volume":"30","author":"Hartley","year":"2001","journal-title":"Kybernetes"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"12606","DOI":"10.3390\/rs71012606","article-title":"Metadata-assisted global motion estimation for medium-altitude unmanned aerial vehicle video applications","volume":"7","author":"Li","year":"2015","journal-title":"Remote Sens."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1628","DOI":"10.1109\/TCSVT.2008.2005611","article-title":"Broadcast court-net sports video analysis using fast 3-d camera modeling","volume":"18","author":"Han","year":"2008","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A computational approach to edge detection","volume":"8","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Jie, G., and Ning, L. (2012, January 28\u201330). An improved adaptive threshold canny edge detection algorithm. Proceedings of the International Conference on Computer Science and Electronics Engineering, Colchester, UK.","DOI":"10.1109\/ICCSEE.2012.154"},{"key":"ref_59","first-page":"37","article-title":"Matching Multi\u2014Sensor Images Based on Edge Similarity","volume":"30","author":"Li","year":"2011","journal-title":"J. Spacecr. TTC Technol."},{"key":"ref_60","unstructured":"Kennedy, J., and Eberhart, R. (December, January 27). Particle swarm optimization. Proceedings of the IEEE International Conference on Neural Networks, Perth, Australia."},{"key":"ref_61","unstructured":"Song, Y., and Eberhart, R. (1998, January 4\u20139). A modified particle swarm optimizer. Proceedings of the 1998 IEEE International Conference on Evolutionary Computation Proceedings, Anchorage, AK, USA."},{"key":"ref_62","first-page":"1648","article-title":"On the performance of linear decreasing inertia weight particle swarm optimization for global optimization","volume":"78","author":"Arasomwan","year":"2013","journal-title":"Sci. World J."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1109\/72.761716","article-title":"Perfect image segmentation using pulse coupled neural networks","volume":"10","author":"Kuntimad","year":"1999","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"882","DOI":"10.1109\/78.127960","article-title":"A filter bank for the directional decomposition of images: Theory and design","volume":"40","author":"Bamberger","year":"1992","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Hore, A., and Ziou, D. (2010, January 23\u201326). Image quality metrics: PSNR vs. SSIM. Proceedings of the 2010 20th International Conference on Pattern Recognition (ICPR), Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.579"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1016\/j.proeng.2010.11.045","article-title":"Multimodal medical image fusion based on IHS and PCA","volume":"7","author":"He","year":"2010","journal-title":"Procedia Eng."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Xin, W., Wei, Y.L., and Fu, L. (2013, January 26\u201328). A new multi-source image sequence fusion algorithm based on sidwt. Proceedings of the Seventh International Conference on Image and Graphics, Qingdao, China.","DOI":"10.1109\/ICIG.2013.119"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/9\/5\/441\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:34:43Z","timestamp":1760207683000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/9\/5\/441"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,5,5]]},"references-count":67,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2017,5]]}},"alternative-id":["rs9050441"],"URL":"https:\/\/doi.org\/10.3390\/rs9050441","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,5,5]]}}}