{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T16:12:05Z","timestamp":1761581525267,"version":"build-2065373602"},"reference-count":32,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2017,2,16]],"date-time":"2017-02-16T00:00:00Z","timestamp":1487203200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, we propose a novel hierarchical framework that combines motion and feature information to implement infrared-visible video registration on nearly planar scenes. In contrast to previous approaches, which involve the direct use of feature matching to find the global homography, the framework adds coarse registration based on the motion vectors of targets to estimate scale and rotation prior to matching. In precise registration based on keypoint matching, the scale and rotation are used in re-location to eliminate their impact on targets and keypoints. To strictly match the keypoints, first, we improve the quality of keypoint matching by using normalized location descriptors and descriptors generated by the histogram of edge orientation. Second, we remove most mismatches by counting the matching directions of correspondences. We tested our framework on a public dataset, where our proposed framework outperformed two recently-proposed state-of-the-art global registration methods in almost all tested videos.<\/jats:p>","DOI":"10.3390\/s17020384","type":"journal-article","created":{"date-parts":[[2017,2,16]],"date-time":"2017-02-16T12:55:34Z","timestamp":1487249734000},"page":"384","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["A Hierarchical Framework Combining Motion and Feature Information for Infrared-Visible Video Registration"],"prefix":"10.3390","volume":"17","author":[{"given":"Xinglong","family":"Sun","sequence":"first","affiliation":[{"name":"School of Optoelectronics, Image Engineering &amp; Video Technology Lab, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tingfa","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Optoelectronics, Image Engineering &amp; Video Technology Lab, Beijing Institute of Technology, Beijing 100081, China"},{"name":"Key Laboratory of Photoelectronic Imaging Technology and System, Ministry of Education of China, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jizhou","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Optoelectronics, Image Engineering &amp; Video Technology Lab, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiangmin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Optoelectronics, Image Engineering &amp; Video Technology Lab, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,2,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"17944","DOI":"10.3390\/s150817944","article-title":"Fusion of Visible and Thermal Descriptors Using Genetic Algorithms for Face Recognition Systems","volume":"15","author":"Hermosilla","year":"2015","journal-title":"Sensors"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.displa.2005.06.007","article-title":"Fusion of visible and infrared imagery for night color vision","volume":"26","author":"Tsagaris","year":"2005","journal-title":"Displays"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.inffus.2016.02.001","article-title":"Infrared and visible image fusion via gradient transfer and total variation minimization","volume":"31","author":"Ma","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Gonz\u00e1lez, A., Fang, Z., Socarras, Y., Serrat, J., V\u00e1zquez, D., Xu, J., and L\u00f3pez, A.M. (2016). Pedestrian Detection at Day\/Night Time with Visible and FIR Cameras: A Comparison. Sensors, 16.","DOI":"10.3390\/s16060820"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"880","DOI":"10.1016\/j.patcog.2007.06.022","article-title":"Integrated multilevel image fusion and match score fusion of visible and infrared face images for robust face recognition","volume":"41","author":"Singh","year":"2008","journal-title":"Pattern Recognit."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1016\/j.cviu.2006.10.008","article-title":"Mutual information based registration of multimodal stereo videos for person tracking","volume":"106","author":"Krotosky","year":"2007","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1376","DOI":"10.1109\/34.735812","article-title":"Robust image corner detection through curvature scale space","volume":"20","author":"Mokhtarian","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"12661","DOI":"10.3390\/s120912661","article-title":"Multispectral image feature points","volume":"12","author":"Aguilera","year":"2012","journal-title":"Sensors"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1007\/s11263-006-6655-0","article-title":"Multiscale fusion of visible and thermal IR images for illumination-invariant face recognition","volume":"71","author":"Kong","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1117\/1.602363","article-title":"Segment-based registration technique for visual-infrared images","volume":"39","author":"Coiras","year":"2000","journal-title":"Opt. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.cviu.2011.10.006","article-title":"An iterative integrated framework for thermal\u2013visible image registration, sensor fusion, and people tracking for video surveillance applications","volume":"116","author":"Torabi","year":"2012","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1007\/s00138-012-0465-x","article-title":"An IR and visible image sequence automatic registration method based on optical flow","volume":"24","author":"Zhang","year":"2013","journal-title":"Mach. Vis. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"St-Charles, P.-L., Bilodeau, G.-A., and Bergevin, R. (2015, January 7\u201312). Online multimodal video registration based on shape matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA.","DOI":"10.1109\/CVPRW.2015.7301293"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sonn, S., Bilodeau, G.-A., and Galinier, P. (2013, January 23\u201328). Fast and accurate registration of visible and infrared videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA.","DOI":"10.1109\/CVPRW.2013.53"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gehrig, S.K., and Rabe, C. (2010, January 13\u201318). Real-time semi-global matching on the CPU. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5543779"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gallup, D., Frahm, J.-M., and Pollefeys, M. (2010, January 13\u201318). Piecewise planar and non-planar stereo for urban scene reconstruction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539804"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Roche, A., Malandain, G., Pennec, X., and Ayache, N. (1998, January 11\u201313). The correlation ratio as a new similarity measure for multimodal image registration. Proceedings of the Springer International Conference on Medical Image Computing and Computer-Assisted Intervention, Cambridge, MA, USA.","DOI":"10.1007\/BFb0056301"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kim, K.S., Lee, J.H., and Ra, J.B. (2005, January 25\u201328). Robust multi-sensor image registration by enhancing statistical correlation. Proceedings of the IEEE 7th International Conference on Information Fusion, Philadelphia, PA, USA.","DOI":"10.1109\/ICIF.2005.1591880"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1023\/A:1007958904918","article-title":"Alignment by maximization of mutual information","volume":"24","author":"Viola","year":"1997","journal-title":"Int. J. Comput. Vis."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1937","DOI":"10.1016\/j.patcog.2014.12.014","article-title":"Feature neighbourhood mutual information for multi-modal image registration: An application to eye fundus imaging","volume":"48","author":"Legg","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.infrared.2014.02.005","article-title":"Thermal-visible registration of human silhouettes: A similarity measure performance evaluation","volume":"64","author":"Bilodeau","year":"2014","journal-title":"Infrared Phys. Technol."},{"key":"ref_22","unstructured":"Hrka\u0107, T., Kalafati\u0107, Z., and Krapac, J. (2007, January 10\u201314). Infrared-visual image registration based on corners and hausdorff distance. Proceedings of the Springer 15th Scandinavian Conference on Image Analysis, Aalborg, Denmark."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1706","DOI":"10.1109\/TIP.2014.2307478","article-title":"Robust point matching via vector field consensus","volume":"23","author":"Ma","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2425","DOI":"10.1109\/TIP.2008.2006441","article-title":"An improved curvature scale-space corner detector and a robust corner matching approach for transformed image identification","volume":"17","author":"Awrangjeb","year":"2008","journal-title":"IEEE Trans. Image Proc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"772","DOI":"10.1016\/j.patcog.2014.09.005","article-title":"Non-rigid visible and infrared face registration via regularized Gaussian fields criterion","volume":"48","author":"Ma","year":"2014","journal-title":"Pattern Recognit."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Bilodeau, G.-A., St-Onge, P.-L., and Garnier, R. (2011, January 20\u201325). Silhouette-based features for visible-infrared registration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.","DOI":"10.1109\/CVPRW.2011.5981676"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1007\/s11263-005-4842-z","article-title":"Feature-based sequence-to-sequence matching","volume":"68","author":"Caspi","year":"2006","journal-title":"Int. J. Comput. Vis."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Torabi, A., Mass\u00e9, G., and Bilodeau, G.-A. (2010, January 13\u201318). Feedback scheme for thermal-visible video registration, sensor fusion, and people tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5543510"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"St-Charles, P.-L., Bilodeau, G.-A., and Bergevin, R. (2015, January 6\u20139). A self-adjusting approach to change detection based on background word consensus. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV.2015.137"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3356","DOI":"10.1016\/j.patcog.2008.04.017","article-title":"Multi-sensor image registration based on intensity and edge orientation information","volume":"41","author":"Kim","year":"2008","journal-title":"Pattern Recognit."},{"key":"ref_32","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201326). Histograms of oriented gradients for human detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/2\/384\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:28:27Z","timestamp":1760207307000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/2\/384"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2,16]]},"references-count":32,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2017,2]]}},"alternative-id":["s17020384"],"URL":"https:\/\/doi.org\/10.3390\/s17020384","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2017,2,16]]}}}