{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T15:09:12Z","timestamp":1761664152464,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2020,9,24]],"date-time":"2020-09-24T00:00:00Z","timestamp":1600905600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Chinese Scholarship Council","award":["201706270113"],"award-info":[{"award-number":["201706270113"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41701540"],"award-info":[{"award-number":["41701540"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Traditional stereo dense image matching (DIM) methods normally predefine a fixed window to compute matching cost, while their performances are limited by the matching window sizes. A large matching window usually achieves robust matching results in weak-textured regions, while it may cause over-smoothness problems in disparity jumps and fine structures. A small window can recover sharp boundaries and fine structures, while it contains high matching uncertainties in weak-textured regions. To address the issue above, we respectively compute matching results with different matching window sizes and then proposes an adaptive fusion method of these matching results so that a better matching result can be generated. The core algorithm designs a Convolutional Neural Network (CNN) to predict the probabilities of large and small windows for each pixel and then refines these probabilities by imposing a global energy function. A compromised solution of the global energy function is utilized by breaking the optimization into sub-optimizations of each pixel in one-dimensional (1D) paths. Finally, the matching results of large and small windows are fused by taking the refined probabilities as weights for more accurate matching. We test our method on aerial image datasets, satellite image datasets, and Middlebury benchmark with different matching cost metrics. Experiments show that our proposed adaptive fusion of multiple-window matching results method has a good transferability across different datasets and outperforms the small windows, the median windows, the large windows, and some state-of-the-art matching window selection methods.<\/jats:p>","DOI":"10.3390\/rs12193138","type":"journal-article","created":{"date-parts":[[2020,9,24]],"date-time":"2020-09-24T08:33:03Z","timestamp":1600936383000},"page":"3138","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Stereo Dense Image Matching by Adaptive Fusion of Multiple-Window Matching Results"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4085-6279","authenticated-orcid":false,"given":"Yilong","family":"Han","sequence":"first","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China"}]},{"given":"Wei","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Civil, Environmental and Geodetic Engineering, The Ohio State University, Columbus, OH 43221, USA"}]},{"given":"Xu","family":"Huang","sequence":"additional","affiliation":[{"name":"Department of Civil, Environmental and Geodetic Engineering, The Ohio State University, Columbus, OH 43221, USA"}]},{"given":"Shugen","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5896-1379","authenticated-orcid":false,"given":"Rongjun","family":"Qin","sequence":"additional","affiliation":[{"name":"Department of Civil, Environmental and Geodetic Engineering, The Ohio State University, Columbus, OH 43221, USA"},{"name":"Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH 43221, USA"},{"name":"Translational Data Analytics, The Ohio State University, Columbus, OH 43221, USA"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1582","DOI":"10.1109\/TPAMI.2008.221","article-title":"Evaluation of stereo matching costs on images with radiometric differences","volume":"31","author":"Scharstein","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Yuan, W., Yuan, X., Xu, S., Gong, J., and Shibasaki, R. (2019). Dense image-matching via optical flow field estimation and fast-guided filter refinement. Remote Sens., 11.","DOI":"10.3390\/rs11202410"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ye, Z., Xu, Y., Chen, H., Zhu, J., Tong, X., and Stilla, U. (2020). Area-Based dense image matching with subpixel accuracy for remote sensing applications: Practical analysis and comparative study. Remote Sens., 12.","DOI":"10.3390\/rs12040696"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"8310","DOI":"10.3390\/rs6098310","article-title":"Building change detection from historical aerial photographs using dense image matching and object-based image analysis","volume":"6","author":"Nebiker","year":"2014","journal-title":"Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.cageo.2017.07.001","article-title":"WASS: An open-source pipeline for 3D stereo reconstruction of ocean waves","volume":"107","author":"Bergamasco","year":"2017","journal-title":"Comput. Geosci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1111\/phor.12310","article-title":"Assessment of dense image matchers for digital surface model generation using airborne and spaceborne images\u2013an update","volume":"35","author":"Han","year":"2020","journal-title":"Photogramm. Rec."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1111\/phor.12063","article-title":"State of the art in high density image matching","volume":"29","author":"Remondino","year":"2014","journal-title":"Photogramm. Rec."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.cageo.2013.05.013","article-title":"Generation of pixel-level resolution lunar DEM based on Chang\u2019E-1 three-line imagery and laser altimeter data","volume":"59","author":"Zhang","year":"2013","journal-title":"Comput. Geosci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"351","DOI":"10.5194\/isprs-annals-V-2-2020-351-2020","article-title":"State of the art in digital surface modelling from multi-view high-resolution satellite images","volume":"2","author":"Han","year":"2020","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1023\/A:1014573219977","article-title":"A taxonomy and evaluation of dense two-frame stereo correspondence algorithms","volume":"47","author":"Scharstein","year":"2002","journal-title":"Int. J. Comput. Vis."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1109\/34.677269","article-title":"A pixel dissimilarity measure that is insensitive to image sampling","volume":"20","author":"Birchfield","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mei, X., Sun, X., Zhou, M., Jiao, S., Wang, H., and Zhang, X. (2011, January 6\u201313). On building an accurate stereo matching system on graphics hardware. Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130280"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hermann, S., and Vaudrey, T. (2010, January 8\u20139). The gradient-a powerful and robust cost function for stereo matching. Proceedings of the 2010 25th International Conference of Image and Vision Computing New Zealand, Queenstown, New Zealand.","DOI":"10.1109\/IVCNZ.2010.6148804"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"67","DOI":"10.5194\/isprs-annals-III-3-67-2016","article-title":"Image-guided non-local dense matching with three-steps optimization","volume":"III-3","author":"Huang","year":"2016","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhou, X., and Boulanger, P. (October, January 30). Radiometric invariant stereo matching based on relative gradients. Proceedings of the 2012 19th IEEE International Conference on Image Processing, Orlando, FL, USA.","DOI":"10.1109\/ICIP.2012.6467528"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1109\/TIP.2015.2395820","article-title":"Accurate stereo matching by two-step energy minimization","volume":"24","author":"Mozerov","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zabih, R., and Woodfill, J. (1994, January 2\u20136). Non-parametric local transforms for computing visual correspondence. Proceedings of the European Conference on Computer Vision, Stockholm, Sweden.","DOI":"10.1007\/BFb0028345"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1109\/TMM.2012.2225041","article-title":"Consistent stereo matching under varying radiometric conditions","volume":"15","author":"Jung","year":"2012","journal-title":"IEEE Trans. MultiMedia"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1109\/MMUL.2014.51","article-title":"Local stereo matching with improved matching cost and disparity refinement","volume":"21","author":"Jiao","year":"2014","journal-title":"IEEE MultiMedia"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.imavis.2014.12.001","article-title":"Enhanced disparity estimation in stereo images","volume":"35","author":"Kordelas","year":"2015","journal-title":"Image Vis. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.cageo.2016.09.002","article-title":"An efficient photogrammetric stereo matching method for high-resolution images","volume":"97","author":"Li","year":"2016","journal-title":"Comput. Geosci."},{"key":"ref_22","first-page":"2","article-title":"Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches","volume":"17","author":"Zbontar","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1109\/TPAMI.2007.1166","article-title":"Stereo Processing by Semiglobal Matching and Mutual Information","volume":"30","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","unstructured":"Erway, C.C., and Ransford, B. (2017). Variable Window Methods for Stereo Disparity Determination. Machine Vision."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Lin, P.-H., Yeh, J.-S., Wu, F.-C., and Chuang, Y.-Y. (2017). Depth estimation for lytro images by adaptive window matching on EPI. J. Imaging, 3.","DOI":"10.3390\/jimaging3020017"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Emlek, A., Peker, M., and Dilaver, K.F. (2017, January 16\u201317). Variable window size for stereo image matching based on edge information. Proceedings of the 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkey.","DOI":"10.1109\/IDAP.2017.8090229"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Koo, H.-S., and Jeong, C.-S. (2001, January 28\u201330). An area-based stereo matching using adaptive search range and window size. Proceedings of the International Conference on Computational Science, San Francisco, USA.","DOI":"10.1007\/3-540-45718-6_6"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.isprsjprs.2012.02.002","article-title":"Locally adaptive template sizes for matching repeat images of Earth surface mass movements","volume":"69","year":"2012","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"He, Y., Wang, P., and Fu, J. (2013, January 24\u201325). An Adaptive Window Stereo Matching Based on Gradient. Proceedings of the 3rd International Conference on Electric and Electronics, Hong Kong, China.","DOI":"10.2991\/eeic-13.2013.103"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"920","DOI":"10.1109\/34.310690","article-title":"A stereo matching algorithm with an adaptive window: Theory and experiment","volume":"16","author":"Kanade","year":"1994","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"013012","DOI":"10.1117\/1.2711817","article-title":"Stereo matching via selective multiple windows","volume":"16","author":"Adhyapak","year":"2007","journal-title":"J. Electron. Imaging"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"4838","DOI":"10.1080\/2150704X.2020.1723168","article-title":"A window size selection network for stereo dense image matching","volume":"41","author":"Huang","year":"2020","journal-title":"Int. J. Remote Sens."},{"key":"ref_33","unstructured":"Hirschm\u00fcller, H. (2005, January 20\u201325). Accurate and efficient stereo processing by semi-global matching and mutual information. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The KITTI dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2129","DOI":"10.1016\/j.patrec.2005.03.022","article-title":"ZNCC-based template matching using bounded partial correlation","volume":"26","author":"Mattoccia","year":"2005","journal-title":"Pattern Recognit. Lett."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2269","DOI":"10.1016\/j.patcog.2015.01.002","article-title":"Cross-trees, edge and superpixel priors-based cost aggregation for stereo matching","volume":"48","author":"Cheng","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Taniai, T., Matsushita, Y., and Naemura, T. (2014, January 23\u201328). Graph cut based continuous stereo matching using locally shared labels. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.209"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Scharstein, D., Hirschm\u00fcller, H., Kitajima, Y., Krathwohl, G., Ne\u0161i\u0107, N., Wang, X., and Westling, P. (2014, January 2\u20135). High-resolution stereo datasets with subpixel-accurate ground truth. Proceedings of the German conference on pattern recognition, M\u00fcnster, Germany.","DOI":"10.1007\/978-3-319-11752-2_3"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"293","DOI":"10.5194\/isprsannals-I-3-293-2012","article-title":"The ISPRS benchmark on urban object classification and 3D building reconstruction","volume":"1","author":"Rottensteiner","year":"2012","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Bosch, M., Foster, K., Christie, G., Wang, S., Hager, G.D., and Brown, M. (2019, January 7\u201311). Semantic stereo for incidental satellite images. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV.2019.00167"},{"key":"ref_41","unstructured":"Saux, B.L., Yokoya, N., H\u00e4nsch, R., and Brown, M. (2020, June 07). Data Fusion Contest 2019 (DFC2019). Available online: https:\/\/ieee-dataport.org\/open-access\/data-fusion-contest-2019-dfc2019."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Olsson, C., Ul\u00e9n, J., and Boykov, Y. (2013, January 23\u201328). In defense of 3d-label stereo. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.226"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/19\/3138\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:13:17Z","timestamp":1760177597000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/19\/3138"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,24]]},"references-count":42,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["rs12193138"],"URL":"https:\/\/doi.org\/10.3390\/rs12193138","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2020,9,24]]}}}