{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T16:06:32Z","timestamp":1775837192801,"version":"3.50.1"},"reference-count":28,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,6,17]],"date-time":"2021-06-17T00:00:00Z","timestamp":1623888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In previous works, we have shown the efficacy of using Deep Belief Networks, paired with clustering, to identify distinct classes of objects within remotely sensed data via cluster analysis and qualitative analysis of the output data in comparison with reference data. In this paper, we quantitatively validate the methodology against datasets currently being generated and used within the remote sensing community, as well as show the capabilities and benefits of the data fusion methodologies used. The experiments run take the output of our unsupervised fusion and segmentation methodology and map them to various labeled datasets at different levels of global coverage and granularity in order to test our models\u2019 capabilities to represent structure at finer and broader scales, using many different kinds of instrumentation, that can be fused when applicable. In all cases tested, our models show a strong ability to segment the objects within input scenes, use multiple datasets fused together where appropriate to improve results, and, at times, outperform the pre-existing datasets. The success here will allow this methodology to be used within use concrete cases and become the basis for future dynamic object tracking across datasets from various remote sensing instruments.<\/jats:p>","DOI":"10.3390\/rs13122364","type":"journal-article","created":{"date-parts":[[2021,6,17]],"date-time":"2021-06-17T11:20:26Z","timestamp":1623928826000},"page":"2364","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["A Quantitative Validation of Multi-Modal Image Fusion and Segmentation for Object Detection and Tracking"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6584-7315","authenticated-orcid":false,"given":"Nicholas","family":"LaHaye","sequence":"first","affiliation":[{"name":"Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91101, USA"},{"name":"Schmid College of Science and Technology, Chapman University, Orange, CA 92866, USA"},{"name":"Machine Learning and Assistive Technology Lab (MLAT), Chapman University, Orange, CA 92866, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1374-5074","authenticated-orcid":false,"given":"Michael J.","family":"Garay","sequence":"additional","affiliation":[{"name":"Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91101, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7856-3570","authenticated-orcid":false,"given":"Brian D.","family":"Bue","sequence":"additional","affiliation":[{"name":"Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91101, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9876-3705","authenticated-orcid":false,"given":"Hesham","family":"El-Askary","sequence":"additional","affiliation":[{"name":"Schmid College of Science and Technology, Chapman University, Orange, CA 92866, USA"},{"name":"Center of Excellence in Earth Systems Modeling and Observations, Chapman University, Orange, CA 92866, USA"},{"name":"Department of Environmental Sciences, Faculty of Science, Alexandria University, Moharem Bek, Alexandria 21522, Egypt"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0174-7002","authenticated-orcid":false,"given":"Erik","family":"Linstead","sequence":"additional","affiliation":[{"name":"Machine Learning and Assistive Technology Lab (MLAT), Chapman University, Orange, CA 92866, USA"},{"name":"Fowler School of Engineering, Chapman University, Orange, CA 92866, USA"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3056","DOI":"10.1109\/JSTARS.2019.2920234","article-title":"Multi-Modal Object Tracking and Image Fusion With Unsupervised Deep Learning","volume":"12","author":"LaHaye","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kanezaki, A. (2018, January 15\u201320). Unsupervised Image Segmentation by Backpropagation. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Calgary, AB, Canada.","DOI":"10.1109\/ICASSP.2018.8462533"},{"key":"ref_3","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). Unsupervised Object Segmentation by Redrawing. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2045","DOI":"10.1038\/s41598-018-31333-5","article-title":"Unsupervised Medical Image Segmentation Based on the Local Center of Mass","volume":"8","author":"Aganj","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Soares, A.R., K\u00f6rting, T.S., Fonseca, L.M.G., and Neves, A.K. (2020, January 22\u201326). An Unsupervised Segmentation Method For Remote Sensing Imagery Based On Conditional Random Fields. Proceedings of the 2020 IEEE Latin American GRSS ISPRS Remote Sensing Conference (LAGIRS), Santiago, Chile.","DOI":"10.1109\/LAGIRS48042.2020.9165623"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1117\/1.JRS.13.048505","article-title":"Unsupervised remote sensing image segmentation based on a dual autoencoder","volume":"13","author":"Zhang","year":"2019","journal-title":"J. Appl. Remote Sens."},{"key":"ref_7","unstructured":"Hinton, G.E., and Salakhutdinov, R. (2009, January 16\u201319). Deep Boltzmann Machines. Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS), Clearwater, FL, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1771","DOI":"10.1162\/089976602760128018","article-title":"Training Products of Experts by Minimizing Contrastive Divergence","volume":"14","author":"Hinton","year":"2002","journal-title":"Neural Comput."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhang, T., Ramakrishnan, R., and Livny, M. (1996, January 4\u20136). BIRCH: An Efficient Data Clustering Method for Very Large Databases. Proceedings of the SIGMOD \u201996: 1996 ACM SIGMOD International Conference on Management of Data, Montreal, QC, Canada.","DOI":"10.1145\/233269.233324"},{"key":"ref_10","unstructured":"(2021, June 12). Lrn2 Cre8 Learning to Create. Available online: https:\/\/web.archive.org\/web\/20160722213851\/http:\/\/lrn2cre8.eu:80\/."},{"key":"ref_11","unstructured":"Roder, M., de Rosa, G.H., and Papa, J.P. (2020). Learnergy: Energy-based Machine Learners. arXiv."},{"key":"ref_12","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_13","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Brodersen, K.H., Ong, C.S., Stephan, K.E., and Buhmann, J.M. (2010, January 23\u201326). The Balanced Accuracy and Its Posterior Distribution. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.764"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1109\/TPAMI.1979.4766909","article-title":"A Cluster Separation Measure","volume":"PAMI-1","author":"Davies","year":"1979","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Prechelt, L. (1998). Early Stopping - But When?. Neural Networks: Tricks of the Trade, Springer.","DOI":"10.1007\/3-540-49430-8_3"},{"key":"ref_17","unstructured":"Diner, D. (2021, June 12). MISR Experiment Overview, Available online: https:\/\/trs.jpl.nasa.gov\/handle\/2014\/18644."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1109\/36.124212","article-title":"Remote Sensing of Cloud, Aerosol, and Water Vapor Properties from the Moderate Resolution Imaging Spectrometer (MODIS)","volume":"30","author":"King","year":"1992","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.rse.2006.06.021","article-title":"An operational MISR pixel classifier using support vector machines","volume":"107","author":"Mazzoni","year":"2007","journal-title":"Remote Sens. Environ."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"32141","DOI":"10.1029\/1998JD200032","article-title":"Discriminating clear sky from clouds with MODIS","volume":"103","author":"Ackerman","year":"1998","journal-title":"J. Geophys. Res. Atmos."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6225","DOI":"10.1016\/j.atmosenv.2011.07.068","article-title":"Satellite-based estimates of ground-level fine particulate matter during extreme events: A case study of the Moscow fires in 2010","volume":"45","author":"Martin","year":"2011","journal-title":"Atmos. Environ."},{"key":"ref_22","unstructured":"Tuia, D., Gabriele, M., Le Saux, B., and Bechtel, B. (2021, June 11). Data Fusion Contest 2017 (DFC2017). Available online: https:\/\/doi.org\/10.21227\/e56j-eh82."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.rse.2014.02.001","article-title":"Landsat-8: Science and product vision for terrestrial global change research","volume":"145","author":"Roy","year":"2014","journal-title":"Remote Sens. Environ."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.rse.2011.11.026","article-title":"Sentinel-2: ESA\u2019s Optical High-Resolution Mission for GMES Operational Services","volume":"120","author":"Drusch","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1907","DOI":"10.1175\/BAMS-D-16-0236.1","article-title":"WUDAPT: An Urban Weather, Climate, and Environmental Modeling Infrastructure for the Anthropocene","volume":"99","author":"Ching","year":"2018","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/S0034-4257(00)00195-4","article-title":"The MODIS\/ASTER airborne simulator (MASTER)\u2014 A new instrument for earth science studies","volume":"76","author":"Hook","year":"2001","journal-title":"Remote Sens. Environ."},{"key":"ref_27","first-page":"214","article-title":"The enhanced MODIS airborne simulator hyperspectral imager","volume":"Volume 8048","author":"Shen","year":"2011","journal-title":"Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XVII"},{"key":"ref_28","unstructured":"Giglio, L., and Justice, C. (2021, June 11). MOD14A2 MODIS\/Terra Thermal Anomalies\/Fire 8-Day L3 Global 1km SIN Grid V006. Available online: https:\/\/doi.org\/10.5067\/MODIS\/MOD14A2.006."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/12\/2364\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:17:33Z","timestamp":1760163453000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/12\/2364"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,17]]},"references-count":28,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["rs13122364"],"URL":"https:\/\/doi.org\/10.3390\/rs13122364","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,17]]}}}