{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T21:10:29Z","timestamp":1781298629534,"version":"3.54.1"},"reference-count":20,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,2,16]],"date-time":"2023-02-16T00:00:00Z","timestamp":1676505600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000181","name":"U.S. Air Force Office of Scientific Research","doi-asserted-by":"publisher","award":["FA9550-20-1-0370"],"award-info":[{"award-number":["FA9550-20-1-0370"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Multimodal fusion approaches that combine data from dissimilar sensors can better exploit human-like reasoning and strategies for situational awareness. The performance of a six-layer convolutional neural network (CNN) and an 18-layer ResNet architecture are compared for a variety of fusion methods using synthetic aperture radar (SAR) and electro-optical (EO) imagery to classify military targets. The dataset used is the Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset, using both original measured SAR data and synthetic EO data. We compare the classification performance of both networks using the data modalities individually, feature level fusion, decision level fusion, and using a novel fusion method based on the three RGB-input channels of a residual neural network (ResNet). In the proposed input channel fusion method, the SAR and the EO imagery are separately fed to each of the three input channels, while the third channel is fed a zero vector. It is found that the input channel fusion method using ResNet was able to consistently perform to a higher classification accuracy in every equivalent scenario.<\/jats:p>","DOI":"10.3390\/s23042207","type":"journal-article","created":{"date-parts":[[2023,2,16]],"date-time":"2023-02-16T01:36:52Z","timestamp":1676511412000},"page":"2207","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Assessment of Various Multimodal Fusion Approaches Using Synthetic Aperture Radar (SAR) and Electro-Optical (EO) Imagery for Vehicle Classification via Neural Networks"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3568-2702","authenticated-orcid":false,"given":"Ram M.","family":"Narayanan","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, The Pennsylvania State University, University Park, State College, PA 16802, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Noah S.","family":"Wood","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, The Pennsylvania State University, University Park, State College, PA 16802, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9281-7779","authenticated-orcid":false,"given":"Benjamin P.","family":"Lewis","sequence":"additional","affiliation":[{"name":"Multi-Sensing Knowledge Branch, AFRL\/RYAP, U.S. Air Force Research Laboratory, Wright-Patterson AFB, Dayton, OH 45433, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1117\/1.JRS.11.042609","article-title":"Comprehensive survey of deep learning in remote sensing: Theories, tools, and challenges for the community","volume":"11","author":"Ball","year":"2017","journal-title":"J. Appl. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kahler, B., and Blasch, E. (2010, January 14\u201316). Predicted radar\/optical feature fusion gains for target identification. Proceedings of the IEEE 2010 National Aerospace and Electronics Conference, Dayton, OH, USA.","DOI":"10.1109\/NAECON.2010.5712986"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Saravanan, R., and Sujatha, P. (2018, January 14\u201315). A state of art techniques on machine learning algorithms: A perspective of supervised learning approaches in data classification. Proceedings of the Second International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India.","DOI":"10.1109\/ICCONS.2018.8663155"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"696","DOI":"10.1371\/journal.pmed.0020124","article-title":"Why most published research findings are false","volume":"2","author":"Ioannidis","year":"2005","journal-title":"PLoS Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1038\/515009a","article-title":"Metascience could rescue the \u201creplication crisis\u201d","volume":"515","author":"Schooler","year":"2014","journal-title":"Nature"},{"key":"ref_7","unstructured":"Bell, S.J., and Kampman, O.P. (2021, January 3\u20137). Perspectives on machine learning from psychology\u2019s reproducibility crisis. Proceedings of the Ninth International Conference on Learning Representations (ICLR), On-Line."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.future.2021.06.014","article-title":"Do machine learning platforms provide out-of-the-box reproducibility?","volume":"126","author":"Gunderson","year":"2022","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1038\/d41586-019-03895-5","article-title":"This AI researcher is trying to ward off a reproducibility crisis","volume":"577","author":"Gibney","year":"2020","journal-title":"Nature"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1080\/00031305.1998.10480559","article-title":"Violin plots; a box plot-density trace synergism","volume":"52","author":"Hintze","year":"1998","journal-title":"Am. Stat."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"022022","DOI":"10.1088\/1742-6596\/1168\/2\/022022","article-title":"An overview of overfitting and its solutions","volume":"1168","author":"Ying","year":"2019","journal-title":"J. Phys.Conf. Ser."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lewis, B., Scarnati, T., Sudkamp, E., Nehrbass, J., Rosencrantz, S., and Zelnio, E. (2019, January 18). A SAR dataset for ATR development: The Synthetic and Measured Paired Labeled Experiment (SAMPLE). Proceedings of the SPIE Conference on Algorithms for Synthetic Aperture Radar Imagery XXVI, Baltimore, MD, USA.","DOI":"10.1117\/12.2523460"},{"key":"ref_13","unstructured":"(2022, December 31). MSTAR Overview. Available online: https:\/\/www.sdms.afrl.af.mil\/index.php?collection=mstar."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Blasch, E., Majumder, U., Zelnio, E., and Velten, V. (2020, January 27). Review of recent advances in AI\/ML using the MSTAR data. Proceedings of the SPIE Conference on Algorithms for Synthetic Aperture Radar Imagery XXVII, Virtual Event.","DOI":"10.1117\/12.2559035"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Morgan, D.A.E. (2015, January 23). Deep convolutional neural networks for ATR from SAR imagery. Proceedings of the SPIE Conference on Algorithms for Synthetic Aperture Radar Imagery XXII, Baltimore, MD, USA.","DOI":"10.1117\/12.2176558"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wang, H., Chen, S., Xu, F., and Jin, Y.-Q. (2015, January 26\u201331). Application of deep-learning algorithms to MSTAR data. Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy.","DOI":"10.1109\/IGARSS.2015.7326637"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wood, N.S., Lewis, B.P., and Narayanan, R.M. (2022, January 3\u20137). Fusing SAR and EO imagery using CNN RGB-input channels, feature level, and decision level fusion. Proceedings of the SPIE Conference on Radar Sensor Technology XXVI, Orlando, FL, USA.","DOI":"10.1117\/12.2622409"},{"key":"ref_18","first-page":"115","article-title":"A survey on machine learning for data fusion","volume":"57","author":"Tong","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ross, T.D., Morgan, D.R., Blasch, E.P., Erickson, K.J., and Kahler, B.D. (2007, January 9\u201311). Survey of approaches and experiments in decision-level fusion of automatic target recognition (ATR) products. Proceedings of the SPIE Conference on Signal Processing, Sensor Fusion, and Target Recognition XVI, Orlando, FL, USA.","DOI":"10.1117\/12.719803"},{"key":"ref_20","first-page":"8024","article-title":"PyTorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke","year":"2019","journal-title":"Adv. Neural Inf. Process. Sys."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/2207\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:37:17Z","timestamp":1760121437000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/2207"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,16]]},"references-count":20,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23042207"],"URL":"https:\/\/doi.org\/10.3390\/s23042207","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,16]]}}}