{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:35:18Z","timestamp":1760232918579,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2022,12,5]],"date-time":"2022-12-05T00:00:00Z","timestamp":1670198400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Different satellite images may consist of variable numbers of channels which have different resolutions, and each satellite has a unique revisit period. For example, the Landsat-8 satellite images have 30 m resolution in their multispectral channels, the Sentinel-2 satellite images have 10 m resolution in the pan-sharp channel, and the National Agriculture Imagery Program (NAIP) aerial images have 1 m resolution. In this study, we propose a simple yet effective arithmetic deep model for multimodal temporal remote sensing image fusion. The proposed model takes both low- and high-resolution remote sensing images at t1 together with low-resolution images at a future time t2 from the same location as inputs and fuses them to generate high-resolution images for the same location at t2. We propose an arithmetic operation applied to the low-resolution images at the two time points in feature space to take care of temporal changes. We evaluated the proposed model on three modality pairs for multimodal temporal image fusion, including downsampled WorldView-2\/original WorldView-2, Landsat-8\/Sentinel-2, and Sentinel-2\/NAIP. Experimental results show that our model outperforms traditional algorithms and recent deep learning-based models by large margins in most scenarios, achieving sharp fused images while appropriately addressing temporal changes.<\/jats:p>","DOI":"10.3390\/rs14236160","type":"journal-article","created":{"date-parts":[[2022,12,5]],"date-time":"2022-12-05T05:31:32Z","timestamp":1670218292000},"page":"6160","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["ArithFusion: An Arithmetic Deep Model for Temporal Remote Sensing Image Fusion"],"prefix":"10.3390","volume":"14","author":[{"given":"Md Reshad Ul","family":"Hoque","sequence":"first","affiliation":[{"name":"Department of Electrical & Computer Engineering, Old Dominion University, Norfolk, VA 23529, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jian","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Old Dominion University, Norfolk, VA 23529, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4341-0769","authenticated-orcid":false,"given":"Chiman","family":"Kwan","sequence":"additional","affiliation":[{"name":"Applied Research LLC, Rockville, MD 20850, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Krzysztof","family":"Koperski","sequence":"additional","affiliation":[{"name":"Maxar, Westminster, CO 80234, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0091-6986","authenticated-orcid":false,"given":"Jiang","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Old Dominion University, Norfolk, VA 23529, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1711","DOI":"10.1016\/j.agrformet.2011.07.003","article-title":"Influences of temperature and precipitation before the growing season on spring phenology in grasslands of the central and eastern Qinghai-Tibetan Plateau","volume":"151","author":"Shen","year":"2011","journal-title":"Agric. For. Meteorol."},{"key":"ref_2","first-page":"132","article-title":"Multitemporal fusion of landsat\/tm and envisat\/meris for crop monitoring","volume":"23","author":"Alonso","year":"2013","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1707","DOI":"10.1016\/j.scitotenv.2018.09.308","article-title":"Using spatio-temporal fusion of landsat-8 and modis data to derive phenology, biomass and yield estimates for corn and soybean","volume":"650","author":"Liao","year":"2019","journal-title":"Sci. Total Environ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.agrformet.2015.11.003","article-title":"Crop yield forecasting on the Canadian prairies by remotely sensed vegetation indices and machine learning methods","volume":"218","author":"Johnson","year":"2016","journal-title":"Agric. For. Meteorol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1775","DOI":"10.1080\/01431160110075802","article-title":"Using a time series of satellite imagery to detect land use and land cover changes in the Atlanta, Georgia metropolitan area","volume":"23","author":"Yang","year":"2002","journal-title":"Int. J. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2818","DOI":"10.1111\/gcb.13562","article-title":"Response of vegetation phenology to urbanization in the conterminous united states","volume":"23","author":"Li","year":"2017","journal-title":"Glob. Chang. Biol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1988","DOI":"10.1016\/j.rse.2009.05.011","article-title":"Generation of dense time series synthetic Landsat data through data blending with modis using a spatial and temporal adaptive reflectance fusion model","volume":"113","author":"Hilker","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"271","DOI":"10.5589\/m02-096","article-title":"Disturbance recognition in the boreal forest using radar and landsat-7","volume":"29","author":"Ranson","year":"2003","journal-title":"Can. J. Remote Sens."},{"key":"ref_9","unstructured":"(2021, February 05). Buying Satellite Imagery: Pricing Information for High Resolution Satellite Imagery. Available online: http:\/\/landinfo.com\/satellite-imagery-pricing\/."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1016\/j.rse.2016.06.019","article-title":"Consistent land surface temperature data generation from irregularly spaced Landsat imagery","volume":"184","author":"Fu","year":"2016","journal-title":"Remote Sens. Environ."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhu, X., Cai, F., Tian, J., and Williams, T. (2018). Spatiotemporal fusion of multisource remote sensing data: Literature survey, taxonomy, principles, applications, and future directions. Remote Sens., 10.","DOI":"10.3390\/rs10040527"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"111425","DOI":"10.1016\/j.rse.2019.111425","article-title":"Deep learning-based fusion of landsat-8 and sentinel-2 images for a harmonized surface reflectance product","volume":"235","author":"Shao","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2207","DOI":"10.1109\/TGRS.2006.872081","article-title":"On the blending of the Landsat and modis surface reflectance: Predicting daily Landsat surface reflectance","volume":"44","author":"Gao","year":"2006","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2610","DOI":"10.1016\/j.rse.2010.05.032","article-title":"An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions","volume":"114","author":"Zhu","year":"2010","journal-title":"Remote Sens. Environ."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4367","DOI":"10.1080\/01431161.2013.777488","article-title":"A spatial and temporal reflectance fusion model considering sensor observation differences","volume":"34","author":"Shen","year":"2013","journal-title":"Int. J. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1109\/LGRS.2008.919685","article-title":"Unmixing-based Landsat tm and meris fr data fusion","volume":"5","author":"Clevers","year":"2008","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1874","DOI":"10.1016\/j.rse.2009.04.011","article-title":"Downscaling time series of meris full resolution data to monitor vegetation seasonal dynamics","volume":"113","author":"Kaiser","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3707","DOI":"10.1109\/TGRS.2012.2186638","article-title":"Spatiotemporal reflectance fusion via sparse representation","volume":"50","author":"Huang","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1883","DOI":"10.1109\/TGRS.2012.2213095","article-title":"Spatiotemporal satellite image fusion through one-pair image learning","volume":"51","author":"Song","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.rse.2015.06.003","article-title":"Downscaling modis images with area-to-point regression kriging","volume":"166","author":"Wang","year":"2015","journal-title":"Remote Sens. Environ."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3885","DOI":"10.1109\/TGRS.2017.2683444","article-title":"Fusion of Landsat 8 oli and sentinel2 msi data","volume":"55","author":"Wang","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_23","unstructured":"Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yang, L., Zhang, M., Li, C., Bendersky, M., and Najork, M. (2020, January 19\u201323). Beyond 512 tokens: Siamese multi-depth transformer-based hierarchical encoder for long-form document matching. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Online.","DOI":"10.1145\/3340531.3411908"},{"key":"ref_25","unstructured":"Hannun, A., Case, C., Casper, J., Catanzaro, B., Diamos, G., Elsen, E., Prenger, R., Satheesh, S., Sengupta, S., and Coates, A. (2014). Deep speech: Scaling up end-to-end speech recognition. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Graves, A., Mohamed, A.-r., and Hinton, G. (2013, January 26\u201331). Speech recognition with deep recurrent neural networks. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Petersson, H., Gustafsson, D., and Bergstrom, D. (2016, January 12\u201315). Hyperspectral image analysis using deep learning\u2014A review. Proceedings of the 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), Oulu, Finland.","DOI":"10.1109\/IPTA.2016.7820963"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MGRS.2017.2762307","article-title":"Deep learning in remote sensing: A comprehensive review and list of resources","volume":"5","author":"Zhu","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image super-resolution using deep convolutional networks","volume":"38","author":"Dong","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"821","DOI":"10.1109\/JSTARS.2018.2797894","article-title":"Spatiotemporal satellite image fusion using deep convolutional neural networks","volume":"11","author":"Song","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11427-019-9817-6","article-title":"A new sensor bias-driven spatio-temporal fusion model based on convolutional neural networks","volume":"63","author":"Li","year":"2020","journal-title":"Sci. China Inf. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chen, B., Li, J., and Jin, Y. (2021). Deep learning for feature-level data fusion: Higher resolution reconstruction of historical Landsat archive. Remote Sens., 13.","DOI":"10.3390\/rs13020167"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3349","DOI":"10.1109\/TPAMI.2020.2983686","article-title":"Deep high-resolution representation learning for visual recognition","volume":"43","author":"Wang","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.mri.2019.07.010","article-title":"Region-of-interest undersampled mri reconstruction: A deep convolutional neural network approach","volume":"63","author":"Sun","year":"2019","journal-title":"Magn. Reson. Imaging"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"7571934","DOI":"10.1155\/2016\/7571934","article-title":"Image reconstruction using analysis model prior","volume":"2016","author":"Han","year":"2016","journal-title":"Comput. Math. Methods Med."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2565","DOI":"10.1109\/TGRS.2014.2361734","article-title":"A critical comparison among pansharpening algorithms","volume":"53","author":"Vivone","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2288-5-35","article-title":"Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data","volume":"5","author":"Vickers","year":"2005","journal-title":"BMC Med. Res. Methodol."},{"key":"ref_39","first-page":"184","article-title":"The differences and similarities between two-sample t-test and paired t-test","volume":"29","author":"Xu","year":"2017","journal-title":"Shanghai Arch. Psychiatry"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"4273","DOI":"10.1109\/TGRS.2020.3010530","article-title":"Remote sensing image spatiotemporal fusion using a generative adversarial network","volume":"59","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_41","unstructured":"(2020, February 07). Keras. Available online: https:\/\/keras.io\/."},{"key":"ref_42","unstructured":"Chitwan, S., Ho, J., Chan, W., Salimans, T., Fleet, D.J., and Norouzi, M. (2021). Image super-resolution via iterative refinement. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ayhan, B., and Kwan, C. (2020). Tree, Shrub, and Grass Classification Using Only RGB Images. Remote Sens, 12.","DOI":"10.3390\/rs12081333"},{"key":"ref_44","unstructured":"Ayhan, B., Kwan, C., Larkin, J., Kwan, L., Skarlatos, D., and Vlachos, M. (May, January 27). Deep learning model for accurate vegetation classification using RGB image only. Proceedings of the SPIE 11398, Geospatial Informatics X, Online."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/23\/6160\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:34:09Z","timestamp":1760146449000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/23\/6160"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,5]]},"references-count":44,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["rs14236160"],"URL":"https:\/\/doi.org\/10.3390\/rs14236160","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2022,12,5]]}}}