{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T18:09:52Z","timestamp":1775326192233,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2017,12,18]],"date-time":"2017-12-18T00:00:00Z","timestamp":1513555200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Using deep learning to improve the capabilities of high-resolution satellite images has emerged recently as an important topic in automatic classification. Deep networks track hierarchical high-level features to identify objects; however, enhancing the classification accuracy from low-level features is often disregarded. We therefore proposed a two-stream deep-learning neural network strategy, with a main stream utilizing fine spatial-resolution panchromatic images to retain low-level information under a supervised residual network structure. An auxiliary line employed an unsupervised net to extract high-level abstract and discriminative features from multispectral images to supplement the spectral information in the main stream. Various feature extraction types from the neural network were selected and jointed in the novel net, as the combined high- and low-level features could provide a superior solution to image classification. In traditional convolutional neural networks, increased network depth might not influence the network performance perceptibly; however, we introduced a residual neural network to develop the expressive ability of the deeper net, increasing the role of net depth in feature extraction. To enhance feature robustness, we proposed a novel consolidation part in feature extraction. An adversarial net improved the feature extraction capabilities and aided digging the inherent and discriminative features from data, with increased extraction efficacy. Tests on satellite images indicated the high overall accuracy of our novel net, verifying that net depth or number of convolution kernels affected the classification capability. Various comparative tests proved the structural rationality for our two-stream structure.<\/jats:p>","DOI":"10.3390\/rs9121328","type":"journal-article","created":{"date-parts":[[2017,12,19]],"date-time":"2017-12-19T03:54:32Z","timestamp":1513655672000},"page":"1328","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["GAN-Assisted Two-Stream Neural Network for High-Resolution Remote Sensing Image Classification"],"prefix":"10.3390","volume":"9","author":[{"given":"Yiting","family":"Tao","sequence":"first","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8434-8009","authenticated-orcid":false,"given":"Miaozhong","family":"Xu","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China"},{"name":"Collaborative Innovation Center of Geospatial Technology, Wuhan University, Wuhan 430079, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9446-5850","authenticated-orcid":false,"given":"Yanfei","family":"Zhong","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China"},{"name":"Collaborative Innovation Center of Geospatial Technology, Wuhan University, Wuhan 430079, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7688-3477","authenticated-orcid":false,"given":"Yufeng","family":"Cheng","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,12,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"7405","DOI":"10.1109\/TGRS.2016.2601622","article-title":"Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images","volume":"54","author":"Cheng","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"10347","DOI":"10.3390\/rs70810347","article-title":"Image fusion-based change detection for flood extent extraction using bi-temporal very high-resolution satellite images","volume":"7","author":"Byun","year":"2015","journal-title":"Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, Q., Huang, X., and Zhang, G. (2017). Urban Area Extraction by Regional and Line Segment Feature Fusion and Urban Morphology Analysis. Remote Sens., 9.","DOI":"10.3390\/rs9070663"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Maboudi, M., Amini, J., Hahn, M., and Saati, M. (2016). Road Network Extraction from VHR Satellite Images Using Context Aware Object Feature Integration and Tensor Voting. Remote Sens., 8.","DOI":"10.3390\/rs8080637"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Huang, Z., Cheng, G., Wang, H., Li, H., Shi, L., and Pan, C. (2016, January 10\u201315). Building extraction from multi-source remote sensing images via deep deconvolution neural networks. Proceedings of the 2016 IEEE International Conference Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China.","DOI":"10.1109\/IGARSS.2016.7729471"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4544","DOI":"10.1109\/TGRS.2016.2543748","article-title":"Spectral\u2013spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach","volume":"54","author":"Zhao","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1967","DOI":"10.1109\/TCYB.2014.2362959","article-title":"Semi-supervised multitask learning for scene recognition","volume":"45","author":"Lu","year":"2015","journal-title":"IEEE Trans. Cybern."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, T., Zhang, H., Lin, H., and Fang, C. (2016). Textural\u2013spectral feature-based species classification of mangroves in Mai Po Nature Reserve from Worldview-3 imagery. Remote Sens., 8.","DOI":"10.3390\/rs8010024"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yu, H., Yang, W., Xia, G.-S., and Liu, G. (2016). A color-texture-structure descriptor for high-resolution satellite image classification. Remote Sens., 8.","DOI":"10.3390\/rs8030259"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Li, S., Wang, S., Zheng, Z., Wan, D., and Feng, J. (2016, January 25\u201328). A new algorithm for water information extraction from high resolution remote sensing imagery. Proceedings of the 2016 IEEE International Conference Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7533183"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"L\u00e4ngkvist, M., Kiselev, A., Alirezaie, M., and Loutfi, A. (2016). Classification and segmentation of satellite orthoimagery using convolutional neural networks. Remote Sens., 8.","DOI":"10.3390\/rs8040329"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Masci, J., Meier, U., Cire\u015fan, D., and Schmidhuber, J. (2011, January 14\u201317). Stacked convolutional auto-encoders for hierarchical feature extraction. Proceedings of the 21st International Conference on Artificial Neural Networks\u2014Volume Part II, Espoo, Finland.","DOI":"10.1007\/978-3-642-21735-7_7"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1793","DOI":"10.1109\/TGRS.2015.2488681","article-title":"Scene classification via a gradient boosting random convolutional network framework","volume":"54","author":"Zhang","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liang, H., and Li, Q. (2016). Hyperspectral imagery classification using sparse representations of convolutional neural network features. Remote Sens., 8.","DOI":"10.3390\/rs8020099"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1109\/TGRS.2015.2478379","article-title":"Unsupervised deep feature extraction for remote sensing image classification","volume":"54","author":"Romero","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Fu, G., Liu, C., Zhou, R., Sun, T., and Zhang, Q. (2017). Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network. Remote Sens., 9.","DOI":"10.3390\/rs9050498"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Pohlen, T., Hermans, A., Mathias, M., and Leibe, B. (2016). Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes. arXiv.","DOI":"10.1109\/CVPR.2017.353"},{"key":"ref_18","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, Fontainebleau Resort, Miami, FL, USA."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"753","DOI":"10.1109\/LGRS.2013.2278551","article-title":"Quality assessment of panchromatic and multispectral image fusion for the ZY-3 satellite: From an information extraction perspective","volume":"11","author":"Huang","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1007\/s11220-016-0135-6","article-title":"Remote sensing image fusion with convolutional neural network","volume":"17","author":"Zhong","year":"2016","journal-title":"Sens. Imaging"},{"key":"ref_21","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_22","unstructured":"Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv."},{"key":"ref_23","unstructured":"Luc, P., Couprie, C., Chintala, S., and Verbeek, J. (2016). Semantic segmentation using adversarial networks. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"5653","DOI":"10.1109\/TGRS.2017.2711275","article-title":"Integrating Multilayer Features of Convolutional Neural Networks for Remote Sensing Scene Classification","volume":"55","author":"Li","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hu, J., Mou, L., Schmitt, A., and Zhu, X.X. (2017, January 6\u20138). FusioNet: A two-stream convolutional neural network for urban scene classification using PolSAR and hyperspectral data. Proceedings of the Urban Remote Sensing Event (JURSE), Dubai, UAE.","DOI":"10.1109\/JURSE.2017.7924565"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1560","DOI":"10.1109\/JPROC.2015.2449668","article-title":"Multimodal classification of remote sensing images: A review and future directions","volume":"103","author":"Tuia","year":"2015","journal-title":"Proc. IEEE"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hu, J., Ghamisi, P., Schmitt, A., and Zhu, X.X. (2016, January 21\u201324). Object based fusion of polarimetric sar and hyperspectral imaging for land use classification. Proceedings of the 2016 8th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), Los Angeles, CA, USA.","DOI":"10.1109\/WHISPERS.2016.8071752"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, Q., Gao, J., and Yuan, Y. Embedding structured contour and location prior in siamesed fully convolutional networks for road detection. IEEE Trans. Intell. Trans. Syst., 2017.","DOI":"10.1109\/TITS.2017.2749964"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"6805","DOI":"10.1109\/TGRS.2017.2734697","article-title":"Unsupervised-Restricted Deconvolutional Neural Network for Very High Resolution Remote-Sensing Image Classification","volume":"55","author":"Tao","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1847","DOI":"10.1109\/TPAMI.2012.272","article-title":"Deep hierarchies in the primate visual cortex: What can we learn for computer vision?","volume":"35","author":"Kruger","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Glorot","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201313). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 8\u201316). Identity mappings in deep residual networks. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Nogueira, K., Dalla Mura, M., and Chanussot, J. (2016, January 4\u20138). Learning to Semantically Segment High-Resolution Remote Sensing Images. Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico.","DOI":"10.1109\/ICPR.2016.7900187"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Cimpoi, M., Maji, S., and Vedaldi, A. (2015, January 7\u201312). Deep filter banks for texture recognition and segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299007"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2094","DOI":"10.1109\/JSTARS.2014.2329330","article-title":"Deep learning-based classification of hyperspectral data","volume":"7","author":"Chen","year":"2014","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_39","unstructured":"Arjovsky, M., Chintala, S., and Bottou, L. (2017). Wasserstein GAN. arXiv."},{"key":"ref_40","unstructured":"Qi, G.J. (2017). Loss-sensitive generative adversarial networks on Lipschitz densities. arXiv."},{"key":"ref_41","unstructured":"Odena, A. (2016). Semi-supervised learning with generative adversarial networks. arXiv."},{"key":"ref_42","unstructured":"Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv."},{"key":"ref_43","unstructured":"Yu, F., and Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Kudo, Y., and Aoki, Y. (2017, January 8\u201312). Dilated convolutions for image classification and object localization. Proceedings of the 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), Nagoya, Japan.","DOI":"10.23919\/MVA.2017.7986898"},{"key":"ref_45","unstructured":"Coates, A., Ng, A.Y., and Lee, H. (2011, January 11\u201313). An analysis of single-layer networks in unsupervised feature learning. Proceedings of the 14th International Conference on Artificial Intelligence Statistics, Ft. Lauderdale, FL, USA."},{"key":"ref_46","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1109\/TGRS.2016.2616585","article-title":"Dense semantic labeling of subdecimeter resolution images with convolutional neural networks","volume":"55","author":"Volpi","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/9\/12\/1328\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:54:28Z","timestamp":1760208868000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/9\/12\/1328"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,12,18]]},"references-count":47,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2017,12]]}},"alternative-id":["rs9121328"],"URL":"https:\/\/doi.org\/10.3390\/rs9121328","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,12,18]]}}}