{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T20:29:22Z","timestamp":1783110562284,"version":"3.54.6"},"reference-count":62,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2022,3,20]],"date-time":"2022-03-20T00:00:00Z","timestamp":1647734400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62171247"],"award-info":[{"award-number":["62171247"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41921781"],"award-info":[{"award-number":["41921781"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Remote sensing (RS) image classification has attracted much attention recently and is widely used in various fields. Different to natural images, the RS image scenes consist of complex backgrounds and various stochastically arranged objects, thus making it difficult for networks to focus on the target objects in the scene. However, conventional classification methods do not have any special treatment for remote sensing images. In this paper, we propose a two-stream swin transformer network (TSTNet) to address these issues. TSTNet consists of two streams (i.e., original stream and edge stream) which use both the deep features of the original images and the ones from the edges to make predictions. The swin transformer is used as the backbone of each stream given its good performance. In addition, a differentiable edge Sobel operator module (DESOM) is included in the edge stream which can learn the parameters of Sobel operator adaptively and provide more robust edge information that can suppress background noise. Experimental results on three publicly available remote sensing datasets show that our TSTNet achieves superior performance over the state-of-the-art (SOTA) methods.<\/jats:p>","DOI":"10.3390\/rs14061507","type":"journal-article","created":{"date-parts":[[2022,3,20]],"date-time":"2022-03-20T21:37:17Z","timestamp":1647812237000},"page":"1507","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":44,"title":["Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image Classification"],"prefix":"10.3390","volume":"14","author":[{"given":"Siyuan","family":"Hao","sequence":"first","affiliation":[{"name":"The College of Communication and Electronic Engineering, Qingdao University of Technology, Qingdao 266520, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bin","family":"Wu","sequence":"additional","affiliation":[{"name":"The College of Communication and Electronic Engineering, Qingdao University of Technology, Qingdao 266520, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9949-4693","authenticated-orcid":false,"given":"Kun","family":"Zhao","sequence":"additional","affiliation":[{"name":"The College of Communication and Electronic Engineering, Qingdao University of Technology, Qingdao 266520, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6843-6722","authenticated-orcid":false,"given":"Yuanxin","family":"Ye","sequence":"additional","affiliation":[{"name":"The Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Wang","sequence":"additional","affiliation":[{"name":"The Department of Information Engineering and Computer Science, University of Trento, Via Sommarive, 38123 Trento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,20]]},"reference":[{"key":"ref_1","first-page":"5609311","article-title":"Generalized Scene Classification From Small-Scale Datasets with Multitask Learning","volume":"60","author":"Zheng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","first-page":"5606616","article-title":"Unsupervised change detection by cross-resolution difference learning","volume":"60","author":"Zheng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_3","first-page":"5606514","article-title":"Mutual attention inception network for remote sensing visual question answering","volume":"60","author":"Zheng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"9059","DOI":"10.1109\/TGRS.2019.2924684","article-title":"Fast and robust matching for multimodal remote sensing image registration","volume":"57","author":"Ye","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_5","first-page":"4017605","article-title":"Robust Matching for SAR and Optical Images Using Multiscale Convolutional Gradient Features","volume":"19","author":"Zhou","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6026","DOI":"10.3390\/rs5116026","article-title":"Exploring the use of Google Earth imagery and object-based methods in land use\/cover mapping","volume":"5","author":"Hu","year":"2013","journal-title":"Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1560","DOI":"10.1109\/JPROC.2015.2449668","article-title":"Multimodal classification of remote sensing images: A review and future directions","volume":"103","author":"Tuia","year":"2015","journal-title":"Proc. IEEE"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1109\/TGRS.2011.2165548","article-title":"Very high resolution multiangle urban classification analysis","volume":"50","author":"Longbotham","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.landurbplan.2010.10.007","article-title":"An urban growth boundary model using neural networks, GIS and radial parameterization: An application to Tehran, Iran","volume":"100","author":"Tayyebi","year":"2011","journal-title":"Landsc. Urban Plan."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"6020","DOI":"10.1109\/TGRS.2016.2579648","article-title":"A three-layered graph-based learning approach for remote sensing image retrieval","volume":"54","author":"Wang","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1109\/TGRS.2012.2205158","article-title":"Geographic image retrieval using local invariant features","volume":"51","author":"Yang","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.rse.2017.05.001","article-title":"Multi-level monitoring of subtle urban changes for the megacities of China using high-resolution multi-view satellite imagery","volume":"196","author":"Huang","year":"2017","journal-title":"Remote Sens. Environ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2692","DOI":"10.1109\/JSTARS.2018.2804440","article-title":"Monitoring of urban impervious surfaces using time series of high-resolution remote sensing images in rapidly urbanized areas: A case study of Shenzhen","volume":"11","author":"Zhang","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"8775","DOI":"10.1109\/TGRS.2019.2922908","article-title":"A multi-level semantic scene interpretation strategy for change interpretation in remote sensing imagery","volume":"57","author":"Ghazouani","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"771","DOI":"10.1080\/01431161.2012.714508","article-title":"Object-based urban vegetation mapping with high-resolution aerial photography as a single data source","volume":"34","author":"Li","year":"2013","journal-title":"Int. J. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1080\/01431161.2013.876120","article-title":"Mapping vegetation morphology types in a dry savanna ecosystem: Integrating hierarchical object-based image analysis with Random Forest","volume":"35","author":"Mishra","year":"2014","journal-title":"Int. J. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"7405","DOI":"10.1109\/TGRS.2016.2601622","article-title":"Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images","volume":"54","author":"Cheng","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1016\/j.isprsjprs.2018.09.014","article-title":"Deep networks under scene-level supervision for multi-class geospatial object detection from remote sensing images","volume":"146","author":"Li","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1109\/TIP.2018.2867198","article-title":"Learning rotation-invariant and fisher discriminative convolutional neural networks for object detection","volume":"28","author":"Cheng","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1016\/j.isprsjprs.2019.11.023","article-title":"Object detection in optical remote sensing images: A survey and a new benchmark","volume":"159","author":"Li","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2337","DOI":"10.1109\/TGRS.2017.2778300","article-title":"Rotation-insensitive and context-augmented object detection in remote sensing images","volume":"56","author":"Li","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Cheng, G., Zhou, P., and Han, J. (2016, January 27\u201330). Rifd-cnn: Rotation-invariant and fisher discriminative convolutional neural networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.315"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Cheng, G., Han, J., Guo, L., and Liu, T. (2015, January 7\u201312). Learning coarse-to-fine sparselets for efficient object detection and scene classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298721"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cheng, G., Ma, C., Zhou, P., Yao, X., and Han, J. (2016, January 10\u201315). Scene classification of high resolution remote sensing images using convolutional neural networks. Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China.","DOI":"10.1109\/IGARSS.2016.7729193"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhou, W., Shao, Z., and Cheng, Q. (2016, January 4\u20136). Deep feature representations for high-resolution remote sensing scene classification. Proceedings of the 2016 4th International Workshop on Earth Observation and Remote Sensing Applications (EORSA), Guangzhou, China.","DOI":"10.1109\/EORSA.2016.7552825"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1109\/TGRS.2018.2864987","article-title":"Scene classification with recurrent attention of VHR remote sensing images","volume":"57","author":"Wang","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_27","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Bazi, Y., Bashmal, L., Rahhal, M.M.A., Dayil, R.A., and Ajlan, N.A. (2021). Vision transformers for remote sensing image classification. Remote Sens., 13.","DOI":"10.3390\/rs13030516"},{"key":"ref_29","first-page":"8020305","article-title":"When CNNs Meet Vision Transformer: A Joint Framework for Remote Sensing Scene Classification","volume":"19","author":"Deng","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhang, J., Zhao, H., and Li, J. (2021). TRS: Transformers for Remote Sensing Scene Classification. Remote Sens., 13.","DOI":"10.3390\/rs13204143"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. arXiv.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1007\/BF00130487","article-title":"Color indexing","volume":"7","author":"Swain","year":"1991","journal-title":"Int. J. Comput. Vis."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1109\/TSMC.1973.4309314","article-title":"Textural features for image classification","volume":"6","author":"Haralick","year":"1973","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1023\/A:1011139631724","article-title":"Modeling the shape of the scene: A holistic representation of the spatial envelope","volume":"42","author":"Oliva","year":"2001","journal-title":"Int. J. Comput. Vis."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_36","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_37","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_38","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Fan, R., Wang, L., Feng, R., and Zhu, Y. (August, January 28). Attention based residual network for high-resolution remote sensing imagery scene classification. Proceedings of the IGARSS 2019\u20142019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan.","DOI":"10.1109\/IGARSS.2019.8900199"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhang, W., Tang, P., and Zhao, L. (2019). Remote sensing image scene classification using CNN-CapsNet. Remote Sens., 11.","DOI":"10.3390\/rs11050494"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/TGRS.2019.2931801","article-title":"Remote sensing scene classification by gated bidirectional network","volume":"58","author":"Sun","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_44","first-page":"5501415","article-title":"A Lightweight and Robust Lie Group-Convolutional Neural Networks Joint Representation for Remote Sensing Scene Classification","volume":"60","author":"Xu","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_45","unstructured":"Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and J\u00e9gou, H. (2021, January 13\u201315). Training data-efficient image transformers & distillation through attention. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Li, W., Cao, D., Peng, Y., and Yang, C. (2021). MSNet: A Multi-Stream Fusion Network for Remote Sensing Spatiotemporal Fusion Based on Transformer and Convolution. Remote Sens., 13.","DOI":"10.3390\/rs13183724"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Xu, Z., Zhang, W., Zhang, T., Yang, Z., and Li, J. (2021). Efficient Transformer for Remote Sensing Image Segmentation. Remote Sens., 13.","DOI":"10.3390\/rs13183585"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"137080","DOI":"10.1109\/ACCESS.2019.2932229","article-title":"Relationship prior and adaptive knowledge mimic based compressed deep network for aerial scene classification","volume":"7","author":"He","year":"2019","journal-title":"IEEE Access"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"He, C., Li, S., Xiong, D., Fang, P., and Liao, M. (2020). Remote sensing image semantic segmentation based on edge information guidance. Remote Sens., 12.","DOI":"10.3390\/rs12091501"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"3965","DOI":"10.1109\/TGRS.2017.2685945","article-title":"AID: A benchmark data set for performance evaluation of aerial scene classification","volume":"55","author":"Xia","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1109\/LGRS.2017.2731997","article-title":"Remote sensing image scene classification using bag of convolutional features","volume":"14","author":"Cheng","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Yang, Y., and Newsam, S. (2010, January 2\u20135). Bag-of-visual-words and spatial extensions for land-use classification. Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, San Jose, CA, USA.","DOI":"10.1145\/1869790.1869829"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Bazi, Y., Al Rahhal, M.M., Alhichri, H., and Alajlan, N. (2019). Simple yet effective fine-tuning of deep CNNs using an auxiliary classification loss for remote sensing scene classification. Remote Sens., 11.","DOI":"10.3390\/rs11242908"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"2030","DOI":"10.1109\/JSTARS.2021.3051569","article-title":"Attention consistent network for remote sensing scene classification","volume":"14","author":"Tang","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"8077","DOI":"10.1109\/TGRS.2020.2987060","article-title":"High-resolution remote sensing image scene classification via key filter bank based on convolutional neural network","volume":"58","author":"Li","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"7918","DOI":"10.1109\/TGRS.2020.3044655","article-title":"Enhanced feature pyramid network with deep semantic embedding for remote sensing scene classification","volume":"59","author":"Wang","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"2636","DOI":"10.1109\/TNNLS.2020.3007412","article-title":"C-CNN: Contourlet convolutional neural networks","volume":"32","author":"Liu","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Luo, Z., Li, J., Chen, C., and Piao, Y. (2020). When self-supervised learning meets scene classification: Remote sensing scene classification based on a multitask learning framework. Remote Sens., 12.","DOI":"10.3390\/rs12203276"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.neucom.2021.01.038","article-title":"Multi-scale stacking attention pooling for remote sensing scene classification","volume":"436","author":"Bi","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"1603","DOI":"10.1109\/LGRS.2019.2949930","article-title":"APDC-Net: Attention pooling-based convolutional network for aerial scene classification","volume":"17","author":"Bi","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Liu, Y., Zhong, Y., Fei, F., Zhu, Q., and Qin, Q. (2018). Scene classification based on a deep random-scale stretched convolutional neural network. Remote Sens., 10.","DOI":"10.3390\/rs10030444"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A computational approach to edge detection","volume":"6","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/6\/1507\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:40:01Z","timestamp":1760136001000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/6\/1507"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,20]]},"references-count":62,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["rs14061507"],"URL":"https:\/\/doi.org\/10.3390\/rs14061507","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,20]]}}}