{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T18:13:01Z","timestamp":1775067181734,"version":"3.50.1"},"reference-count":56,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2023,9,9]],"date-time":"2023-09-09T00:00:00Z","timestamp":1694217600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U20A2082"],"award-info":[{"award-number":["U20A2082"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41971151"],"award-info":[{"award-number":["41971151"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["HSDSSCX2022-117"],"award-info":[{"award-number":["HSDSSCX2022-117"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Harbin Normal University Postgraduate Innovation Project","award":["U20A2082"],"award-info":[{"award-number":["U20A2082"]}]},{"name":"Harbin Normal University Postgraduate Innovation Project","award":["41971151"],"award-info":[{"award-number":["41971151"]}]},{"name":"Harbin Normal University Postgraduate Innovation Project","award":["HSDSSCX2022-117"],"award-info":[{"award-number":["HSDSSCX2022-117"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Hyperspectral image (HSI) classification has been extensively applied for analyzing remotely sensed images. HSI data consist of multiple bands that provide abundant spatial information. Convolutional neural networks (CNNs) have emerged as powerful deep learning methods for processing visual data. In recent work, CNNs have shown impressive results in HSI classification. In this paper, we propose a hierarchical neural network architecture called feature extraction with hybrid spectral CNN (FE-HybridSN) to extract superior spectral\u2013spatial features. FE-HybridSN effectively captures more spectral\u2013spatial information while reducing computational complexity. Considering the prevalent issue of class imbalance in experimental datasets (IP, UP, SV) and real-world hyperspectral datasets, we apply the focal loss to mitigate these problems. The focal loss reconstructs the loss function and facilitates effective achievement of the aforementioned goals. We propose a framework (FEHN-FL) that combines FE-HybridSN and the focal loss for HSI classification and then conduct extensive HSI classification experiments using three remote sensing datasets: Indian Pines (IP), University of Pavia (UP), and Salinas Scene (SV). Using cross-entropy loss as a baseline, we assess the hyperspectral classification performance of various backbone networks and examine the influence of different spatial sizes on classification accuracy. After incorporating focal loss as our loss function, we not only compare the classification performance of the FE-HybridSN backbone network under different loss functions but also evaluate their convergence rates during training. The proposed classification framework demonstrates satisfactory performance compared to state-of-the-art end-to-end deep-learning-based methods, such as 2D-CNN, 3D-CNN, etc.<\/jats:p>","DOI":"10.3390\/rs15184439","type":"journal-article","created":{"date-parts":[[2023,9,11]],"date-time":"2023-09-11T09:09:21Z","timestamp":1694423361000},"page":"4439","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["A Hybrid 3D\u20132D Feature Hierarchy CNN with Focal Loss for Hyperspectral Image Classification"],"prefix":"10.3390","volume":"15","author":[{"given":"Xiaoyan","family":"Wen","sequence":"first","affiliation":[{"name":"School of Computer Science and Information Engineering, Harbin Normal University, Harbin 150025, China"}]},{"given":"Xiaodong","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information Engineering, Harbin Normal University, Harbin 150025, China"},{"name":"Key Laboratory of Intelligent Education and Information Engineering, Heilongjiang Universities, Harbin 150025, China"}]},{"given":"Yufan","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information Engineering, Harbin Normal University, Harbin 150025, China"}]},{"given":"Cuiping","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information Engineering, Harbin Normal University, Harbin 150025, China"}]},{"given":"Yu","family":"Sun","sequence":"additional","affiliation":[{"name":"Department of Municipal and Environmental Engineering, Heilongjiang Institute of Construction Technology, Harbin 150025, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MGRS.2013.2244672","article-title":"Hyperspectral remote sensing data analysis and future challenges","volume":"1","author":"Plaza","year":"2013","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/79.974727","article-title":"Spectral unmixing","volume":"19","author":"Keshava","year":"2002","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bioucas-Dias, J.M., and Plaza, A. (2011, January 24\u201329). An overview on hyperspectral unmixing: Geometrical, statistical, and sparse regression based approaches. Proceedings of the 2011 IEEE International Geoscience and Remote Sensing Symposium, Vancouver, BC, Canada.","DOI":"10.1109\/IGARSS.2011.6049397"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3818","DOI":"10.1109\/TNNLS.2019.2944869","article-title":"Nonpeaked discriminant analysis for data representation","volume":"30","author":"Ye","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1109\/TNNLS.2020.3027588","article-title":"Learning Robust Discriminant Subspace Based on Joint L2,p -and L2,s -Norm Distance Metrics","volume":"33","author":"Fu","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"108060","DOI":"10.1016\/j.sigpro.2021.108060","article-title":"TSLRLN: Tensor subspace low-rank learning with non-local prior for hyperspectral image mixed denoising","volume":"184","author":"He","year":"2021","journal-title":"Signal Process."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Huang, J., Liu, K., and Li, X. (2022). Locality constrained low rank representation and automatic dictionary learning for hyperspectral anomaly detection. Remote Sens., 14.","DOI":"10.3390\/rs14061327"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"8125","DOI":"10.1109\/TGRS.2020.2987436","article-title":"Estimating vertical chlorophyll concentrations in maize in different health states using hyperspectral LiDAR","volume":"58","author":"Bi","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1775","DOI":"10.1002\/jsfa.10214","article-title":"Identification of heat damage in imported soybeans based on hyperspectral imaging technology","volume":"100","author":"Liu","year":"2020","journal-title":"J. Sci. Food Agric."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1109\/JSTARS.2015.2508448","article-title":"Hypergraph-regularized sparse NMF for hyperspectral unmixing","volume":"9","author":"Wang","year":"2016","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1109\/MGRS.2019.2902525","article-title":"Hypersectral imaging for military and security applications: Combining myriad processing and sensing techniques","volume":"7","author":"Shimoni","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Barberio, M., Benedicenti, S., Pizzicannella, M., Felli, E., Collins, T., Jansen-Winkeln, B., Marescaux, J., Viola, M.G., and Diana, M. (2021). Intraoperative guidance using hyperspectral imaging: A review for surgeons. Diagnostics, 11.","DOI":"10.3390\/diagnostics11112066"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1109\/LGRS.2017.2776113","article-title":"Hyperspectral image classification via multiscale joint collaborative representation with locally adaptive dictionary","volume":"15","author":"Yang","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1109\/LGRS.2005.857031","article-title":"Composite kernels for hyperspectral image classification","volume":"3","year":"2006","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1109\/LGRS.2017.2774253","article-title":"Hyperspectral image classification using joint sparse model and discontinuity preserving relaxation","volume":"15","author":"Gao","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1109\/LGRS.2018.2872540","article-title":"Band selection of hyperspectral images using multiobjective optimization-based sparse self-representation","volume":"16","author":"Hu","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1109\/LGRS.2017.2787338","article-title":"Hyperspectral image classification via fusing correlation coefficient and joint sparse representation","volume":"15","author":"Tu","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1109\/LGRS.2018.2871273","article-title":"Sparse representation-based hyperspectral image classification using multiscale superpixels and guided filter","volume":"16","author":"Dundar","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1109\/LGRS.2018.2872358","article-title":"Boltzmann entropy-based unsupervised band selection for hyperspectral image classification","volume":"16","author":"Gao","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1109\/79.974718","article-title":"Hyperspectral image data analysis","volume":"19","author":"Landgrebe","year":"2002","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6232","DOI":"10.1109\/TGRS.2016.2584107","article-title":"Deep feature extraction and classification of hyperspectral images based on convolutional neural networks","volume":"54","author":"Chen","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"042607","DOI":"10.1117\/1.JRS.11.042607","article-title":"Collaborative classification of hyperspectral and visible images with convolutional neural network","volume":"11","author":"Zhang","year":"2017","journal-title":"J. Appl. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1999","DOI":"10.1049\/iet-ipr.2019.1282","article-title":"Smart feature extraction and classification of hyperspectral images based on convolutional neural networks","volume":"14","author":"Hamouda","year":"2020","journal-title":"IET Image Process."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1497","DOI":"10.1109\/LGRS.2016.2593789","article-title":"Dimensionality reduction based on group-based tensor model for hyperspectral image classification","volume":"13","author":"An","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yu, X., Ding, R., Shao, J., and Li, X. (2021). Hyperspectral Remote Sensing Image Feature Representation Method Based on CAE-H with Nuclear Norm Constraint. Electronics, 10.","DOI":"10.3390\/electronics10212667"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1109\/LGRS.2018.2878773","article-title":"Data augmentation for hyperspectral image classification with deep CNN","volume":"16","author":"Li","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mei, S., Ji, J., Bi, Q., Hou, J., Du, Q., and Li, W. (2016, January 10\u201315). Integrating spectral and spatial information into deep convolutional neural networks for hyperspectral classification. Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China.","DOI":"10.1109\/IGARSS.2016.7730321"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1016\/j.isprsjprs.2017.11.021","article-title":"A new deep convolutional neural network for fast hyperspectral image classification","volume":"145","author":"Paoletti","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1016\/j.patcog.2016.10.019","article-title":"Hyperspectral image reconstruction by deep convolutional neural network for classification","volume":"63","author":"Li","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yang, J., Zhao, Y.Q., Chan, J.C.W., and Xiao, L. (2019). A multi-scale wavelet 3D-CNN for hyperspectral image super-resolution. Remote Sens., 11.","DOI":"10.3390\/rs11131557"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1109\/LGRS.2019.2918719","article-title":"HybridSN: Exploring 3-D\u20132-D CNN feature hierarchy for hyperspectral image classification","volume":"17","author":"Roy","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, K., Zhong, P., Zheng, Y., Yang, K., and Liu, M. (2018). P_VggNet: A convolutional neural network (CNN) with pixel-based attention map. PLoS ONE, 13.","DOI":"10.1371\/journal.pone.0208497"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"847","DOI":"10.1109\/TGRS.2017.2755542","article-title":"Spectral\u2013spatial residual network for hyperspectral image classification: A 3-D deep learning framework","volume":"56","author":"Zhong","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_34","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_35","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11\u201317). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Liu, B., Liu, Y., Zhang, W., Tian, Y., and Kong, W. (2023). Spectral Swin Transformer Network for Hyperspectral Image Classification. Remote Sens., 15.","DOI":"10.3390\/rs15153721"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"4025","DOI":"10.1080\/01431161.2022.2105668","article-title":"SpectralSWIN: A spectral-swin transformer network for hyperspectral image classification","volume":"43","author":"Ayas","year":"2022","journal-title":"Int. J. Remote Sens."},{"key":"ref_39","unstructured":"Zhao, E., Guo, Z., Li, Y., and Zhang, D. (2023). SaaFormer: Spectral-spatial Axial Aggregation Transformer for Hyperspectral Image Classification. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wang, Y., and Duan, H. (2018). Classification of Hyperspectral Images by SVM Using a Composite Kernel by Employing Spectral, Spatial and Hierarchical Structure Information. Remote Sens., 10.","DOI":"10.3390\/rs10030441"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"8065","DOI":"10.1109\/TGRS.2019.2918080","article-title":"Visual attention-driven hyperspectral image classification","volume":"57","author":"Haut","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"7831","DOI":"10.1109\/TGRS.2020.3043267","article-title":"Attention-based adaptive spectral\u2013spatial kernel ResNet for hyperspectral image classification","volume":"59","author":"Roy","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhou, B., Cui, Q., Wei, X.S., and Chen, Z.M. (2020, January 13\u201319). Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00974"},{"key":"ref_44","unstructured":"Ochal, M., Patacchiola, M., Storkey, A., Vazquez, J., and Wang, S. (2021). Few-shot learning with class imbalance. arXiv."},{"key":"ref_45","unstructured":"Kubat, M., and Matwin, S. (1997, January 8\u201312). Addressing the curse of imbalanced training sets: One-sided selection. Proceedings of the Fourteenth International Conference on Machine Learning (ICML 1997), Nashville, TN, USA."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1109\/TPAMI.2012.59","article-title":"3D convolutional neural networks for human action recognition","volume":"35","author":"Ji","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2249","DOI":"10.1162\/neco_a_01316","article-title":"ReLU networks are universal approximators via piecewise linear or constant functions","volume":"32","author":"Huang","year":"2020","journal-title":"Neural Comput."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1109\/MGRS.2019.2912563","article-title":"Deep learning for classification of hyperspectral data: A comparative review","volume":"7","author":"Audebert","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1109\/MGRS.2016.2548504","article-title":"Domain adaptation for the classification of remote sensing data: An overview of recent advances","volume":"4","author":"Tuia","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Landgrebe, D.A. (2003). Signal Theory Methods in Multispectral Remote Sensing, John Wiley & Sons.","DOI":"10.1002\/0471723800"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Makantasis, K., Karantzalos, K., Doulamis, A., and Doulamis, N. (2015, January 26\u201331). Deep supervised learning for hyperspectral data classification through convolutional neural networks. Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy.","DOI":"10.1109\/IGARSS.2015.7326945"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"4420","DOI":"10.1109\/TGRS.2018.2818945","article-title":"3-D deep learning approach for remote sensing image classification","volume":"56","author":"Hamida","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"He, M., Li, B., and Chen, H. (2017, January 17\u201320). Multi-scale 3D deep convolutional neural network for hyperspectral image classification. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.","DOI":"10.1109\/ICIP.2017.8297014"},{"key":"ref_56","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/18\/4439\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:47:52Z","timestamp":1760129272000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/18\/4439"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,9]]},"references-count":56,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["rs15184439"],"URL":"https:\/\/doi.org\/10.3390\/rs15184439","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,9]]}}}