{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T06:40:48Z","timestamp":1784011248533,"version":"3.55.0"},"reference-count":45,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2019,4,10]],"date-time":"2019-04-10T00:00:00Z","timestamp":1554854400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Deep learning models combining spectral and spatial features have been proven to be effective for hyperspectral image (HSI) classification. However, most spatial feature integration methods only consider a single input spatial scale regardless of various shapes and sizes of objects over the image plane, leading to missing scale-dependent information. In this paper, we propose a hierarchical multi-scale convolutional neural networks (CNNs) with auxiliary classifiers (HMCNN-AC) to learn hierarchical multi-scale spectral\u2013spatial features for HSI classification. First, to better exploit the spatial information, multi-scale image patches for each pixel are generated at different spatial scales. These multi-scale patches are all centered at the same central spectrum but with shrunken spatial scales. Then, we apply multi-scale CNNs to extract spectral\u2013spatial features from each scale patch. The obtained multi-scale convolutional features are considered as structured sequential data with spectral\u2013spatial dependency, and a bidirectional LSTM is proposed to capture the correlation and extract a hierarchical representation for each pixel. To better train the whole network, weighted auxiliary classifiers are employed for the multi-scale CNNs and optimized together with the main loss function. Experimental results on three public HSI datasets demonstrate the superiority of our proposed framework over some state-of-the-art methods.<\/jats:p>","DOI":"10.3390\/s19071714","type":"journal-article","created":{"date-parts":[[2019,4,10]],"date-time":"2019-04-10T11:25:08Z","timestamp":1554895508000},"page":"1714","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Hierarchical Multi-Scale Convolutional Neural Networks for Hyperspectral Image Classification"],"prefix":"10.3390","volume":"19","author":[{"given":"Simin","family":"Li","sequence":"first","affiliation":[{"name":"Department of Electronic Engineering, Tsinghua University, Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9596-6227","authenticated-orcid":false,"given":"Xueyu","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of Mathematics, University of Iowa, Iowa City, IA 52242, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Bao","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Tsinghua University, Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,4,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Jensen, J.R., and Lulla, K. (1987). Introductory digital image processing: A remote sensing perspective. Geocarto Int., 2.","DOI":"10.1080\/10106048709354084"},{"key":"ref_2","first-page":"79","article-title":"Hyperspectral image processing for automatic target detection applications","volume":"14","author":"Manolakis","year":"2003","journal-title":"Linc. Lab. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"S110","DOI":"10.1016\/j.rse.2007.07.028","article-title":"Recent advances in techniques for hyperspectral image processing","volume":"113","author":"Plaza","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"895","DOI":"10.13031\/2013.16087","article-title":"Hyperspectral image data mining for band selection in agricultural applications","volume":"47","author":"Bajwa","year":"2004","journal-title":"Trans. ASAE"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"010901","DOI":"10.1117\/1.JBO.19.1.010901","article-title":"Medical hyperspectral imaging: A review","volume":"19","author":"Lu","year":"2014","journal-title":"J. Biomed. Opt."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"795","DOI":"10.1016\/j.crte.2011.09.003","article-title":"Improved remote sensing detection of soil salinity from a semi-arid climate in Northeast Brazil","volume":"343","author":"Bouaziz","year":"2011","journal-title":"C. R. Geosci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1016\/S0034-4257(97)00049-7","article-title":"Decision tree classification of land cover from remotely sensed data","volume":"61","author":"Friedl","year":"1997","journal-title":"Remote Sens. Environ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1778","DOI":"10.1109\/TGRS.2004.831865","article-title":"Classification of hyperspectral remote sensing images with support vector machines","volume":"42","author":"Melgani","year":"2004","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","first-page":"18","article-title":"Classification and regression by randomForest","volume":"2","author":"Liaw","year":"2002","journal-title":"R News"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Akaike, H. (1998). Information theory and an extension of the maximum likelihood principle. Selected Papers of Hirotugu Akaike, Springer.","DOI":"10.1007\/978-1-4612-1694-0_15"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2297","DOI":"10.1109\/TGRS.2009.2039484","article-title":"Feature selection for classification of hyperspectral data by SVM","volume":"48","author":"Pal","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"736","DOI":"10.1109\/LGRS.2010.2047711","article-title":"SVM and MRF-based method for accurate classification of hyperspectral images","volume":"7","author":"Tarabalka","year":"2010","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3457","DOI":"10.1109\/TGRS.2010.2046494","article-title":"Hyperspectral region classification using a three-dimensional Gabor filterbank","volume":"48","author":"Bau","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"780","DOI":"10.1109\/36.752194","article-title":"Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices","volume":"37","author":"Soh","year":"1999","journal-title":"IEEE Trans. Geosci. Remote. Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1109\/TGRS.2004.842478","article-title":"Classification of hyperspectral data from urban areas based on extended morphological profiles","volume":"43","author":"Benediktsson","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_16","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Fragkiadaki, K., Levine, S., Felsen, P., and Malik, J. (2015, January 7\u201313). Recurrent network models for human dynamics. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.494"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1109\/TPAMI.2015.2437384","article-title":"Region-based convolutional networks for accurate object detection and segmentation","volume":"38","author":"Girshick","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2381","DOI":"10.1109\/JSTARS.2015.2388577","article-title":"Spectral\u2013spatial classification of hyperspectral data based on deep belief network","volume":"8","author":"Chen","year":"2015","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2094","DOI":"10.1109\/JSTARS.2014.2329330","article-title":"Deep learning-based classification of hyperspectral data","volume":"7","author":"Chen","year":"2014","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"258619","DOI":"10.1155\/2015\/258619","article-title":"Deep convolutional neural networks for hyperspectral image classification","volume":"2015","author":"Hu","year":"2015","journal-title":"J. Sens."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Makantasis, K., Karantzalos, K., Doulamis, A., and Doulamis, N. (2015, January 26\u201331). Deep supervised learning for hyperspectral data classification through convolutional neural networks. Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy.","DOI":"10.1109\/IGARSS.2015.7326945"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6232","DOI":"10.1109\/TGRS.2016.2584107","article-title":"Deep feature extraction and classification of hyperspectral images based on convolutional neural networks","volume":"54","author":"Chen","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Mandic, D.P., and Chambers, J. (2001). Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability, John Wiley & Sons, Inc.","DOI":"10.1002\/047084535X"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3639","DOI":"10.1109\/TGRS.2016.2636241","article-title":"Deep recurrent neural networks for hyperspectral image classification","volume":"55","author":"Mou","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.neucom.2018.02.105","article-title":"Hyperspectral Image Classification Using Spectral-Spatial LSTMs","volume":"328","author":"Zhou","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1016\/j.patcog.2011.03.035","article-title":"A spatial\u2013spectral kernel-based approach for the classification of remote-sensing images","volume":"45","author":"Fauvel","year":"2012","journal-title":"Pattern Recognit."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.isprsjprs.2016.01.004","article-title":"Learning multiscale and deep representations for classifying remotely sensed imagery","volume":"113","author":"Zhao","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3\u20137). Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA.","DOI":"10.1145\/2647868.2654889"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_31","first-page":"3104","article-title":"Sequence to sequence learning with neural networks","volume":"27","author":"Sutskever","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/S0375-9601(00)00725-8","article-title":"Extended Tanh-function Method and its Applications to Nonlinear Equations","volume":"277","author":"Fan","year":"2000","journal-title":"Phys. Lett. A"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2673","DOI":"10.1109\/78.650093","article-title":"Bidirectional recurrent neural networks","volume":"45","author":"Schuster","year":"1997","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Bottou, L. (2010). Large-scale machine learning with stochastic gradient descent. Proceedings of COMPSTAT\u20192010, Springer.","DOI":"10.1007\/978-3-7908-2604-3_16"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1016\/j.isprsjprs.2017.11.021","article-title":"A new deep convolutional neural network for fast hyperspectral image classification","volume":"145","author":"Paoletti","year":"2017","journal-title":"Isprs J. Photogramm. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.engappai.2017.10.015","article-title":"Hyperspectral classification based on spectral\u2013spatial convolutional neural networks","volume":"68","author":"Chen","year":"2018","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_38","unstructured":"Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16\u201321). Rectifier nonlinearities improve neural network acoustic models. Proceedings of the ICML 2013, Atlanta, GA, USA."},{"key":"ref_39","first-page":"381","article-title":"Softmax to softassign: Neural network algorithms for combinatorial optimization","volume":"2","author":"Gold","year":"1996","journal-title":"J. Artif. Neural Netw."},{"key":"ref_40","unstructured":"Computational Intelligence Group of the Basque University (UPV\/EHU) (2018, July 12). Hyperspectral Remote Sensing Scenes. Available online: http:\/\/www.ehu.eus\/ccwintco\/index.php\/Hyperspectral_Remote_Sensing_Scenes."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1016\/0895-4356(88)90031-5","article-title":"A reappraisal of the kappa coefficient","volume":"41","author":"Thompson","year":"1988","journal-title":"J. Clin. Epidemiol."},{"key":"ref_42","first-page":"27","article-title":"LIBSVM: A library for support vector machines","volume":"2","author":"Chang","year":"2011","journal-title":"ACM Trans. Intell. Syst. Technol. (TIST)"},{"key":"ref_43","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning-Volume 37, Lille, France."},{"key":"ref_44","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_45","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/7\/1714\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:44:19Z","timestamp":1760186659000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/7\/1714"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,10]]},"references-count":45,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2019,4]]}},"alternative-id":["s19071714"],"URL":"https:\/\/doi.org\/10.3390\/s19071714","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,4,10]]}}}