{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,6]],"date-time":"2026-01-06T02:12:29Z","timestamp":1767665549739,"version":"build-2065373602"},"reference-count":48,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2020,7,12]],"date-time":"2020-07-12T00:00:00Z","timestamp":1594512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Tubitak","award":["118E258"],"award-info":[{"award-number":["118E258"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>In this article, we propose an end-to-end deep network for the classification of multi-spectral time series and apply them to crop type mapping. Long short-term memory networks (LSTMs) are well established in this regard, thanks to their capacity to capture both long and short term temporal dependencies. Nevertheless, dealing with high intra-class variance and inter-class similarity still remain significant challenges. To address these issues, we propose a straightforward approach where LSTMs are combined with metric learning. The proposed architecture accommodates three distinct branches with shared weights, each containing a LSTM module, that are merged through a triplet loss. It thus not only minimizes classification error, but enforces the sub-networks to produce more discriminative deep features. It is validated via Breizhcrops, a very recently introduced and challenging time series dataset for crop type mapping.<\/jats:p>","DOI":"10.3390\/jimaging6070068","type":"journal-article","created":{"date-parts":[[2020,7,14]],"date-time":"2020-07-14T04:46:01Z","timestamp":1594701961000},"page":"68","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["A Discriminative Long Short Term Memory Network with Metric Learning Applied to Multispectral Time Series Classification"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5532-4948","authenticated-orcid":false,"given":"Merve","family":"Bozo","sequence":"first","affiliation":[{"name":"Department of Computer Engineering, Istanbul Technical University, Maslak, Istanbul 34469, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6168-2883","authenticated-orcid":false,"given":"Erchan","family":"Aptoula","sequence":"additional","affiliation":[{"name":"Institute of Information Technologies, Gebze Technical University, Kocaeli 41400, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zehra","family":"\u00c7ataltepe","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Istanbul Technical University, Maslak, Istanbul 34469, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1016\/j.rse.2017.10.005","article-title":"Sentinel-2 cropland mapping using pixel-based and object-based time-weighted dynamic time warping analysis","volume":"204","author":"Belgiu","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1560","DOI":"10.1109\/JPROC.2015.2449668","article-title":"Multimodal Classification of Remote Sensing Images: A Review and Future Directions","volume":"103","author":"Tuia","year":"2015","journal-title":"Proc. IEEE"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1016\/j.rse.2018.11.007","article-title":"Near real-time agriculture monitoring at national scale at parcel resolution: Performance assessment of the Sen2-Agri automated system in various cropping systems around the world","volume":"221","author":"Defourny","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_4","first-page":"225","article-title":"Automated cropland mapping of continental Africa using Google EarthEngine cloud computing","volume":"126","author":"Xiong","year":"2017","journal-title":"Remote Sens. Environ."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1016\/j.rse.2014.11.024","article-title":"Improved time series land cover classification by missing-observation-adaptive nonlinear dimensionality reduction","volume":"158","author":"Yan","year":"2015","journal-title":"Remote Sens. Environ."},{"key":"ref_6","unstructured":"Solano-Correa, Y.T., Bovolo, F., and Bruzzone, L. (August, January 28). A Semi-supervised Crop-type Classification Based on Sentinel-2 NDVI Satellite Image Time Series and Phenological Parameters. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3081","DOI":"10.1109\/TGRS.2011.2179050","article-title":"Satellite image time series analysis under time warping","volume":"50","author":"Petitjean","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1143","DOI":"10.1109\/LGRS.2013.2288358","article-title":"Efficient Satellite Image Time Series Analysis Under Time Warping","volume":"11","author":"Petitjean","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Csillik, O., Belgiu, M., Asner, G.P., and Kelly, M. (2019). Object-Based Time-Constrained Dynamic Time Warping Classification of Crops Using Sentinel-2. Remote Sens., 11.","DOI":"10.3390\/rs11101257"},{"key":"ref_10","unstructured":"Furby, S., Caccetta, P., Wu, X., and Chia, J. (2008, January 1\u20136). Continental scale land cover change monitoring in Australia using Landsat imagery. Proceedings of the International Earth Conference: Studying, Modeling and Sense Making of Planet Earth, Lesvos, Greece."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1303","DOI":"10.1080\/014311600210191","article-title":"Development of a global land cover characteristics database and IGBP DISCover from 1 km AVHRR data","volume":"21","author":"Loveland","year":"2000","journal-title":"Int. J. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Tang, P., and Corpetti, T. (2016, January 10\u201315). Satellite image time series clustering via affinity propagation. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Beijing, China.","DOI":"10.1109\/IGARSS.2016.7729624"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3926","DOI":"10.3390\/rs6053923","article-title":"MAD-MEX: Automatic wall-to-wall land cover monitoring for the Mexican REDD-MRV program using all Landsat data","volume":"6","author":"Gebhardt","year":"2014","journal-title":"Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.rse.2012.08.013","article-title":"Land change for all municipalities in Latin America and the Caribbean assessed from 250-m MODIS imagery (2001\u20132010)","volume":"126","author":"Clark","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Inglada, J., Vincent, A., Arias, M., Tardy, B., Morin, D., and Rodes, I. (2017). Operational high resolution land cover map production at the country scale using satellite image time series. Remote Sens., 9.","DOI":"10.3390\/rs9010095"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.rse.2016.10.010","article-title":"Assessing the robustness of Random Forests to map land cover with high resolution satellite image time series over large areas","volume":"187","author":"Pelletier","year":"2016","journal-title":"Remote Sens. Environ."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.isprsjprs.2016.03.008","article-title":"Optical remotely sensed time series data for land cover classification: A review","volume":"116","author":"Gomez","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_19","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20138). ImageNet classification with deep convolutional neural networks. Proceedings of the International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/j.isprsjprs.2019.04.015","article-title":"Deep learning in remote sensing applications: A meta-analysis and review","volume":"152","author":"Ma","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sun, Y., Luo, J., Wu, T., Yang, Y., Liu, H., Dong, W., Gao, L., and Hu, X. (2019, January 16\u201319). Geo-parcel based Crops Classification with Sentinel-1 Time Series Data via Recurrent Reural Network. Proceedings of the IEEE International Conference on Agro-Geoinformatics, Istanbul, Turkey.","DOI":"10.1109\/Agro-Geoinformatics.2019.8820218"},{"key":"ref_23","first-page":"1303","article-title":"Using Long Short-Term Memory Recurrent Neural Network in land cover classification on Land-sat and Cropland data layer time series","volume":"40","author":"Sun","year":"2018","journal-title":"Int. J. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1016\/j.rse.2018.11.032","article-title":"Deep learning based multi-temporal crop classification","volume":"221","author":"Zhong","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_25","unstructured":"Avolio, C., Tricomi, A., Mammone, C., Zavagli, M., and Costantini, M. (August, January 28). A deep learning architecture for heterogeneous and irregularly sampled remote sensing time series. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan."},{"key":"ref_26","unstructured":"Garnot, V.S.F., Landrieu, L., Giordano, S., and Chehata, N. (August, January 28). Time-Space Tradeoff in Deep Learning Models for Crop Classification on Satellite Multi-Spectral Image Time Series. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Luo, J., Fen, L., and Zhou, X. (2019). DCN-Based Spatial Features for Improving Parcel-Based Crop Classification Using High-Resolution Optical Images and Multi-Temporal SAR Data. Remote Sens., 11.","DOI":"10.3390\/rs11131619"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ru\u00dfwurm, M., and K\u00f6rner, M. (2017, January 21\u201326). Temporal Vegetation Modelling Using Long Short-Term Memory Networks for Crop Identification from Medium-Resolution Multi-spectral Satellite Images. Proceedings of the Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.193"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ru\u00dfwurm, M., and K\u00f6rner, M. (2018). Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders. ISPRS Int. J. Geo Inf., 7.","DOI":"10.3390\/ijgi7040129"},{"key":"ref_30","unstructured":"Pelletier, C., Webb, G.I., and Petitjean, F. (August, January 28). Deep Learning for the Classification of Sentinel-2 Image Time Series. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan."},{"key":"ref_31","unstructured":"Xing, E.P., Ng, A.Y., Jordan, M.I., and Russel, S. (2002, January 9\u201314). Distance Metric Learning, with Application to Clustering with Side-Information. Proceedings of the International Conference on Neural Information Processing, Vancouver, BC, Canada."},{"key":"ref_32","unstructured":"Tian, Z., Zhang, Z., Mei, S., Jiang, R., Wan, S., and Du, Q. (August, January 28). Discriminative CNN via Metric Learning for Hyperspectral Classification. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1109\/JSTARS.2017.2761800","article-title":"Scene classification via triplet networks","volume":"11","author":"Liu","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2811","DOI":"10.1109\/TGRS.2017.2783902","article-title":"When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs","volume":"56","author":"Cheng","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1109\/LGRS.2019.2894399","article-title":"Siamese Convolutional Neural Networks for Remote Sensing Scene Classification","volume":"16","author":"Liu","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_36","unstructured":"Bromley, J., Guyon, I., LeCun, Y., S\u00e4ckinger, E., and Shah, R. (December, January 29). Signature Verification using a \u201cSiamese\u201d Time Delay Neural Network. Proceedings of the Conference on Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1845","DOI":"10.1109\/LGRS.2017.2738149","article-title":"Change Detection Based on Deep Siamese Convolutional Network for Optical Aerial Images","volume":"14","author":"Zhan","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","unstructured":"Chopra, S., Hadsell, R., and LeCun, Y. (2005, January 20\u201325). Learning a similarity metric discriminatively, with application to face verification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hoffer, E., and Ailon, N. (2015, January 12\u201314). Deep Metric Learning Using Triplet Network. Proceedings of the International Workshop on Similarity-Based Pattern Recognition, Copenhagen, Denmark.","DOI":"10.1007\/978-3-319-24261-3_7"},{"key":"ref_40","unstructured":"Ru\u00dfwurm, M., Lef\u00e8vre, S., and K\u00f6rner, M. (2019, January 10\u201315). BreizhCrops: A Time Series Dataset for Crop Type Identification. Proceedings of the Time Series Workshop of the 36th International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1002\/rob.21869","article-title":"Transfer learning between crop types for semantic segmentation of crops versus weeds in precision agriculture","volume":"37","author":"Bosilj","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Eigen, D., and Fergus, R. (2015, January 11\u201318). Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. Proceedings of the IEEE International Conference on Computer Vision, Las Condes, Chile.","DOI":"10.1109\/ICCV.2015.304"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ru\u00dfwurm, M., Pelletier, C., Zollner, M., Lef\u00e8vre, S., and K\u00f6rner, M. (2020). BreizhCrops: A Time Series Dataset for Crop Type Mapping. arXiv.","DOI":"10.5194\/isprs-archives-XLIII-B2-2020-1545-2020"},{"key":"ref_44","unstructured":"Kingma, D., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representation, San Diego, CA, USA."},{"key":"ref_45","unstructured":"Turkoglu, M.O., D\u2019Aronco, S., Wegner, J.D., and Schindler, K. (2019). Gating Revisited: Deep Multi-layer RNNs That Can BeTrained. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"111604","DOI":"10.1016\/j.rse.2019.111604","article-title":"Seamless retrievals of chlorophyll-a from Sentinel-2 (MSI) and Sentinel-3 (OLCI) in inland and coastal waters: A machine-learning approach","volume":"240","author":"Pahlevan","year":"2020","journal-title":"Remote Sens. Environ."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1685","DOI":"10.1109\/LGRS.2017.2728698","article-title":"Land Cover Classification via Multitemporal Spatial Data by Deep Recurrent Neural Networks","volume":"14","author":"Ienco","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1109\/LGRS.2017.2681128","article-title":"Deep learning classification of land cover and crop types using remote sensing data","volume":"14","author":"Kussul","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/6\/7\/68\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:50:37Z","timestamp":1760176237000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/6\/7\/68"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,12]]},"references-count":48,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2020,7]]}},"alternative-id":["jimaging6070068"],"URL":"https:\/\/doi.org\/10.3390\/jimaging6070068","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2020,7,12]]}}}