{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T05:23:01Z","timestamp":1772601781620,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2020,12,7]],"date-time":"2020-12-07T00:00:00Z","timestamp":1607299200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Unmanned aerial vehicle (UAV) based remote sensing is gaining momentum worldwide in a variety of agricultural and environmental monitoring and modelling applications. At the same time, the increasing availability of yield monitoring devices in harvesters enables input-target mapping of in-season RGB and crop yield data in a resolution otherwise unattainable by openly availabe satellite sensor systems. Using time series UAV RGB and weather data collected from nine crop fields in Pori, Finland, we evaluated the feasibility of spatio-temporal deep learning architectures in crop yield time series modelling and prediction with RGB time series data. Using Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) networks as spatial and temporal base architectures, we developed and trained CNN-LSTM, convolutional LSTM and 3D-CNN architectures with full 15 week image frame sequences from the whole growing season of 2018. The best performing architecture, the 3D-CNN, was then evaluated with several shorter frame sequence configurations from the beginning of the season. With 3D-CNN, we were able to achieve 218.9 kg\/ha mean absolute error (MAE) and 5.51% mean absolute percentage error (MAPE) performance with full length sequences. The best shorter length sequence performance with the same model was 292.8 kg\/ha MAE and 7.17% MAPE with four weekly frames from the beginning of the season.<\/jats:p>","DOI":"10.3390\/rs12234000","type":"journal-article","created":{"date-parts":[[2020,12,7]],"date-time":"2020-12-07T21:37:42Z","timestamp":1607377062000},"page":"4000","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":135,"title":["Crop Yield Prediction Using Multitemporal UAV Data and Spatio-Temporal Deep Learning Models"],"prefix":"10.3390","volume":"12","author":[{"given":"Petteri","family":"Nevavuori","sequence":"first","affiliation":[{"name":"Mtech Digital Solutions Oy, 01301 Vantaa, Finland"}]},{"given":"Nathaniel","family":"Narra","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Communication Sciences, Tampere University, 33014 Tampere, Finland"}]},{"given":"Petri","family":"Linna","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Communication Sciences, Tampere University, 33014 Tampere, Finland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5112-2425","authenticated-orcid":false,"given":"Tarmo","family":"Lipping","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Communication Sciences, Tampere University, 33014 Tampere, Finland"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,7]]},"reference":[{"key":"ref_1","first-page":"175","article-title":"A Data Driven Approach to Decision Support in Farming","volume":"Volume 321","author":"Narra","year":"2020","journal-title":"Information Modelling and Knowledge Bases XXXI"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"104859","DOI":"10.1016\/j.compag.2019.104859","article-title":"Crop yield prediction with deep convolutional neural networks","volume":"163","author":"Nevavuori","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sainath, T.N., Vinyals, O., Senior, A., and Sak, H. (2015, January 19\u201324). Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia.","DOI":"10.1109\/ICASSP.2015.7178838"},{"key":"ref_4","first-page":"802","article-title":"Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting","volume":"28","author":"Shi","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 7\u201313). Learning spatiotemporal features with 3D convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.510"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sun, J., Di, L., Sun, Z., Shen, Y., and Lai, Z. (2019). County-Level Soybean Yield Prediction Using Deep CNN-LSTM Model. Sensors, 19.","DOI":"10.3390\/s19204363"},{"key":"ref_7","unstructured":"Rustowicz, R., Cheong, R., Wang, L., Ermon, S., Burke, M., and Lobell, D. (2019, January 16\u201320). Semantic Segmentation of Crop Type in Africa: A Novel Dataset and Analysis of Deep Learning Methods. Proceedings of the CVPR Workshops, Long Beach, CA, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"105664","DOI":"10.1016\/j.compag.2020.105664","article-title":"Pre-season crop type mapping using deep neural networks","volume":"176","author":"Yaramasu","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ji, S., Zhang, C., Xu, A., Shi, Y., and Duan, Y. (2018). 3D Convolutional Neural Networks for Crop Classification with Multi-Temporal Remote Sensing Images. Remote Sens., 10.","DOI":"10.3390\/rs10010075"},{"key":"ref_10","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, CA, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liu, Q., Zhou, F., Hang, R., and Yuan, X. (2017). Bidirectional-Convolutional LSTM Based Spectral-Spatial Feature Learning for Hyperspectral Image Classification. Remote Sens., 9.","DOI":"10.3390\/rs9121330"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.isprsjprs.2019.09.016","article-title":"Combining Sentinel-1 and Sentinel-2 Satellite Image Time Series for land cover mapping via a multi-source deep learning architecture","volume":"158","author":"Ienco","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"105351","DOI":"10.1016\/j.compag.2020.105351","article-title":"Prediction of maize growth stages based on deep learning","volume":"172","author":"Yue","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Li, Y., Zhang, H., and Shen, Q. (2017). Spectral\u2013Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network. Remote Sens., 9.","DOI":"10.3390\/rs9010067"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Barbosa, A., Trevisan, R., Hovakimyan, N., and Martin, N.F. (2020). Modeling yield response to crop management using convolutional neural networks. Comput. Electron. Agric., 170.","DOI":"10.1016\/j.compag.2019.105197"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1007\/978-3-319-10590-1_53","article-title":"Visualizing and Understanding Convolutional Networks","volume":"Volume 8689","author":"Zeiler","year":"2014","journal-title":"Computer Vision\u2014ECCV 2014"},{"key":"ref_19","unstructured":"Ioffe, S., and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep Learning in Neural Networks: An Overview","volume":"61","author":"Schmidhuber","year":"2014","journal-title":"Neural Netw."},{"key":"ref_22","unstructured":"Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2017, January 4\u20139). Automatic differentiation in PyTorch. Proceedings of the NIPS-W, Long Beach, CA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2673","DOI":"10.1109\/78.650093","article-title":"Bidirectional recurrent neural networks","volume":"45","author":"Schuster","year":"1997","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Graves, A. (2013). Generating Sequences with Recurrent Neural Networks. arXiv.","DOI":"10.1007\/978-3-642-24797-2_3"},{"key":"ref_25","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_26","first-page":"281","article-title":"Random Search for Hyper-Parameter Optimization","volume":"13","author":"Bergstra","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_27","unstructured":"Zeiler, M.D. (2012). ADADELTA: An Adaptive Learning Rate Method. arXiv."},{"key":"ref_28","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. ICLR, 1\u201315."},{"key":"ref_29","unstructured":"Tietz, M., Fan, T.J., Nouri, D., Bossan, B., and Skorch Developers (2020, October 16). Skorch: A Scikit-Learn Compatible Neural Network Library That Wraps PyTorch. Available online: https:\/\/skorch.readthedocs.io\/."},{"key":"ref_30","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Glorot","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Messina, G., and Modica, G. (2020). Applications of UAV thermal imagery in precision agriculture: State of the art and future research outlook. Remote Sens., 12.","DOI":"10.3390\/rs12091491"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sun, C., Feng, L., Zhang, Z., Ma, Y., Crosby, T., Naber, M., and Wang, Y. (2020). Prediction of end-of-season tuber yield and tuber set in potatoes using in-season uav-based hyperspectral imagery and machine learning. Sensors, 20.","DOI":"10.3390\/s20185293"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"454","DOI":"10.1080\/07038992.2020.1788384","article-title":"Intra-Field Canopy Nitrogen Retrieval from Unmanned Aerial Vehicle Imagery for Wheat and Corn Fields","volume":"46","author":"Lee","year":"2020","journal-title":"Can. J. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Fu, Z., Jiang, J., Gao, Y., Krienke, B., Wang, M., Zhong, K., Cao, Q., Tian, Y., Zhu, Y., and Cao, W. (2020). Wheat growth monitoring and yield estimation based on multi-rotor unmanned aerial vehicle. Remote Sens., 12.","DOI":"10.3390\/rs12030508"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1109\/MGRS.2019.2898520","article-title":"A review of change detection in multitemporal hyperspectral images: Current techniques, applications, and challenges","volume":"7","author":"Liu","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Borra-Serrano, I., Swaef, T.D., Quataert, P., Aper, J., Saleem, A., Saeys, W., Somers, B., Rold\u00e1n-Ruiz, I., and Lootens, P. (2020). Closing the phenotyping gap: High resolution UAV time series for soybean growth analysis provides objective data from field trials. Remote Sens., 12.","DOI":"10.3390\/rs12101644"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MGRS.2018.2890023","article-title":"Multisource and multitemporal data fusion in remote sensing: A comprehensive review of the state of the art","volume":"7","author":"Ghamisi","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Hauglin, M., and \u00d8rka, H.O. (2016). Discriminating between native norway spruce and invasive sitka spruce\u2014A comparison of multitemporal Landsat 8 imagery, aerial images and airborne laser scanner data. Remote Sens., 8.","DOI":"10.3390\/rs8050363"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/23\/4000\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:41:51Z","timestamp":1760179311000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/23\/4000"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,7]]},"references-count":38,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["rs12234000"],"URL":"https:\/\/doi.org\/10.3390\/rs12234000","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,7]]}}}