{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T08:49:20Z","timestamp":1773391760930,"version":"3.50.1"},"reference-count":58,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,11]],"date-time":"2022-09-11T00:00:00Z","timestamp":1662854400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Gruppi operativi del PEI per la produttivit\u00e0 e la sostenibilit\u00e0 dell\u2019agricoltura di precisione","award":["2144\/2018"],"award-info":[{"award-number":["2144\/2018"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Estimating key crop parameters (e.g., phenology, yield prediction) is a prerequisite for optimizing agrifood supply chains through the use of satellite imagery, but requires timely and accurate crop mapping. The moment in the season and the number of training sites used are two main drivers of crop classification performance. The combined effect of these two parameters was analysed for tomato crop classification, through 125 experiments, using the three main machine learning (ML) classifiers (neural network, random forest, and support vector machine) using a response surface methodology (RSM). Crop classification performance between minority (tomato) and majority (\u2018other crops\u2019) classes was assessed through two evaluation metrics: Overall Accuracy (OA) and G-Mean (GM), which were calculated on large independent test sets (over 400,000 fields). RSM results demonstrated that lead time and the interaction between the number of majority and minority classes were the two most important drivers for crop classification performance for all three ML classifiers. The results demonstrate the feasibility of preharvest classification of tomato with high performance, and that an RSM-based approach enables the identification of simultaneous effects of several factors on classification performance. SVM achieved the best grading performances across the three ML classifiers, according to both evaluation metrics. SVM reached highest accuracy (0.95 of OA and 0.97 of GM) earlier in the season (low lead time) and with less training sites than the other two classifiers, permitting a reduction in cost and time for ground truth collection through field campaigns.<\/jats:p>","DOI":"10.3390\/rs14184540","type":"journal-article","created":{"date-parts":[[2022,9,13]],"date-time":"2022-09-13T04:05:41Z","timestamp":1663041941000},"page":"4540","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Impact of Training Set Size and Lead Time on Early Tomato Crop Mapping Accuracy"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7356-2774","authenticated-orcid":false,"given":"Michele","family":"Croci","sequence":"first","affiliation":[{"name":"Department of Sustainable Crop Production, Universit\u00e0 Cattolica del Sacro Cuore, 29122 Piacenza, Italy"},{"name":"Remote Sensing and Spatial Analysis Research Center (CRAST), Universit\u00e0 Cattolica del Sacro Cuore, 29122 Piacenza, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9878-6595","authenticated-orcid":false,"given":"Giorgio","family":"Impollonia","sequence":"additional","affiliation":[{"name":"Department of Sustainable Crop Production, Universit\u00e0 Cattolica del Sacro Cuore, 29122 Piacenza, Italy"},{"name":"Remote Sensing and Spatial Analysis Research Center (CRAST), Universit\u00e0 Cattolica del Sacro Cuore, 29122 Piacenza, Italy"}]},{"given":"Henri","family":"Blandini\u00e8res","sequence":"additional","affiliation":[{"name":"Department of Sustainable Crop Production, Universit\u00e0 Cattolica del Sacro Cuore, 29122 Piacenza, Italy"}]},{"given":"Michele","family":"Colauzzi","sequence":"additional","affiliation":[{"name":"Department of Sustainable Crop Production, Universit\u00e0 Cattolica del Sacro Cuore, 29122 Piacenza, Italy"}]},{"given":"Stefano","family":"Amaducci","sequence":"additional","affiliation":[{"name":"Department of Sustainable Crop Production, Universit\u00e0 Cattolica del Sacro Cuore, 29122 Piacenza, Italy"},{"name":"Remote Sensing and Spatial Analysis Research Center (CRAST), Universit\u00e0 Cattolica del Sacro Cuore, 29122 Piacenza, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,11]]},"reference":[{"key":"ref_1","first-page":"22","article-title":"Efficiency Assessment of Using Satellite Data for Crop Area Estimation in Ukraine","volume":"29","author":"Gallego","year":"2014","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_2","unstructured":"Craig, M., and Atkinson, D. (2013). A Literature Review of Crop Area Estimation, FAO Publication."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.compind.2019.02.002","article-title":"Sensing, Smart and Sustainable Technologies for Agri-Food 4.0","volume":"108","author":"Miranda","year":"2019","journal-title":"Comput. Ind."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"103187","DOI":"10.1016\/j.compind.2020.103187","article-title":"Agri-Food 4.0: A Survey of the Supply Chains and Technologies for the Future Agriculture","volume":"117","author":"Lezoche","year":"2020","journal-title":"Comput. Ind."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Immitzer, M., Vuolo, F., and Atzberger, C. (2016). First Experience with Sentinel-2 Data for Crop and Tree Species Classifications in Central Europe. Remote Sens., 8.","DOI":"10.3390\/rs8030166"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.rse.2014.02.015","article-title":"Good Practices for Estimating Area and Assessing Accuracy of Land Change","volume":"148","author":"Olofsson","year":"2014","journal-title":"Remote Sens. Environ."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kavats, O., Khramov, D., Sergieieva, K., and Vasyliev, V. (2020). Monitoring of Sugarcane Harvest in Brazil Based on Optical and {SAR} Data. Remote Sens., 12.","DOI":"10.3390\/rs12244080"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kavats, O., Khramov, D., Sergieieva, K., and Vasyliev, V. (2019). Monitoring Harvesting by Time Series of Sentinel-1 {SAR} Data. Remote Sens., 11.","DOI":"10.3390\/rs11212496"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.34133\/2021\/8379391","article-title":"Mapping Crop Phenology in Near Real-Time Using Satellite Remote Sensing: Challenges and Opportunities","volume":"2021","author":"Gao","year":"2021","journal-title":"J. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"112232","DOI":"10.1016\/j.rse.2020.112232","article-title":"Comparing Land Surface Phenology of Major European Crops as Derived from {SAR} and Multispectral Data of Sentinel-1 and -2","volume":"253","author":"Meroni","year":"2021","journal-title":"Remote Sens. Environ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.isprsjprs.2019.11.008","article-title":"Estimating Wheat Yields in Australia Using Climate Records, Satellite Image Time Series and Machine Learning Methods","volume":"160","author":"Kamir","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"108555","DOI":"10.1016\/j.agrformet.2021.108555","article-title":"Yield Forecasting with Machine Learning and Small Data: What Gains for Grains?","volume":"308\u2013309","author":"Meroni","year":"2021","journal-title":"Agric. For. Meteorol."},{"key":"ref_13","unstructured":"FAO, IFAD, IMF, OECD, UNCTAD, WFP, World Bank, WTO, IFPRI, and United Nations High Level Task Force on Global Food and Nutrition (2011). Price Volatility in Food and Agricultural Markets: Policy Responses, World Bank."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"361","DOI":"10.5721\/EuJRS20164920","article-title":"Assessing In-Season Crop Classification Performance Using Satellite Data: A Test Case in Northern Italy","volume":"49","author":"Azar","year":"2016","journal-title":"Eur. J. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.rse.2006.03.004","article-title":"Training Set Size Requirements for the Classification of a Specific Class","volume":"104","author":"Foody","year":"2006","journal-title":"Remote Sens. Environ."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ramezan, C.A., Warner, T.A., Maxwell, A.E., and Price, B.S. (2021). Effects of Training Set Size on Supervised Machine-Learning Land-Cover Classification of Large-Area High-Resolution Remotely Sensed Data. Remote Sens., 13.","DOI":"10.3390\/rs13030368"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"799","DOI":"10.1080\/014311697218764","article-title":"An Evaluation of Some Factors Affecting the Accuracy of Classification by an Artificial Neural Network","volume":"18","author":"Foody","year":"1997","journal-title":"Int. J. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1335","DOI":"10.1109\/TGRS.2004.827257","article-title":"A Relative Evaluation of Multiclass Image Classification by Support Vector Machines","volume":"42","author":"Foody","year":"2004","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Congalton, R.G., and Green, K. (2008). Assessing the Accuracy of Remotely Sensed Data: Principles and Practices, CRC Press. [2nd ed.].","DOI":"10.1201\/9781420055139"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S0034-4257(01)00295-4","article-title":"Status of Land Cover Classification Accuracy Assessment","volume":"80","author":"Foody","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1707","DOI":"10.1080\/01431169508954507","article-title":"The Effect of Training Set Size and Composition on Artificial Neural Network Classification","volume":"16","author":"Foody","year":"1995","journal-title":"Int. J. Remote Sens"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"8489","DOI":"10.3390\/rs70708489","article-title":"On the Importance of Training Data Sample Selection in Random Forest Image Classification: A Case Study in Peatland Ecosystem Mapping","volume":"7","author":"Millard","year":"2015","journal-title":"Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"153","DOI":"10.3390\/rs70100153","article-title":"Comparing Machine Learning Classifiers for Object-Based Land Cover Classification Using Very High Resolution Imagery","volume":"7","author":"Qian","year":"2015","journal-title":"Remote Sens"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"648","DOI":"10.1016\/j.rse.2017.09.035","article-title":"Effect of Classifier Selection, Reference Sample Size, Reference Class Distribution and Scene Heterogeneity in per-Pixel Classification Accuracy Using 26 Landsat Sites","volume":"204","author":"Heydari","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_25","first-page":"18","article-title":"Comparison of Random Forest, k-Nearest Neighbor, and Support Vector Machine Classifiers for Land Cover Classification Using Sentinel-2 Imagery","volume":"18","author":"Noi","year":"2018","journal-title":"Sensors"},{"key":"ref_26","first-page":"13","article-title":"Effect of Feature Dimensionality on Object-Based Land Cover Classification: A Comparison of Three Classifiers","volume":"2","author":"Myburgh","year":"2013","journal-title":"South Afr. J. Geomat."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Dean, A., Voss, D., and Dragulji\u0107, D. (2017). Response Surface Methodology. Springer Texts in Statistics, Springer International Publishing.","DOI":"10.1007\/978-3-319-52250-0_16"},{"key":"ref_28","first-page":"439","article-title":"Updated World Map of the K\u00f6ppen-Geiger Climate Classification","volume":"4","author":"Peel","year":"2007","journal-title":"Hydrol. Earth Syst. Sci. Discuss."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.rse.2011.11.026","article-title":"Sentinel-2: ESA\u2019s Optical High-Resolution Mission for GMES Operational Services","volume":"120","author":"Drusch","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_30","unstructured":"(2021, January 20). THEIA Value-Adding Products and Algorithms for Land Surfaces. Available online: https:\/\/www.theia-land.fr\/."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Comer\u00f3n, A., Kassianov, E.I., and Sch\u00e4fer, K. (2016). MACCS-ATCOR Joint Algorithm (MAJA). Proceedings of the Remote Sensing of Clouds and the Atmosphere XXI, SPIE.","DOI":"10.1117\/12.2240935"},{"key":"ref_32","unstructured":"(2021, December 20). GDAL Documentation. Available online: www.gdal.org."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.rse.2018.10.031","article-title":"Intra-Annual Reflectance Composites from Sentinel-2 and Landsat for National-Scale Crop and Land Cover Mapping","volume":"220","author":"Griffiths","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_34","first-page":"309","article-title":"Monitoring Vegetation Systems in the Great Plains with ERTS","volume":"1","author":"Rouse","year":"1973","journal-title":"NASA ERTS Symp."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/1011-1344(93)06963-4","article-title":"Quantitative Estimation of Chlorophyll-a Using Reflectance Spectra: Experiments with Autumn Chestnut and Maple Leaves","volume":"22","author":"Gitelson","year":"1994","journal-title":"J. Photochem. Photobiol. B"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1016\/S0034-4257(96)00067-3","article-title":"NDWI\u2014A Normalized Difference Water Index for Remote Sensing of Vegetation Liquid Water from Space","volume":"58","author":"Gao","year":"1996","journal-title":"Remote Sens. Environ."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1080\/10106040108542184","article-title":"Spatially Located Platform and Aerial Photography for Documentation of Grazing Impacts on Wheat","volume":"16","author":"Louhaichi","year":"2001","journal-title":"Geocarto Int."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11119-008-9075-z","article-title":"A Broad-Band Leaf Chlorophyll Vegetation Index at the Canopy Scale","volume":"9","author":"Vincini","year":"2008","journal-title":"Precis. Agric."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1078\/0176-1617-01176","article-title":"Wide Dynamic Range Vegetation Index for Remote Quantification of Biophysical Characteristics of Vegetation","volume":"161","author":"Gitelson","year":"2004","journal-title":"J. Plant Physiol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v032.i07","article-title":"Response-Surface Methods InR, Usingrsm","volume":"32","author":"Lenth","year":"2009","journal-title":"J. Stat. Softw."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/0925-2312(91)90023-5","article-title":"Multilayer Perceptrons for Classification and Regression","volume":"2","author":"Murtagh","year":"1991","journal-title":"Neurocomputing"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Vapnik, V. (1998). The Support Vector Method of Function Estimation. Nonlinear Modeling, Springer US.","DOI":"10.7551\/mitpress\/1130.003.0006"},{"key":"ref_44","unstructured":"Kuhn, M., and Johnson, K. (2019). Applied Predictive Modeling, Springer."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1214\/09-SS054","article-title":"A Survey of Cross-Validation Procedures for Model Selection","volume":"4","author":"Arlot","year":"2010","journal-title":"Stat. Surv."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Ramezan, C.A., Warner, T.A., and Maxwell, A.E. (2019). Evaluation of Sampling and Cross-Validation Tuning Strategies for Regional-Scale Machine Learning Classification. Remote Sens., 11.","DOI":"10.3390\/rs11020185"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1080\/01621459.1984.10478083","article-title":"Cross-Validation of Regression Models","volume":"79","author":"Picard","year":"1984","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from Imbalanced Data","volume":"21","author":"He","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"111375","DOI":"10.1016\/j.rse.2019.111375","article-title":"Needle in a Haystack: Mapping Rare and Infrequent Crops Using Satellite Imagery and Data Balancing Methods","volume":"233","author":"Waldner","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_50","first-page":"102114","article-title":"All Pixels Are Useful, but Some Are More Useful: Efficient in Situ Data Collection for Crop-Type Mapping Using Sequential Exploration Methods","volume":"91","author":"Fowler","year":"2020","journal-title":"ITC J."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1122","DOI":"10.1080\/2150704X.2017.1362124","article-title":"The Impact of Training Class Proportions on Binary Cropland Classification","volume":"8","author":"Waldner","year":"2017","journal-title":"Remote Sens. Lett."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"105164","DOI":"10.1016\/j.compag.2019.105164","article-title":"Pre-Harvest Classification of Crop Types Using a Sentinel-2 Time-Series and Machine Learning","volume":"169","author":"Maponya","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1016\/j.rse.2017.07.015","article-title":"Understanding the Temporal Behavior of Crops Using Sentinel-1 and Sentinel-2-like Data for Agricultural Applications","volume":"199","author":"Veloso","year":"2017","journal-title":"Remote Sens. Environ."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1016\/j.isprsjprs.2016.11.004","article-title":"Optimizing Selection of Training and Auxiliary Data for Operational Land Cover Classification for the LCMAP Initiative","volume":"122","author":"Zhu","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1333","DOI":"10.1007\/s12524-018-0777-z","article-title":"Effects of Training Samples and Classifiers on Classification of Landsat-8 Imagery","volume":"46","author":"Shang","year":"2018","journal-title":"J. Ind. Soc. Remote Sens."},{"key":"ref_56","first-page":"103","article-title":"A Support Vector Machine to Identify Irrigated Crop Types Using Time-Series Landsat NDVI Data","volume":"34","author":"Zheng","year":"2015","journal-title":"ITC J."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.compag.2004.06.003","article-title":"Determining Temporal Windows for Crop Discrimination with Remote Sensing: A Case Study in South-Eastern Australia","volume":"45","author":"McVicar","year":"2004","journal-title":"Comput. Electron. Agric."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"13208","DOI":"10.3390\/rs71013208","article-title":"An Automated Method for Annual Cropland Mapping along the Season for Various Globally-Distributed Agrosystems Using High Spatial and Temporal Resolution Time Series","volume":"7","author":"Matton","year":"2015","journal-title":"Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/18\/4540\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:29:28Z","timestamp":1760142568000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/18\/4540"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,11]]},"references-count":58,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["rs14184540"],"URL":"https:\/\/doi.org\/10.3390\/rs14184540","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,11]]}}}