{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,10]],"date-time":"2026-05-10T20:13:29Z","timestamp":1778444009786,"version":"3.51.4"},"reference-count":45,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,6,24]],"date-time":"2022-06-24T00:00:00Z","timestamp":1656028800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"FCT\/MCTES through national funds","award":["UIDB\/50008\/2020"],"award-info":[{"award-number":["UIDB\/50008\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Precision irrigation and optimization of water use have become essential factors in agriculture because water is critical for crop growth. The proper management of an irrigation system should enable the farmer to use water efficiently to increase productivity, reduce production costs, and maximize the return on investment. Efficient water application techniques are essential prerequisites for sustainable agricultural development based on the conservation of water resources and preservation of the environment. In a previous work, an off-policy deep reinforcement learning model, Deep Q-Network, was implemented to optimize irrigation. The performance of the model was tested for tomato crop at a site in Portugal. In this paper, an on-policy model, Advantage Actor\u2013Critic, is implemented to compare irrigation scheduling with Deep Q-Network for the same tomato crop. The results show that the on-policy model Advantage Actor\u2013Critic reduced water consumption by 20% compared to Deep Q-Network with a slight change in the net reward. These models can be developed to be applied to other cultures with high production in Portugal, such as fruit, cereals, and wine, which also have large water requirements.<\/jats:p>","DOI":"10.3390\/computers11070104","type":"journal-article","created":{"date-parts":[[2022,6,25]],"date-time":"2022-06-25T10:39:13Z","timestamp":1656153553000},"page":"104","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Comparison of On-Policy Deep Reinforcement Learning A2C with Off-Policy DQN in Irrigation Optimization: A Case Study at a Site in Portugal"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2319-8211","authenticated-orcid":false,"given":"Khadijeh","family":"Alibabaei","sequence":"first","affiliation":[{"name":"C-MAST Center for Mechanical and Aerospace Science and Technologies, University of Beira Interior, 6201-001 Covilh\u00e3, Portugal"},{"name":"Department of Electromechanical Engineering, University of Beira Interior, Rua Marqu\u00eas d\u2019\u00c1vila e Bolama, 6201-001 Covilh\u00e3, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1691-1709","authenticated-orcid":false,"given":"Pedro D.","family":"Gaspar","sequence":"additional","affiliation":[{"name":"C-MAST Center for Mechanical and Aerospace Science and Technologies, University of Beira Interior, 6201-001 Covilh\u00e3, Portugal"},{"name":"Department of Electromechanical Engineering, University of Beira Interior, Rua Marqu\u00eas d\u2019\u00c1vila e Bolama, 6201-001 Covilh\u00e3, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6027-7763","authenticated-orcid":false,"given":"Eduardo","family":"Assun\u00e7\u00e3o","sequence":"additional","affiliation":[{"name":"C-MAST Center for Mechanical and Aerospace Science and Technologies, University of Beira Interior, 6201-001 Covilh\u00e3, Portugal"},{"name":"Department of Electromechanical Engineering, University of Beira Interior, Rua Marqu\u00eas d\u2019\u00c1vila e Bolama, 6201-001 Covilh\u00e3, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4440-7211","authenticated-orcid":false,"given":"Saeid","family":"Alirezazadeh","sequence":"additional","affiliation":[{"name":"C4\u2014Cloud Computing Competence Centre (C4-UBI), University of Beira Interior, Rua Marqu\u00eas d\u2019\u00c1vila e Bolama, 6201-001 Covilh\u00e3, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7540-3854","authenticated-orcid":false,"given":"T\u00e2nia M.","family":"Lima","sequence":"additional","affiliation":[{"name":"C-MAST Center for Mechanical and Aerospace Science and Technologies, University of Beira Interior, 6201-001 Covilh\u00e3, Portugal"},{"name":"Department of Electromechanical Engineering, University of Beira Interior, Rua Marqu\u00eas d\u2019\u00c1vila e Bolama, 6201-001 Covilh\u00e3, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8057-5474","authenticated-orcid":false,"given":"Vasco N. G. J.","family":"Soares","sequence":"additional","affiliation":[{"name":"Polytechnic Institute of Castelo Branco, Av. Pedro \u00c1lvares Cabral n\u00ba 12, 6000-084 Castelo Branco, Portugal"},{"name":"Instituto de Telecomunica\u00e7\u00f5es, Rua Marqu\u00eas d\u2019\u00c1vila e Bolama, 6201-001 Covilh\u00e3, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5830-3790","authenticated-orcid":false,"given":"Jo\u00e3o M. L. P.","family":"Caldeira","sequence":"additional","affiliation":[{"name":"Polytechnic Institute of Castelo Branco, Av. Pedro \u00c1lvares Cabral n\u00ba 12, 6000-084 Castelo Branco, Portugal"},{"name":"Instituto de Telecomunica\u00e7\u00f5es, Rua Marqu\u00eas d\u2019\u00c1vila e Bolama, 6201-001 Covilh\u00e3, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Muimba-Kankolongo, A. (2018). Chapter 3\u2013Factors Important to Crop Production. Food Crop Production by Smallholder Farmers in Southern Africa, Academic Press.","DOI":"10.1016\/B978-0-12-814383-4.00003-7"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Fahad, S., Bajwa, A.A., Nazir, U., Anjum, S.A., Farooq, A., Zohaib, A., Sadia, S., Nasim, W., Adkins, S., and Saud, S. (2017). Crop Production under Drought and Heat Stress: Plant Responses and Management Options. Front. Plant Sci., 8.","DOI":"10.3389\/fpls.2017.01147"},{"key":"ref_3","unstructured":"FAO (2020, June 05). World Agriculture 2030: Main Findings. Available online: http:\/\/www.fao.org\/english\/newsroom\/news\/2002\/7833-en.html."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1079\/PAVSNNR201611035","article-title":"Internet of Things in agriculture","volume":"11","author":"Verdouw","year":"2016","journal-title":"CAB Rev."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lohchab, V., Kumar, M., Suryan, G., Gautam, V., and Das, R.K. (2018, January 20\u201321). A Review of IoT based Smart Farm Monitoring. Proceedings of the 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), Coimbatore, India.","DOI":"10.1109\/ICICCT.2018.8473337"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1016\/j.procs.2019.11.016","article-title":"Smart Farming using IoT, a solution for optimally monitoring farming conditions","volume":"160","author":"Doshi","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_7","first-page":"559","article-title":"Soil moisture sensor for agricultural applications inspired from state of art study of surfaces scattering models & semi-empirical soil moisture models","volume":"20","author":"Shakya","year":"2021","journal-title":"J. Saudi Soc. Agric. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"110513","DOI":"10.1016\/j.measurement.2021.110513","article-title":"Design of novel Penta core PCF SPR RI sensor based on fusion of IMD and EMD techniques for analysis of water and transformer oil","volume":"188","author":"Singh","year":"2022","journal-title":"Measurement"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"70","DOI":"10.3390\/agriengineering4010006","article-title":"Precision Irrigation Management Using Machine Learning and Digital Farming Solutions","volume":"4","author":"Abioye","year":"2022","journal-title":"AgriEngineering"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liakos, K., Busato, P.B., Moshou, D., Pearson, S., and Bochtis, D. (2018). Machine Learning in Agriculture: A Review. Sensors, 18.","DOI":"10.3390\/s18082674"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.compag.2018.02.016","article-title":"Deep learning in agriculture: A survey","volume":"147","author":"Kamilaris","year":"2018","journal-title":"Comput. Electron. Agric."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Alibabaei, K., Gaspar, P.D., Lima, T.M., Campos, R.M., Gir\u00e3o, I., Monteiro, J., and Lopes, C.M. (2022). A Review of the Challenges of Using Deep Learning Algorithms to Support Decision-Making in Agricultural Activities. Remote Sens., 14.","DOI":"10.3390\/rs14030638"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/s10462-018-09679-z","article-title":"Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: A survey","volume":"52","author":"Nguyen","year":"2019","journal-title":"Artif. Intell. Rev."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"39501","DOI":"10.1109\/ACCESS.2018.2855437","article-title":"A Survey of Clustering with Deep Learning: From the Perspective of Network Architecture","volume":"6","author":"Min","year":"2018","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zia, H., Rehman, A., Harris, N.R., Fatima, S., and Khurram, M. (2021). An Experimental Comparison of IoT-Based and Traditional Irrigation Scheduling on a Flood-Irrigated Subtropical Lemon Farm. Sensors, 21.","DOI":"10.3390\/s21124175"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Tseng, D., Wang, D., Chen, C., Miller, L., Song, W., Viers, J., Vougioukas, S., Carpin, S., Ojea, J.A., and Goldberg, K. (2018, January 20\u201324). Towards Automating Precision Irrigation: Deep Learning to Infer Local Soil Moisture Conditions from Synthetic Aerial Agricultural Images. Proceedings of the 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), Munich, Germany.","DOI":"10.1109\/COASE.2018.8560431"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1007\/s40333-016-0049-0","article-title":"Modeling spatio-temporal distribution of soil moisture by deep learning-based cellular automata model","volume":"8","author":"Song","year":"2016","journal-title":"J. Arid. Land"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1016\/j.compag.2018.11.031","article-title":"Reference evapotranspiration estimation and modeling of the Punjab Northern India using deep learning","volume":"156","author":"Saggi","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"105700","DOI":"10.1016\/j.compag.2020.105700","article-title":"Reference evapotranspiration time series forecasting with ensemble of convolutional neural networks","volume":"177","author":"Alves","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ahmed, A.A.M., Deo, R.C., Raj, N., Ghahramani, A., Feng, Q., Yin, Z., and Yang, L. (2021). Deep Learning Forecasts of Soil Moisture: Convolutional Neural Network and Gated Recurrent Unit Models Coupled with Satellite-Derived MODIS, Observations and Synoptic-Scale Climate Index Data. Remote Sens., 13.","DOI":"10.3390\/rs13040554"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Adab, H., Morbidelli, R., Saltalippi, C., Moradian, M., and Ghalhari, G.A.F. (2020). Machine Learning to Estimate Surface Soil Moisture from Remote Sensing Data. Water, 12.","DOI":"10.3390\/w12113223"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1327","DOI":"10.13031\/trans.13765","article-title":"Evaluation of two recurrent neural network methods for prediction of irrigation rate and timing","volume":"63","author":"Jimenez","year":"2020","journal-title":"Trans. ASABE"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1016\/j.future.2019.04.041","article-title":"A smart agriculture IoT system based on deep reinforcement learning","volume":"99","author":"Bu","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"106838","DOI":"10.1016\/j.agwat.2021.106838","article-title":"A reinforcement learning approach to irrigation decision-making for rice using weather forecasts","volume":"250","author":"Chen","year":"2021","journal-title":"Agric. Water Manag."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"107480","DOI":"10.1016\/j.agwat.2022.107480","article-title":"Irrigation optimization with a deep reinforcement learning model: Case study on a site in Portugal","volume":"263","author":"Alibabaei","year":"2022","journal-title":"Agric. Water Manag."},{"key":"ref_26","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Harley, T., Lillicrap, T.P., Silver, D., and Kavukcuoglu, K. Asynchronous Methods for Deep Reinforcement Learning. In Proceedings of the 33rd International Conference on Machine Learning, PMLR, New York, NY, USA, 19\u201324 June 2016, Volume 48."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hoogenboom, G., Porter, C.H., Boote, K.J., Shelia, V., Wilkens, P.W., Singh, U., White, J.W., Asseng, S., Lizaso, J.I., and Cadena, P.M. (2019). The DSSAT crop modeling ecosystem. Advances in Crop Modelling for a Sustainable Agriculture, Burleigh Dodds Science Publishing.","DOI":"10.19103\/AS.2019.0061.10"},{"key":"ref_28","unstructured":"Hoogenboom, G., Porter, C., Shelia, V., Boote, K., Singh, U., White, J., Hunt, L., Ogoshi, R., Lizaso, J., and Koo, J. (2019). Decision Support System for Agrotechnology Transfer (DSSAT) Version 4.7.5, DSSAT Foundation."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Alibabaei, K., Gaspar, P.D., and Lima, T.M. (2021). Crop Yield Estimation Using Deep Learning Based on Climate Big Data and Irrigation Scheduling. Energies, 14.","DOI":"10.3390\/en14113004"},{"key":"ref_30","unstructured":"Allen, R.G., Pereira, L.S., and Raes, M.S.D. (1998). Crop Evapotranspiration\u2013Guidelines for Computing Crop Water Requirements FAO Irrigation and Drainage Paper 56, FAO\u2013Food and Agriculture Organization of the United Nations."},{"key":"ref_31","unstructured":"Montgomery, D.C., Jennings, C.L., and Kulahci, M. (2011). Introduction to Time Series Analysis and Forecasting, Wiley."},{"key":"ref_32","unstructured":"James, G., Witten, D., Hastie, T., and Tibshirani, R. (2014). An Introduction to Statistical Learning: With Applications in R, Springer Publishing Company, Incorporated."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Jain, L.C., and Medsker, L.R. (1999). Recurrent Neural Networks: Design and Applications, CRC Press, Inc.. [1st ed.].","DOI":"10.1201\/9781420049176"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"8995","DOI":"10.1029\/JC090iC05p08995","article-title":"Statistics for the evaluation and comparison of models","volume":"90","author":"Willmott","year":"1985","journal-title":"J. Geophys. Res. Ocean."},{"key":"ref_35","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press. [2nd ed.]."},{"key":"ref_36","unstructured":"Konda, V.R., and Tsitsiklis, J.N. (2000). Actor\u2013Critic Algorithms, MIT Press."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/BF00115009","article-title":"Learning to Predict by the Methods of Temporal Differences","volume":"3","author":"Sutton","year":"1988","journal-title":"Mach. Learn."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1291","DOI":"10.1109\/TSMCC.2012.2218595","article-title":"A Survey of Actor\u2013Critic Reinforcement Learning: Standard and Natural Policy Gradients","volume":"42","author":"Grondman","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part C Appl. Rev."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1080\/09540099108946587","article-title":"Function Optimization using Connectionist Reinforcement Learning Algorithms","volume":"3","author":"Williams","year":"1991","journal-title":"Connect. Sci."},{"key":"ref_40","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing Atari with Deep Reinforcement Learning. arXiv."},{"key":"ref_41","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2020, December 01). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available online: tensorflow.org."},{"key":"ref_42","unstructured":"Chollet, F. (2020, December 01). Available online: https:\/\/keras.io."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Alibabaei, K., Gaspar, P.D., and Lima, T.M. (2021). Modeling Soil Water Content and Reference Evapotranspiration from Climate Data Using Deep Learning Method. Appl. Sci., 11.","DOI":"10.3390\/app11115029"},{"key":"ref_44","unstructured":"Patterson, J., and Gibson, A. (2017). Deep Learning: A Practitioner\u2019s Approach, O\u2019Reilly Media."},{"key":"ref_45","unstructured":"Rodrigues, L.C. (2020, July 01). Water Resources Fee in Portugali, 2016. Led by the Institute for European Environmental Policy. Available online: https:\/\/ieep.eu\/."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/11\/7\/104\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:39:44Z","timestamp":1760139584000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/11\/7\/104"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,24]]},"references-count":45,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["computers11070104"],"URL":"https:\/\/doi.org\/10.3390\/computers11070104","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,24]]}}}