{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T15:37:49Z","timestamp":1791473869269,"version":"4.3.4"},"reference-count":96,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,3,12]],"date-time":"2024-03-12T00:00:00Z","timestamp":1710201600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,12]],"date-time":"2024-03-12T00:00:00Z","timestamp":1710201600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007052","name":"Universit\u00e0 degli Studi di Verona","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007052","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Environmental sustainability is a worldwide key challenge attracting increasing attention due to climate change, pollution, and biodiversity decline. Reinforcement learning, initially employed in gaming contexts, has been recently applied to real-world domains, including the environmental sustainability realm, where uncertainty challenges strategy learning and adaptation. In this work, we survey the literature to identify the main applications of reinforcement learning in environmental sustainability and the predominant methods employed to address these challenges. We analyzed 181 papers and answered seven research questions, e.g., \u201cHow many academic studies have been published from 2003 to 2023 about RL for environmental sustainability?\u201d and \u201cWhat were the application domains and the methodologies used?\u201d. Our analysis reveals an exponential growth in this field over the past two decades, with a rate of 0.42 in the number of publications (from 2 papers in 2007 to 53 in 2022), a strong interest in sustainability issues related to energy fields, and a preference for single-agent RL approaches to deal with sustainability. Finally, this work provides practitioners with a clear overview of the main challenges and open problems that should be tackled in future research.<\/jats:p>","DOI":"10.1007\/s10462-024-10706-5","type":"journal-article","created":{"date-parts":[[2024,3,12]],"date-time":"2024-03-12T20:02:37Z","timestamp":1710273757000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":50,"title":["Reinforcement learning applications in environmental sustainability: a review"],"prefix":"10.1007","volume":"57","author":[{"given":"Maddalena","family":"Zuccotto","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alberto","family":"Castellini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Davide La","family":"Torre","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lapo","family":"Mola","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alessandro","family":"Farinelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,3,12]]},"reference":[{"key":"10706_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.iswa.2023.200216","author":"L Ajao","year":"2023","unstructured":"Ajao L, Apeh S (2023) Secure edge computing vulnerabilities in smart cities sustainability using petri net and genetic algorithm-based reinforcement learning. Intell Syst Appl. https:\/\/doi.org\/10.1016\/j.iswa.2023.200216","journal-title":"Intell Syst Appl"},{"key":"10706_CR2","doi-asserted-by":"publisher","unstructured":"Al-Jawad A, Com\u015fa I, Shah P, et\u00a0al (2021) REDO: a reinforcement learning-based dynamic routing algorithm selection method for SDN. In: IEEE conference on network function virtualization and software defined networks (NFV-SDN), pp 54\u201359, https:\/\/doi.org\/10.1109\/NFV-SDN53031.2021.9665140","DOI":"10.1109\/NFV-SDN53031.2021.9665140"},{"key":"10706_CR3","doi-asserted-by":"publisher","DOI":"10.1016\/j.scs.2021.103445","author":"K Alanne","year":"2022","unstructured":"Alanne K, Sierla S (2022) An overview of machine learning applications for smart buildings. Sustain Cities Soc. https:\/\/doi.org\/10.1016\/j.scs.2021.103445","journal-title":"Sustain Cities Soc"},{"key":"10706_CR4","doi-asserted-by":"publisher","unstructured":"Alizadeh\u00a0Shabestray SM, Abdulhai B (2019) Multimodal iNtelligent Deep (MiND) traffic signal controller. In: IEEE intelligent transportation systems conference (ITSC), pp 4532\u20134539, https:\/\/doi.org\/10.1109\/ITSC.2019.8917493","DOI":"10.1109\/ITSC.2019.8917493"},{"key":"10706_CR5","doi-asserted-by":"publisher","DOI":"10.1145\/3057730","author":"F Auffenberg","year":"2017","unstructured":"Auffenberg F, Snow S, Stein S et al (2017) A comfort-based approach to smart heating and air conditioning. ACM Trans Intell Syst Technol. https:\/\/doi.org\/10.1145\/3057730","journal-title":"ACM Trans Intell Syst Technol"},{"key":"10706_CR6","doi-asserted-by":"publisher","DOI":"10.1080\/15472450.2017.1387546","author":"H Aziz","year":"2018","unstructured":"Aziz H, Zhu F, Ukkusuri S (2018) Learning-based traffic signal control algorithms with neighborhood information sharing: an application for sustainable mobility. J Intell Trans Syst Technol Plan Operat. https:\/\/doi.org\/10.1080\/15472450.2017.1387546","journal-title":"J Intell Trans Syst Technol Plan Operat"},{"key":"10706_CR7","doi-asserted-by":"publisher","unstructured":"Azzalini D, Castellini A, Luperto M, et\u00a0al (2020) HMMs for anomaly detection in autonomous robots. In: Proceedings of the 2020 international conference on autonomous agents and multiagent systems, AAMAS, p 105\u2013113, https:\/\/doi.org\/10.5555\/3398761.3398779","DOI":"10.5555\/3398761.3398779"},{"issue":"3","key":"10706_CR8","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1007\/s13173-011-0038-2","volume":"17","author":"ALC Bazzan","year":"2011","unstructured":"Bazzan ALC, Peleteiro-Ramallo A, Burguillo-Rial JC (2011) Learning to cooperate in the iterated prisoner\u2019s dilemma by means of social attachments. J Braz Comput Soc 17(3):163\u2013174. https:\/\/doi.org\/10.1007\/s13173-011-0038-2","journal-title":"J Braz Comput Soc"},{"key":"10706_CR9","doi-asserted-by":"publisher","unstructured":"Bianchi F, Castellini A, Tarocco P, et\u00a0al (2019) Load forecasting in district heating networks: Model comparison on a real-world case study. In: Machine learning, optimization, and data science: 5th international conference, LOD 2019, proceedings. Springer-Verlag, p 553\u2013565, https:\/\/doi.org\/10.1007\/978-3-030-37599-7_46","DOI":"10.1007\/978-3-030-37599-7_46"},{"key":"10706_CR10","unstructured":"Bianchi F, Corsi D, Marzari L, et\u00a0al (2023) Safe and efficient reinforcement learning for environmental monitoring. In: Proceedings of Ital-IA 2023: 3rd National Conference on Artificial Intelligence, CEUR Workshop Proceedings, vol 3486. CEUR-WS.org, pp 2610\u2013615"},{"key":"10706_CR11","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1016\/j.artint.2017.02.004","volume":"246","author":"F Bistaffa","year":"2017","unstructured":"Bistaffa F, Farinelli A, Chalkiadakis G et al (2017) A cooperative game-theoretic approach to the social ridesharing problem. Artif Intell 246:86\u2013117. https:\/\/doi.org\/10.1016\/j.artint.2017.02.004","journal-title":"Artif Intell"},{"issue":"1","key":"10706_CR12","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1109\/TITS.2019.2954982","volume":"22","author":"F Bistaffa","year":"2021","unstructured":"Bistaffa F, Blum C, Cerquides J et al (2021) A computational approach to quantify the benefits of ridesharing for policy makers and travellers. IEEE Trans Intell Transport Syst 22(1):119\u2013130. https:\/\/doi.org\/10.1109\/TITS.2019.2954982","journal-title":"IEEE Trans Intell Transport Syst"},{"key":"10706_CR13","doi-asserted-by":"publisher","unstructured":"Blij NHVD, Chaifouroosh D, Ca\u00f1izares CA, et\u00a0al (2020) Improved power flow methods for DC grids. In: 29th IEEE international symposium on industrial electronics, ISIE. IEEE, pp 1135\u20131140, https:\/\/doi.org\/10.1109\/ISIE45063.2020.9152570","DOI":"10.1109\/ISIE45063.2020.9152570"},{"key":"10706_CR14","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3002538","author":"O Bouhamed","year":"2020","unstructured":"Bouhamed O, Ghazzai H, Besbes H et al (2020) A UAV-assisted data collection for wireless sensor networks: Autonomous navigation and scheduling. IEEE Access. https:\/\/doi.org\/10.1109\/ACCESS.2020.3002538","journal-title":"IEEE Access"},{"key":"10706_CR15","doi-asserted-by":"publisher","unstructured":"Brown J, Abate A, Rogers A (2021) QUILT: quantify, infer and label the thermal efficiency of heating and cooling residential homes. In: BuildSys \u201921: The 8th ACM international conference on systems for energy-efficient buildings, cities, and transportation. ACM, pp 51\u201360, https:\/\/doi.org\/10.1145\/3486611.3486653","DOI":"10.1145\/3486611.3486653"},{"key":"10706_CR16","doi-asserted-by":"publisher","unstructured":"Capuzzo M, Zanella A, Zuccotto M, et\u00a0al (2022) IoT systems for healthy and safe life environments. In: IEEE forum on research and technologies for society and industry innovation (RTSI), pp 31\u201337, https:\/\/doi.org\/10.1109\/RTSI55261.2022.9905193","DOI":"10.1109\/RTSI55261.2022.9905193"},{"key":"10706_CR17","doi-asserted-by":"publisher","unstructured":"Castellini A, Chalkiadakis G, Farinelli A (2019) Influence of state-variable constraints on partially observable monte carlo planning. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI 2019. International Joint Conferences on Artificial Intelligence Organization, pp 5540\u20135546, https:\/\/doi.org\/10.24963\/ijcai.2019\/769","DOI":"10.24963\/ijcai.2019\/769"},{"key":"10706_CR18","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2020.103499","author":"A Castellini","year":"2020","unstructured":"Castellini A, Bicego M, Masillo F et al (2020) Time series segmentation for state-model generation of autonomous aquatic drones: a systematic framework. Eng Appl Artif Intell. https:\/\/doi.org\/10.1016\/j.engappai.2020.103499","journal-title":"Eng Appl Artif Intell"},{"issue":"4","key":"10706_CR19","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/MIS.2020.3005903","volume":"36","author":"A Castellini","year":"2021","unstructured":"Castellini A, Bianchi F, Farinelli A (2021) Predictive model generation for load forecasting in district heating networks. IEEE Intell Syst 36(4):86\u201395. https:\/\/doi.org\/10.1109\/MIS.2020.3005903","journal-title":"IEEE Intell Syst"},{"issue":"9","key":"10706_CR20","doi-asserted-by":"publisher","first-page":"9621","DOI":"10.1007\/s10489-021-02949-4","volume":"52","author":"A Castellini","year":"2022","unstructured":"Castellini A, Bianchi F, Farinelli A (2022) Generation and interpretation of parsimonious predictive models for load forecasting in smart heating networks. Appl Intell 52(9):9621\u20139637. https:\/\/doi.org\/10.1007\/s10489-021-02949-4","journal-title":"Appl Intell"},{"key":"10706_CR21","unstructured":"Castellini A, Bianchi F, Zorzi E, et\u00a0al (2023) Scalable safe policy improvement via Monte Carlo tree search. In: Proceedings of the 40th international conference on machine learning, proceedings of machine learning research, vol 202. PMLR, pp 3732\u20133756"},{"key":"10706_CR22","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2022.103212","author":"N Charef","year":"2023","unstructured":"Charef N, Ben Mnaouer A, Aloqaily M et al (2023) Artificial intelligence implication on energy sustainability in internet of things: a survey. Info Process Manag. https:\/\/doi.org\/10.1016\/j.ipm.2022.103212","journal-title":"Info Process Manag"},{"key":"10706_CR23","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2016.2517084","author":"H Chen","year":"2016","unstructured":"Chen H, Li X, Zhao F (2016) A reinforcement learning-based sleep scheduling algorithm for desired area coverage in solar-powered wireless sensor networks. IEEE Sensors Journal. https:\/\/doi.org\/10.1109\/JSEN.2016.2517084","journal-title":"IEEE Sensors Journal"},{"key":"10706_CR24","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2958175","author":"H Chen","year":"2019","unstructured":"Chen H, Zhao T, Li C et al (2019) Green internet of vehicles: Architecture, enabling technologies, and applications. IEEE Access. https:\/\/doi.org\/10.1109\/ACCESS.2019.2958175","journal-title":"IEEE Access"},{"key":"10706_CR25","doi-asserted-by":"publisher","DOI":"10.1016\/j.chemosphere.2021.130498","author":"K Chen","year":"2021","unstructured":"Chen K, Wang H, Valverde-P\u00e9rez B et al (2021) Optimal control towards sustainable wastewater treatment plants based on multi-agent reinforcement learning. Chemosphere. https:\/\/doi.org\/10.1016\/j.chemosphere.2021.130498","journal-title":"Chemosphere"},{"key":"10706_CR26","doi-asserted-by":"publisher","DOI":"10.1016\/j.enbuild.2015.06.045","author":"A De Gracia","year":"2015","unstructured":"De Gracia A, Fern\u00e1ndez C, Castell A et al (2015) Control of a PCM ventilated facade using reinforcement learning techniques. Energy Build. https:\/\/doi.org\/10.1016\/j.enbuild.2015.06.045","journal-title":"Energy Build"},{"key":"10706_CR27","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2992480","author":"D Elavarasan","year":"2020","unstructured":"Elavarasan D, Durairaj Vincent P (2020) Crop yield prediction using deep reinforcement learning model for sustainable agrarian applications. IEEE Access. https:\/\/doi.org\/10.1109\/ACCESS.2020.2992480","journal-title":"IEEE Access"},{"key":"10706_CR28","doi-asserted-by":"publisher","DOI":"10.1016\/j.jenvman.2022.117046","author":"O Emamjomehzadeh","year":"2023","unstructured":"Emamjomehzadeh O, Kerachian R, Emami-Skardi M et al (2023) Combining urban metabolism and reinforcement learning concepts for sustainable water resources management: a nexus approach. J Environ Manag. https:\/\/doi.org\/10.1016\/j.jenvman.2022.117046","journal-title":"J Environ Manag"},{"key":"10706_CR29","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2022.3202252","author":"Y Feng","year":"2023","unstructured":"Feng Y, Zhang X, Jia R et al (2023) Intelligent trajectory design for mobile energy harvesting and data transmission. IEEE Internet Things J. https:\/\/doi.org\/10.1109\/JIOT.2022.3202252","journal-title":"IEEE Internet Things J"},{"key":"10706_CR30","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclepro.2022.135782","author":"Y Gao","year":"2023","unstructured":"Gao Y, Chang D, Chen CH (2023) A digital twin-based approach for optimizing operation energy consumption at automated container terminals. J Clean Prod. https:\/\/doi.org\/10.1016\/j.jclepro.2022.135782","journal-title":"J Clean Prod"},{"key":"10706_CR31","doi-asserted-by":"publisher","DOI":"10.1016\/j.phycom.2022.101766","author":"MK Giri","year":"2022","unstructured":"Giri MK, Majumder S (2022) Deep Q-learning based optimal resource allocation method for energy harvested cognitive radio networks. Phys Commun. https:\/\/doi.org\/10.1016\/j.phycom.2022.101766","journal-title":"Phys Commun"},{"issue":"1","key":"10706_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1146\/annurev.es.26.110195.000245","volume":"26","author":"R Goodland","year":"1995","unstructured":"Goodland R (1995) The concept of environmental sustainability. Ann Rev Ecol Syst 26(1):1\u201324. https:\/\/doi.org\/10.1146\/annurev.es.26.110195.000245","journal-title":"Ann Rev Ecol Syst"},{"key":"10706_CR33","doi-asserted-by":"publisher","DOI":"10.3390\/s23084176","author":"Z Gu","year":"2023","unstructured":"Gu Z, Liu Z, Wang Q et al (2023) Reinforcement learning-based approach for minimizing energy loss of driving platoon decisions. Sensors. https:\/\/doi.org\/10.3390\/s23084176","journal-title":"Sensors"},{"key":"10706_CR34","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2020.2990733","author":"M Han","year":"2020","unstructured":"Han M, Duan J, Khairy S et al (2020) Enabling sustainable underwater IoT networks with energy harvesting: a decentralized reinforcement learning approach. IEEE Internet Things J. https:\/\/doi.org\/10.1109\/JIOT.2020.2990733","journal-title":"IEEE Internet Things J"},{"key":"10706_CR35","doi-asserted-by":"publisher","DOI":"10.1016\/j.energy.2021.121958","author":"D Harrold","year":"2022","unstructured":"Harrold D, Cao J, Fan Z (2022) Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning. Energy. https:\/\/doi.org\/10.1016\/j.energy.2021.121958","journal-title":"Energy"},{"key":"10706_CR36","unstructured":"Hausknecht M, Stone P (2015) Deep recurrent Q-learning for partially observable MDPs. Preprint at https:\/\/arxiv.org\/abs\/1507.06527"},{"issue":"4","key":"10706_CR37","doi-asserted-by":"publisher","first-page":"660","DOI":"10.1109\/TWC.2002.804190","volume":"1","author":"W Heinzelman","year":"2002","unstructured":"Heinzelman W, Chandrakasan A, Balakrishnan H (2002) An application-specific protocol architecture for wireless microsensor networks. IEEE Trans Wireless Commun 1(4):660\u2013670. https:\/\/doi.org\/10.1109\/TWC.2002.804190","journal-title":"IEEE Trans Wireless Commun"},{"key":"10706_CR38","doi-asserted-by":"crossref","unstructured":"Hessel M, Modayil J, van Hasselt H, et\u00a0al (2018) Rainbow: combining improvements in deep reinforcement learning. In: Proceedings of the AAAI conference on artificial intelligence, pp 3215\u20133222","DOI":"10.1609\/aaai.v32i1.11796"},{"issue":"6","key":"10706_CR39","doi-asserted-by":"publisher","first-page":"4929","DOI":"10.1007\/s10462-022-10286-2","volume":"56","author":"Y Himeur","year":"2022","unstructured":"Himeur Y, Elnour M, Fadli F et al (2022) Ai-big data analytics for building automation and management systems: a survey, actual challenges and future perspectives. Artif Intell Rev 56(6):4929\u20135021. https:\/\/doi.org\/10.1007\/s10462-022-10286-2","journal-title":"Artif Intell Rev"},{"key":"10706_CR40","doi-asserted-by":"publisher","DOI":"10.1109\/TETC.2014.2316518","author":"R Hsu","year":"2014","unstructured":"Hsu R, Liu CT, Wang HL (2014) A reinforcement learning-based ToD provisioning dynamic power management for sustainable operation of energy harvesting wireless sensor node. IEEE Trans Emerg Topics Comput. https:\/\/doi.org\/10.1109\/TETC.2014.2316518","journal-title":"IEEE Trans Emerg Topics Comput"},{"key":"10706_CR41","doi-asserted-by":"publisher","DOI":"10.1016\/j.resconrec.2022.106664","author":"D Huo","year":"2023","unstructured":"Huo D, Sari Y, Kealey R et al (2023) Reinforcement learning-based fleet dispatching for greenhouse gas emission reduction in open-pit mining operations. Resour Conserv Recycl. https:\/\/doi.org\/10.1016\/j.resconrec.2022.106664","journal-title":"Resour Conserv Recycl"},{"key":"10706_CR42","doi-asserted-by":"publisher","DOI":"10.1016\/j.segan.2022.100919","author":"I Jendoubi","year":"2022","unstructured":"Jendoubi I, Bouffard F (2022) Data-driven sustainable distributed energy resources\u2019 control based on multi-agent deep reinforcement learning. Sustain Energy Grids Netw. https:\/\/doi.org\/10.1016\/j.segan.2022.100919","journal-title":"Sustain Energy Grids Netw"},{"issue":"1\u20132","key":"10706_CR43","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/S0004-3702(98)00023-X","volume":"101","author":"LP Kaelbling","year":"1998","unstructured":"Kaelbling LP, Littman ML, Cassandra AR (1998) Planning and acting in partially observable stochastic domains. Artif Intell 101(1\u20132):99\u2013134. https:\/\/doi.org\/10.1016\/S0004-3702(98)00023-X","journal-title":"Artif Intell"},{"key":"10706_CR44","doi-asserted-by":"publisher","DOI":"10.1016\/j.egyai.2021.100101","author":"A Kathirgamanathan","year":"2021","unstructured":"Kathirgamanathan A, Mangina E, Finn D (2021) Development of a soft actor critic deep reinforcement learning approach for harnessing energy flexibility in a large office building. Energy AI. https:\/\/doi.org\/10.1016\/j.egyai.2021.100101","journal-title":"Energy AI"},{"key":"10706_CR45","doi-asserted-by":"publisher","DOI":"10.1016\/j.scs.2022.104311","author":"M Khalid","year":"2023","unstructured":"Khalid M, Wang L, Wang K et al (2023) Deep reinforcement learning-based long-range autonomous valet parking for smart cities. Sustain Cities Soc. https:\/\/doi.org\/10.1016\/j.scs.2022.104311","journal-title":"Sustain Cities Soc"},{"issue":"5","key":"10706_CR46","doi-asserted-by":"publisher","first-page":"2128","DOI":"10.1109\/TITS.2019.2914087","volume":"21","author":"AM Koufakis","year":"2020","unstructured":"Koufakis AM, Rigas ES, Bassiliades N et al (2020) Offline and online electric vehicle charging scheduling with V2V energy transfer. IEEE Trans Intell Transport Syst 21(5):2128\u20132138. https:\/\/doi.org\/10.1109\/TITS.2019.2914087","journal-title":"IEEE Trans Intell Transport Syst"},{"issue":"143\u2013155","key":"10706_CR47","first-page":"18","volume":"19","author":"Y LeCun","year":"1989","unstructured":"LeCun Y (1989) Generalization and network design strategies. Connect Perspect 19(143\u2013155):18","journal-title":"Connect Perspect"},{"key":"10706_CR48","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclepro.2020.124405","author":"J Leng","year":"2021","unstructured":"Leng J, Ruan G, Song Y et al (2021) A loosely-coupled deep reinforcement learning approach for order acceptance decision of mass-individualized printed circuit board manufacturing in industry 4.0. J Clean Prod. https:\/\/doi.org\/10.1016\/j.jclepro.2020.124405","journal-title":"J Clean Prod"},{"key":"10706_CR49","doi-asserted-by":"publisher","DOI":"10.1080\/21680566.2023.2179461","author":"C Li","year":"2023","unstructured":"Li C, Bai L, Yao L et al (2023) A bibliometric analysis and review on reinforcement learning for transportation applications. Transportmetrica B. https:\/\/doi.org\/10.1080\/21680566.2023.2179461","journal-title":"Transportmetrica B"},{"key":"10706_CR50","unstructured":"Lillicrap TP, Hunt JJ, Pritzel A, et\u00a0al (2016) Continuous control with deep reinforcement learning. In: International conference on learning representations, ICLR"},{"key":"10706_CR51","doi-asserted-by":"publisher","DOI":"10.1109\/MWC.016.2100179","author":"Q Liu","year":"2021","unstructured":"Liu Q, Sun S, Rong B et al (2021) Intelligent reflective surface based 6G communications for sustainable energy infrastructure. IEEE Wireless Commun. https:\/\/doi.org\/10.1109\/MWC.016.2100179","journal-title":"IEEE Wireless Commun"},{"key":"10706_CR52","unstructured":"Lowe R, Wu Y, Tamar A, et\u00a0al (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. In: Proceedings of the international conference on neural information processing systems, NIPS, p 6382-6393"},{"issue":"2","key":"10706_CR53","doi-asserted-by":"publisher","first-page":"1222","DOI":"10.1109\/COMST.2019.2962526","volume":"22","author":"D Ma","year":"2020","unstructured":"Ma D, Lan G, Hassan M et al (2020) Sensing, computing, and communications for energy harvesting IoTs: a survey. IEEE Commun Surv Tutor 22(2):1222\u20131250. https:\/\/doi.org\/10.1109\/COMST.2019.2962526","journal-title":"IEEE Commun Surv Tutor"},{"key":"10706_CR54","doi-asserted-by":"publisher","DOI":"10.1016\/j.enbuild.2021.111085","author":"P Mabina","year":"2021","unstructured":"Mabina P, Mukoma P, Booysen M (2021) Sustainability matchmaking: linking renewable sources to electric water heating through machine learning. Energy Build. https:\/\/doi.org\/10.1016\/j.enbuild.2021.111085","journal-title":"Energy Build"},{"key":"10706_CR55","doi-asserted-by":"publisher","unstructured":"Marchesini E, Corsi D, Farinelli A (2021) Benchmarking safe deep reinforcement learning in aquatic navigation. In: IEEE\/RSJ international conference on intelligent robots and systems, IROS. IEEE, pp 5590\u20135595, https:\/\/doi.org\/10.1109\/IROS51168.2021.9635925","DOI":"10.1109\/IROS51168.2021.9635925"},{"key":"10706_CR56","doi-asserted-by":"publisher","unstructured":"Mazzi G, Castellini A, Farinelli A (2021) Rule-based shielding for partially observable monte-carlo planning. In: Proceedings of the international conference on automated planning and scheduling pp 243\u2013251. https:\/\/doi.org\/10.1609\/icaps.v31i1.15968","DOI":"10.1609\/icaps.v31i1.15968"},{"key":"10706_CR57","doi-asserted-by":"publisher","first-page":"103987","DOI":"10.1016\/j.artint.2023.103987","volume":"324","author":"G Mazzi","year":"2023","unstructured":"Mazzi G, Castellini A, Farinelli A (2023) Risk-aware shielding of partially observable monte carlo planning policies. Artif Intell 324:103987","journal-title":"Artif Intell"},{"key":"10706_CR58","doi-asserted-by":"publisher","unstructured":"Miozzo M, Giupponi L, Rossi M, et\u00a0al (2015) Distributed Q-learning for energy harvesting heterogeneous networks. In: IEEE international conference on communication workshop (ICCW), pp 2006\u20132011, https:\/\/doi.org\/10.1109\/ICCW.2015.7247475","DOI":"10.1109\/ICCW.2015.7247475"},{"key":"10706_CR59","doi-asserted-by":"publisher","unstructured":"Miozzo M, Giupponi L, Rossi M, et\u00a0al (2017) Switch-on\/off policies for energy harvesting small cells through distributed Q-learning. In: IEEE wireless communications and networking conference workshops (WCNCW), pp 1\u20136, https:\/\/doi.org\/10.1109\/WCNCW.2017.7919075","DOI":"10.1109\/WCNCW.2017.7919075"},{"key":"10706_CR60","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-023-10441-3","author":"S Mischos","year":"2023","unstructured":"Mischos S, Dalagdi E, Vrakas D (2023) Intelligent energy management systems: a review. Artif Intell Rev. https:\/\/doi.org\/10.1007\/s10462-023-10441-3","journal-title":"Artif Intell Rev"},{"issue":"7540","key":"10706_CR61","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih V, Kavukcuoglu K, Silver D et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529\u2013533. https:\/\/doi.org\/10.1038\/nature14236","journal-title":"Nature"},{"key":"10706_CR62","doi-asserted-by":"publisher","unstructured":"Moerland TM, Broekens J, Plaat A, et\u00a0al (2020) Model-based reinforcement learning: a survey. arXiv abs\/2206.09328. https:\/\/doi.org\/10.48550\/ARXIV.2206.09328","DOI":"10.48550\/ARXIV.2206.09328"},{"key":"10706_CR63","unstructured":"Ng A, Harada D, Russell SJ (1999) Policy invariance under reward transformations: theory and application to reward shaping. In: Proceedings of the international conference on machine learning, ICML, p 278\u2013287"},{"key":"10706_CR64","doi-asserted-by":"publisher","unstructured":"Orfanoudakis S, Chalkiadakis G (2023) A novel aggregation framework for the efficient integration of distributed energy resources in the smart grid. In: Proceedings of the 2023 international conference on autonomous agents and multiagent systems. AAMAS. ACM, pp 2514\u20132516, https:\/\/doi.org\/10.5555\/3545946.3598986","DOI":"10.5555\/3545946.3598986"},{"key":"10706_CR65","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-12423-5_16","author":"C Ounoughi","year":"2022","unstructured":"Ounoughi C, Touibi G, Yahia S (2022) EcoLight: eco-friendly traffic signal control driven by urban noise prediction. Lecture Notes Comput Sci. https:\/\/doi.org\/10.1007\/978-3-031-12423-5_16","journal-title":"Lecture Notes Comput Sci"},{"key":"10706_CR66","unstructured":"Panagopoulos AA, Alam M, Rogers A, et\u00a0al (2015) AdaHeat: a general adaptive intelligent agent for domestic heating control. In: Proceedings of the 2015 international conference on autonomous agents and multiagent systems, AAMAS. ACM, pp 1295\u20131303"},{"issue":"4","key":"10706_CR67","doi-asserted-by":"publisher","first-page":"1178","DOI":"10.1016\/j.joi.2016.10.006","volume":"10","author":"A Perianes-Rodriguez","year":"2016","unstructured":"Perianes-Rodriguez A, Waltman L, van Eck NJ (2016) Constructing bibliometric networks: a comparison between full and fractional counting. J Info 10(4):1178\u20131195. https:\/\/doi.org\/10.1016\/j.joi.2016.10.006","journal-title":"J Info"},{"key":"10706_CR68","doi-asserted-by":"publisher","DOI":"10.1002\/9780470316887","volume-title":"Markov decision processes: discrete stochastic dynamic programming","author":"ML Puterman","year":"1994","unstructured":"Puterman ML (1994) Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, Hoboken"},{"key":"10706_CR69","doi-asserted-by":"publisher","first-page":"117268","DOI":"10.1016\/j.apenergy.2021.117268","volume":"298","author":"S Radini","year":"2021","unstructured":"Radini S, Marinelli E, Akyol \u00c7a\u011fr\u0131 et al (2021) Urban water-energy-food-climate nexus in integrated wastewater and reuse systems: cyber-physical framework and innovations. Appl Energy 298:117268","journal-title":"Appl Energy"},{"key":"10706_CR70","doi-asserted-by":"publisher","DOI":"10.36680\/j.itcon.2022.043","author":"L Rampini","year":"2022","unstructured":"Rampini L, Re Cecconi F (2022) Artificial intelligence in construction asset management: A review of present status, challenges and future opportunities. J Info Technol Construct. https:\/\/doi.org\/10.36680\/j.itcon.2022.043","journal-title":"J Info Technol Construct"},{"key":"10706_CR71","doi-asserted-by":"publisher","DOI":"10.1016\/j.cherd.2021.08.013","author":"D Rangel-Martinez","year":"2021","unstructured":"Rangel-Martinez D, Nigam K, Ricardez-Sandoval L (2021) Machine learning on sustainable energy: a review and outlook on renewable energy systems, catalysis, smart grid and energy storage. Chem Eng Res Design. https:\/\/doi.org\/10.1016\/j.cherd.2021.08.013","journal-title":"Chem Eng Res Design"},{"issue":"5","key":"10706_CR72","doi-asserted-by":"publisher","first-page":"1654","DOI":"10.1007\/s11036-017-0893-y","volume":"24","author":"M Roncalli","year":"2019","unstructured":"Roncalli M, Bistaffa F, Farinelli A (2019) Decentralized power distribution in the smart grid with ancillary lines. Mobile Netw Appl 24(5):1654\u20131662. https:\/\/doi.org\/10.1007\/s11036-017-0893-y","journal-title":"Mobile Netw Appl"},{"issue":"6088","key":"10706_CR73","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1038\/323533a0","volume":"323","author":"DE Rumelhart","year":"1986","unstructured":"Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533\u2013536. https:\/\/doi.org\/10.1038\/323533a0","journal-title":"Nature"},{"key":"10706_CR74","doi-asserted-by":"publisher","first-page":"101622","DOI":"10.1109\/ACCESS.2022.3208899","volume":"10","author":"S Sabet","year":"2022","unstructured":"Sabet S, Farooq B (2022) Green vehicle routing problem: State of the art and future directions. IEEE Access 10:101622\u2013101642. https:\/\/doi.org\/10.1109\/ACCESS.2022.3208899","journal-title":"IEEE Access"},{"key":"10706_CR75","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2021.3074304","author":"A Sacco","year":"2021","unstructured":"Sacco A, Esposito F, Marchetto G et al (2021) Sustainable task offloading in UAV networks via multi-agent reinforcement learning. IEEE Trans Vehicul Technol. https:\/\/doi.org\/10.1109\/TVT.2021.3074304","journal-title":"IEEE Trans Vehicul Technol"},{"key":"10706_CR76","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2021.101722","author":"R Shaw","year":"2022","unstructured":"Shaw R, Howley E, Barrett E (2022) Applying reinforcement learning towards automating energy efficient virtual machine consolidation in cloud data centers. Info Syst. https:\/\/doi.org\/10.1016\/j.is.2021.101722","journal-title":"Info Syst"},{"key":"10706_CR77","doi-asserted-by":"publisher","DOI":"10.1016\/j.scs.2016.04.001","author":"A Sheikhi","year":"2016","unstructured":"Sheikhi A, Rayati M, Ranjbar A (2016) Dynamic load management for a residential customer; reinforcement learning approach. Sustain Cities Soc. https:\/\/doi.org\/10.1016\/j.scs.2016.04.001","journal-title":"Sustain Cities Soc"},{"key":"10706_CR78","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961","author":"D Silver","year":"2016","unstructured":"Silver D, Huang A, Maddison CJ et al (2016) Mastering the game of Go with deep neural networks and tree search. Nature. https:\/\/doi.org\/10.1038\/nature16961","journal-title":"Nature"},{"key":"10706_CR79","doi-asserted-by":"publisher","DOI":"10.1038\/nature24270","author":"D Silver","year":"2017","unstructured":"Silver D, Schrittwieser J, Simonyan K et al (2017) Mastering the game of go without human knowledge. Nature. https:\/\/doi.org\/10.1038\/nature24270","journal-title":"Nature"},{"issue":"12","key":"10706_CR80","doi-asserted-by":"publisher","first-page":"15109","DOI":"10.1609\/aaai.v37i12.26763","volume":"37","author":"TD Sim\u00e3o","year":"2023","unstructured":"Sim\u00e3o TD, Suilen M, Jansen N (2023) Safe policy improvement for POMDPs via finite-state controllers. Proc AAAI Conf Artif Intell 37(12):15109\u201315117. https:\/\/doi.org\/10.1609\/aaai.v37i12.26763","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"10706_CR81","doi-asserted-by":"publisher","DOI":"10.3390\/en16031512","author":"K Sivamayil","year":"2023","unstructured":"Sivamayil K, Rajasekar E, Aljafari B et al (2023) A systematic study on reinforcement learning based applications. Energies. https:\/\/doi.org\/10.3390\/en16031512","journal-title":"Energies"},{"key":"10706_CR82","doi-asserted-by":"publisher","DOI":"10.1016\/j.jhydrol.2020.124757","author":"M Skardi","year":"2020","unstructured":"Skardi M, Kerachian R, Abdolhay A (2020) Water and treated wastewater allocation in urban areas considering social attachments. J Hydrol. https:\/\/doi.org\/10.1016\/j.jhydrol.2020.124757","journal-title":"J Hydrol"},{"key":"10706_CR83","doi-asserted-by":"publisher","first-page":"103346","DOI":"10.1016\/j.robot.2019.103346","volume":"124","author":"L Steccanella","year":"2020","unstructured":"Steccanella L, Bloisi D, Castellini A et al (2020) Waterline and obstacle detection in images from low-cost autonomous boats for environmental monitoring. Robot Auton Syst 124:103346","journal-title":"Robot Auton Syst"},{"key":"10706_CR84","doi-asserted-by":"publisher","DOI":"10.1016\/j.est.2023.106987","author":"S Sultanuddin","year":"2023","unstructured":"Sultanuddin S, Vibin R, Rajesh Kumar A et al (2023) Development of improved reinforcement learning smart charging strategy for electric vehicle fleet. J Energy Storage. https:\/\/doi.org\/10.1016\/j.est.2023.106987","journal-title":"J Energy Storage"},{"key":"10706_CR85","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. A Bradford Book, Denver"},{"key":"10706_CR86","unstructured":"United Nations (2015) Transforming our world: the 2030 agenda for sustainable development"},{"key":"10706_CR87","doi-asserted-by":"publisher","unstructured":"van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double Q-earning. In: Proceedings of the AAAI conference on artificial intelligence, pp 2094\u20132100, https:\/\/doi.org\/10.1609\/aaai.v30i1.10295","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"10706_CR88","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-22698-4\\_6","author":"V Venkataswamy","year":"2023","unstructured":"Venkataswamy V, Grigsby J, Grimshaw A et al (2023) RARE: renewable energy aware resource management in datacenters. Lecture Notes Comput Sci. https:\/\/doi.org\/10.1007\/978-3-031-22698-4\\_6","journal-title":"Lecture Notes Comput Sci"},{"key":"10706_CR89","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2021.3106168","author":"JJ Wang","year":"2022","unstructured":"Wang JJ, Wang L (2022) A cooperative memetic algorithm with learning-based agent for energy-aware distributed hybrid flow-shop scheduling. IEEE Trans Evol Comput. https:\/\/doi.org\/10.1109\/TEVC.2021.3106168","journal-title":"IEEE Trans Evol Comput"},{"key":"10706_CR90","volume-title":"Learning from delayed rewards","author":"CJCH Watkins","year":"1989","unstructured":"Watkins CJCH (1989) Learning from delayed rewards. King\u2019s College, Oxford"},{"key":"10706_CR91","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1016\/j.arcontrol.2020.03.001","volume":"49","author":"T Yang","year":"2020","unstructured":"Yang T, Zhao L, Li W et al (2020) Reinforcement learning in sustainable energy and electric systems: a survey. Ann Rev Control 49:145\u2013163. https:\/\/doi.org\/10.1016\/j.arcontrol.2020.03.001","journal-title":"Ann Rev Control"},{"key":"10706_CR92","doi-asserted-by":"publisher","DOI":"10.3390\/en16052456","author":"R Yao","year":"2023","unstructured":"Yao R, Hu Y, Varga L (2023) Applications of agent-based methods in multi-energy systems\u2014a systematic literature review. Energies. https:\/\/doi.org\/10.3390\/en16052456","journal-title":"Energies"},{"key":"10706_CR93","doi-asserted-by":"publisher","unstructured":"Zhang W, Liu H, Wang F, et\u00a0al (2021a) Intelligent electric vehicle charging recommendation based on multi-agent reinforcement learning. In: Proceedings of the web conference, WWW, p 1856\u20131867, https:\/\/doi.org\/10.1145\/3442381.3449934","DOI":"10.1145\/3442381.3449934"},{"key":"10706_CR94","doi-asserted-by":"publisher","DOI":"10.1016\/j.seta.2021.101208","author":"X Zhang","year":"2021","unstructured":"Zhang X, Manogaran G, Muthu B (2021b) IoT enabled integrated system for green energy into smart cities. Sustain Energy Technol Assess. https:\/\/doi.org\/10.1016\/j.seta.2021.101208","journal-title":"Sustain Energy Technol Assess"},{"key":"10706_CR95","doi-asserted-by":"crossref","unstructured":"Zuccotto M, Castellini A, Farinelli A (2022a) Learning state-variable relationships for improving POMCP performance. In: Proceedings of the 37th ACM\/SIGAPP symposium on applied computing. Association for Computing Machinery, SAC, p 739\u2013747","DOI":"10.1145\/3477314.3507049"},{"key":"10706_CR96","first-page":"183","volume":"2022","author":"M Zuccotto","year":"2022","unstructured":"Zuccotto M, Piccinelli M, Castellini A et al (2022b) Learning state-variable relationships in POMCP: a framework for mobile robots. Front Robotics AI 2022:183","journal-title":"Front Robotics AI"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-10706-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10462-024-10706-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-10706-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,13]],"date-time":"2024-04-13T03:10:26Z","timestamp":1712977826000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10462-024-10706-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,12]]},"references-count":96,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["10706"],"URL":"https:\/\/doi.org\/10.1007\/s10462-024-10706-5","relation":{},"ISSN":["1573-7462"],"issn-type":[{"value":"1573-7462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,12]]},"assertion":[{"value":"12 March 2024","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"88"}}