{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T04:45:49Z","timestamp":1775709949082,"version":"3.50.1"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"S4","license":[{"start":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T00:00:00Z","timestamp":1671580800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T00:00:00Z","timestamp":1671580800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Energy Inform"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In the past decade, the global distribution of energy resources has expanded significantly. The increasing number of prosumers creates the prospect for a more decentralized and accessible energy market, where the peer-to-peer energy trading paradigm emerges. This paper proposes a methodology to optimize the participation in peer-to-peer markets based on the double-auction trading mechanism. This novel methodology is based on two reinforcement learning algorithms, used separately, to optimize the amount of energy to be transacted and the price to pay\/charge for the purchase\/sale of energy. The proposed methodology uses a competitive approach, and that is why all agents seek the best result for themselves, which in this case means reducing as much as possible the costs related to the purchase of energy, or if we are talking about sellers, maximizing profits. The proposed methodology was integrated into an agent-based ecosystem where there is a direct connection with agents, thus allowing application to real contexts in a more efficient way. To test the methodology, a case study was carried out in an energy community of 50 players, where each of the proposed models were used in 20 different players, and 10 were left without training. The players with training managed, over the course of a week, to save 44.65 EUR when compared to a week of peer-to-peer without training, a positive result, while the players who were left without training increased costs by 17.07 EUR.<\/jats:p>","DOI":"10.1186\/s42162-022-00235-2","type":"journal-article","created":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T00:17:28Z","timestamp":1671581848000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Peer-to-peer energy trading optimization in energy communities using multi-agent deep reinforcement learning"],"prefix":"10.1186","volume":"5","author":[{"given":"Helder","family":"Pereira","sequence":"first","affiliation":[]},{"given":"Luis","family":"Gomes","sequence":"additional","affiliation":[]},{"given":"Zita","family":"Vale","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,12,21]]},"reference":[{"key":"235_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.esr.2019.100418","volume":"26","author":"O Abrishambaf","year":"2019","unstructured":"Abrishambaf O, Lezama F, Faria P, Vale Z (2019) Towards transactive energy systems: an analysis on current trends. Energy Strateg Rev 26:100418","journal-title":"Energy Strateg Rev"},{"key":"235_CR2","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","volume":"34","author":"K Arulkumaran","year":"2017","unstructured":"Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34:26\u201338","journal-title":"IEEE Signal Process Mag"},{"key":"235_CR3","doi-asserted-by":"publisher","first-page":"408","DOI":"10.1016\/j.tics.2019.02.006","volume":"23","author":"M Botvinick","year":"2019","unstructured":"Botvinick M, Ritter S, Wang JX, Kurth-Nelson Z, Blundell C, Hassabis D (2019) Reinforcement learning, fast and slow. Trends Cogn Sci 23:408\u2013422","journal-title":"Trends Cogn Sci"},{"key":"235_CR4","unstructured":"Brockman G, Cheung V, Pettersson L, Schneider J, Schulman J, Tang J, Zaremba W (2016) OpenAI Gym."},{"key":"235_CR5","doi-asserted-by":"publisher","unstructured":"Chen T, Bu S (2019) Realistic peer-to-peer energy trading model for microgrids using deep reinforcement learning. Proc 2019 IEEE PES Innov Smart Grid Technol Eur ISGT-Europe 2019. https:\/\/doi.org\/10.1109\/ISGTEUROPE.2019.8905731","DOI":"10.1109\/ISGTEUROPE.2019.8905731"},{"key":"235_CR6","doi-asserted-by":"publisher","DOI":"10.1016\/j.scitotenv.2021.148445","volume":"792","author":"YC Chen","year":"2021","unstructured":"Chen YC, Liu HM (2021) Evaluation of greenhouse gas emissions and the feed-in tariff system of waste-to-energy facilities using a system dynamics model. Sci Total Environ 792:148445","journal-title":"Sci Total Environ"},{"key":"235_CR7","doi-asserted-by":"publisher","first-page":"715","DOI":"10.1109\/TSG.2021.3124465","volume":"13","author":"T Chen","year":"2022","unstructured":"Chen T, Bu S, Liu X, Kang J, Yu FR, Han Z (2022) Peer-to-peer energy trading and energy conversion in interconnected multi-energy microgrids using multi-agent deep reinforcement learning. IEEE Trans Smart Grid 13:715\u2013727","journal-title":"IEEE Trans Smart Grid"},{"key":"235_CR8","doi-asserted-by":"crossref","unstructured":"Chicco G, Somma M Di, Graditi G (2021) Overview of distributed energy resources in the context of local integrated energy systems. Distrib Energy Resour Local Integr Energy Syst Optim Oper Plan 1\u201329","DOI":"10.1016\/B978-0-12-823899-8.00002-9"},{"key":"235_CR9","doi-asserted-by":"publisher","first-page":"985","DOI":"10.1109\/JSYST.2021.3059000","volume":"16","author":"WY Chiu","year":"2022","unstructured":"Chiu WY, Hu CW, Chiu KY (2022) Renewable energy bidding strategies using multiagent Q-learning in double-sided auctions. IEEE Syst J 16:985\u2013996","journal-title":"IEEE Syst J"},{"key":"235_CR10","doi-asserted-by":"publisher","DOI":"10.1016\/j.esr.2021.100678","volume":"36","author":"JD de S\u00e3o","year":"2021","unstructured":"de S\u00e3o JD, Faria P, Vale Z (2021) Smart energy community: a systematic review with metanalysis. Energy Strateg Rev 36:100678","journal-title":"Energy Strateg Rev"},{"key":"235_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.apenergy.2021.117434","volume":"301","author":"V Dudjak","year":"2021","unstructured":"Dudjak V, Neves D, Alskaif T et al (2021) Impact of local energy markets integration in power systems layer: a comprehensive review. Appl Energy 301:117434","journal-title":"Appl Energy"},{"key":"235_CR12","doi-asserted-by":"publisher","DOI":"10.4324\/9780429492532","volume-title":"The double auction market: institutions, theories, and evidence","author":"D Friedman","year":"2018","unstructured":"Friedman D (2018) The double auction market: institutions, theories, and evidence. Routledge"},{"key":"235_CR13","unstructured":"Fujimoto S, Van Hoof H, Meger D (2018) Addressing function approximation error in actor-critic methods. 35th Int Conf Mach Learn ICML 2018 4:2587\u20132601"},{"key":"235_CR14","doi-asserted-by":"publisher","first-page":"3543","DOI":"10.3390\/en15103543","volume":"15","author":"L Gomes","year":"2022","unstructured":"Gomes L, Morais H, Gon\u00e7alves C, Gomes E, Pereira L, Vale Z (2022) Impact of forecasting models errors in a peer-to-peer energy sharing market. Energies 15:3543","journal-title":"Energies"},{"key":"235_CR15","doi-asserted-by":"crossref","unstructured":"Goncalves C, Barreto R, Faria P, Gomes L, Vale Z (2022) Dataset of an energy community\u2019s consumption and generation with appliance allocation for one year. https:\/\/doi.org\/10.5281\/ZENODO.6778401","DOI":"10.1016\/j.dib.2022.108590"},{"key":"235_CR16","doi-asserted-by":"publisher","first-page":"895","DOI":"10.1007\/s10462-021-09996-w","volume":"55","author":"S Gronauer","year":"2021","unstructured":"Gronauer S, Diepold K (2021) Multi-agent deep reinforcement learning: a survey. Artif Intell Rev 55:895\u2013943","journal-title":"Artif Intell Rev"},{"key":"235_CR17","doi-asserted-by":"publisher","DOI":"10.1016\/j.rser.2021.111859","volume":"154","author":"M Gr\u017eani\u0107","year":"2022","unstructured":"Gr\u017eani\u0107 M, Capuder T, Zhang N, Huang W (2022) Prosumers as active market participants: a systematic review of evolution of opportunities, models and challenges. Renew Sustain Energy Rev 154:111859","journal-title":"Renew Sustain Energy Rev"},{"key":"235_CR18","unstructured":"Liang E, Liaw R, Moritz P, Nishihara R, Fox R, Goldberg K, Gonzalez JE, Jordan MI, Stoica I (2017) RLlib: abstractions for distributed reinforcement learning. 35th Int Conf Mach Learn ICML 2018 7:4768\u20134780"},{"key":"235_CR19","doi-asserted-by":"publisher","DOI":"10.1186\/s42162-021-00151-x","author":"B Mota","year":"2021","unstructured":"Mota B, Albergaria M, Pereira H, Silva J, Gomes L, Vale Z, Ramos C (2021) Climatization and luminosity optimization of buildings using genetic algorithm, random forest, and regression models. Energy Inform. https:\/\/doi.org\/10.1186\/s42162-021-00151-x","journal-title":"Energy Inform"},{"key":"235_CR20","doi-asserted-by":"publisher","first-page":"3590","DOI":"10.1007\/s10489-020-01758-5","volume":"50","author":"S Padakandla","year":"2020","unstructured":"Padakandla S, Bhatnagar KJP (2020) Reinforcement learning algorithm for non-stationary environments. Appl Intell 50:3590\u20133606","journal-title":"Appl Intell"},{"key":"235_CR21","doi-asserted-by":"publisher","first-page":"182537","DOI":"10.1109\/ACCESS.2020.3027357","volume":"8","author":"J Palanca","year":"2020","unstructured":"Palanca J, Terrasa A, Julian V, Carrascosa C (2020) Spade 3: supporting the new generation of multi-agent systems. IEEE Access 8:182537\u2013182549","journal-title":"IEEE Access"},{"key":"235_CR22","doi-asserted-by":"publisher","DOI":"10.1186\/s42162-021-00155-7","author":"H Pereira","year":"2021","unstructured":"Pereira H, Gomes L, Faria P, Vale Z, Coelho C (2021) Web-based platform for the management of citizen energy communities and their members. Energy Inform. https:\/\/doi.org\/10.1186\/s42162-021-00155-7","journal-title":"Energy Inform"},{"key":"235_CR23","first-page":"2913","volume":"3","author":"D Qiu","year":"2021","unstructured":"Qiu D, Wang J, Wang J, Strbac G (2021a) Multi-agent reinforcement learning for automated peer-to-peer energy trading in double-side auction market. IJCAI Int Jt Conf Artif Intell 3:2913\u20132920","journal-title":"IJCAI Int Jt Conf Artif Intell"},{"key":"235_CR24","doi-asserted-by":"publisher","DOI":"10.1016\/j.apenergy.2021.116940","volume":"292","author":"D Qiu","year":"2021","unstructured":"Qiu D, Ye Y, Papadaskalopoulos D, Strbac G (2021b) Scalable coordinated management of peer-to-peer energy trading: a multi-cluster deep reinforcement learning approach. Appl Energy 292:116940","journal-title":"Appl Energy"},{"key":"235_CR25","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1146\/annurev-control-053018-023825","volume":"2","author":"B Recht","year":"2019","unstructured":"Recht B (2019) A tour of reinforcement learning: the view from continuous control. Annu Rev Control Robot Auton Syst 2:253\u2013279","journal-title":"Annu Rev Control Robot Auton Syst"},{"key":"235_CR26","doi-asserted-by":"publisher","DOI":"10.1016\/j.rser.2021.111013","volume":"144","author":"FGI Reis","year":"2021","unstructured":"Reis FGI, Gon\u00e7alves I, Lopes ARM, Henggeler Antunes C (2021) Business models for energy communities: a review of key issues and trends. Renew Sustain Energy Rev 144:111013","journal-title":"Renew Sustain Energy Rev"},{"key":"235_CR27","doi-asserted-by":"publisher","DOI":"10.1016\/j.apenergy.2022.119123","volume":"317","author":"C Samende","year":"2022","unstructured":"Samende C, Cao J, Fan Z (2022) Multi-agent deep deterministic policy gradient algorithm for peer-to-peer energy trading considering distribution network constraints. Appl Energy 317:119123","journal-title":"Appl Energy"},{"key":"235_CR28","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1016\/j.energy.2017.10.068","volume":"142","author":"V Venizelou","year":"2018","unstructured":"Venizelou V, Philippou N, Hadjipanayi M, Makrides G, Efthymiou V, Georghiou GE (2018) Development of a novel time-of-use tariff algorithm for residential prosumer price-based demand side management. Energy 142:633\u2013646","journal-title":"Energy"},{"key":"235_CR29","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijepes.2020.106593","volume":"126","author":"Y Wu","year":"2021","unstructured":"Wu Y, Wu Y, Guerrero JM, Vasquez JC (2021) Digitalization and decentralization driving transactive energy Internet: Key technologies and infrastructures. Int J Electr Power Energy Syst 126:106593","journal-title":"Int J Electr Power Energy Syst"}],"container-title":["Energy Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s42162-022-00235-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s42162-022-00235-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s42162-022-00235-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T00:27:55Z","timestamp":1671582475000},"score":1,"resource":{"primary":{"URL":"https:\/\/energyinformatics.springeropen.com\/articles\/10.1186\/s42162-022-00235-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,21]]},"references-count":29,"journal-issue":{"issue":"S4","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["235"],"URL":"https:\/\/doi.org\/10.1186\/s42162-022-00235-2","relation":{},"ISSN":["2520-8942"],"issn-type":[{"value":"2520-8942","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,21]]},"assertion":[{"value":"11 October 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 December 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"44"}}