{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T14:53:48Z","timestamp":1767192828103,"version":"build-2065373602"},"reference-count":55,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,30]],"date-time":"2023-01-30T00:00:00Z","timestamp":1675036800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010880","name":"State Grid Corporation of China","doi-asserted-by":"publisher","award":["5108-202155042A-0-0-00"],"award-info":[{"award-number":["5108-202155042A-0-0-00"]}],"id":[{"id":"10.13039\/501100010880","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Axioms"],"abstract":"<jats:p>Power retail companies in the electricity market make profits through buying and selling power energy in the wholesale and retail markets, respectively. Traditionally, they are assumed to bid in the wholesale market with the same objective, i.e., maximize the profit. This paper proposes a multiagent reinforcement learning (MRL)-based model to simulate the diverse bidding decision-making concerning various operation objectives and the profit-sharing modes of power retail companies in China\u2019s wholesale electricity market, which contributes to a more realistic modeling and simulation of the retail companies. Specifically, three types of operation objectives and five types of profit-sharing modes are mathematically formulated. After that, a complete electricity market optimization model is established, and a case study with 30 retail companies is carried out. The simulation results show that the proposed method can effectively model the diverse bidding decision-making of the power retail companies, which can further assist their decision-making and further contribute to the analysis and simulations of the electricity market.<\/jats:p>","DOI":"10.3390\/axioms12020142","type":"journal-article","created":{"date-parts":[[2023,1,30]],"date-time":"2023-01-30T07:34:41Z","timestamp":1675064081000},"page":"142","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["MRL-Based Model for Diverse Bidding Decision-Makings of Power Retail Company in the Wholesale Electricity Market of China"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9498-6426","authenticated-orcid":false,"given":"Ying","family":"Wang","sequence":"first","affiliation":[{"name":"Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing 210096, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chang","family":"Liu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing 210096, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weihong","family":"Yuan","sequence":"additional","affiliation":[{"name":"Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing 210096, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lili","family":"Li","sequence":"additional","affiliation":[{"name":"NARI Group Corporation (State Grid Electric Power Research Institute), Nanjing 211106, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"111717","DOI":"10.1016\/j.enpol.2020.111717","article-title":"Power market reform in China: Motivations, progress, and recommendations","volume":"145","author":"Guo","year":"2020","journal-title":"Energy Policy"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"105003","DOI":"10.1016\/j.eneco.2020.105003","article-title":"In between the state and the market: An empirical assessment of the early achievements of China's 2015 electricity reform","volume":"93","author":"Zheng","year":"2021","journal-title":"Energy Econ."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Jain, K., Saxena, A., Alshamrani, A.M., Alrasheedi, A.F., Alnowibet, K.A., and Mohamed, A.W. (2022). An amended whale optimization algorithm for optimal bidding in day ahead electricity market. Axioms, 11.","DOI":"10.3390\/axioms11090456"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.enpol.2014.03.023","article-title":"The value of information in explicit crossborder capacity auction regimes in electricity markets","volume":"70","author":"Richter","year":"2014","journal-title":"Energy Policy"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1049\/iet-gtd.2019.0493","article-title":"Optimal dispatch and bidding strategy of a virtual power plant based on a Stackelberg game","volume":"14","author":"Wu","year":"2020","journal-title":"IET Gener. Transm. Distrib."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"630","DOI":"10.1016\/j.epsr.2006.05.012","article-title":"Supplier bidding strategy based on non-cooperative game theory concepts in single auction power pools","volume":"77","author":"Kang","year":"2007","journal-title":"Electric Power Syst. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1109\/TPWRS.2009.2036824","article-title":"Bid Cost Minimization Versus Payment Cost Minimization: A Game Theoretic Study of Electricity Auctions","volume":"25","author":"Zhao","year":"2010","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"100279","DOI":"10.1016\/j.jik.2022.100279","article-title":"A game theoretic technique for risk-based optimal bidding strategies in energy aggregators of markets: Knowledge management approach","volume":"7","author":"Wang","year":"2022","journal-title":"J. Innov. Knowl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"123205","DOI":"10.1016\/j.energy.2022.123205","article-title":"Coordinated optimal bidding strategies methods of aggregated microgrids: A game theory-based demand side management under an electricity market environment","volume":"245","author":"Saeian","year":"2022","journal-title":"Energy"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1016\/j.energy.2017.12.122","article-title":"Cooperative game of electricity retailers in China's spot electricity market","volume":"145","author":"Peng","year":"2018","journal-title":"Energy"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3440","DOI":"10.1109\/TPWRS.2013.2252633","article-title":"Pool strategy of a price-maker wind power producer","volume":"28","author":"Zugno","year":"2013","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1109\/TSTE.2021.3110864","article-title":"Bi-level strategic bidding model of gas-fired units in interdependent electricity and natural gas markets","volume":"13","author":"Jiang","year":"2022","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Boyd, S., Boyd, S.P., and Vandenberghe, L. (2004). Convex Optimization, Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1000","DOI":"10.1016\/j.apenergy.2017.03.121","article-title":"Competition, risk and learning in electricity markets: An agent-based simulation study","volume":"195","author":"Kaya","year":"2017","journal-title":"Appl. Energy"},{"key":"ref_15","first-page":"368","article-title":"Agent-based modeling and simulation of the electricity market with residential demand response","volume":"7","author":"Xu","year":"2020","journal-title":"CSEE J. Power Energy Syst."},{"key":"ref_16","unstructured":"North, M., Macal, C., Conzelmann, G., Koritarov, V., Thimmapuram, P., and Veselka, T. (2022, December 20). Multi-Agent Electricity Market Modeling with EMCAS (No. ANL\/DIS\/CP-108599), Available online: https:\/\/www.osti.gov\/servlets\/purl\/801575."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Samaan, N.A. (2021). Deploying Intra-Hour Uncertainty Analysis Tools to ABB\u2019s GridView-CRADA 445 (No. PNNL-SA-160084), Pacific Northwest National Laboratory (PNNL).","DOI":"10.2172\/1827780"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1007\/s10614-007-9095-1","article-title":"Dynamic testing of wholesale electricity market designs: An open-source agent-based framework","volume":"30","author":"Sun","year":"2007","journal-title":"Comput. Econ."},{"key":"ref_19","unstructured":"Thomas, R.J., Mount, T.D., and Zimmerman, R.D. (2004, January 10\u201313). PowerWeb testing of various auction clearing mechanisms for electricity. Proceedings of the IEEE PES Power Systems Conference and Exposition, New York, NY, USA."},{"key":"ref_20","first-page":"72","article-title":"Intelligent Agent Based Bidding Simulation Method for Multi input Decision Factors of Power Suppliers","volume":"42","author":"Feng","year":"2018","journal-title":"Autom. Electr. Power Syst."},{"key":"ref_21","first-page":"848","article-title":"Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria","volume":"88","author":"Erev","year":"1998","journal-title":"Am. Econ. Rev."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"106762","DOI":"10.1016\/j.asoc.2020.106762","article-title":"New interactive agent based reinforcement learning approach towards smart generator bidding in electricity market with micro grid integration","volume":"97","author":"Kiran","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"810","DOI":"10.1007\/s40565-017-0270-7","article-title":"Agent-based simulation for symmetric electricity market considering price-based demand response","volume":"5","author":"Jiang","year":"2017","journal-title":"J. Mod. Power Syst. Clean Energy"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1080\/15325008.2020.1758840","article-title":"Roth-Erev reinforcement learning approach for smart generator bidding towards long term electricity market operation using agent based dynamic modeling","volume":"48","author":"Purushothaman","year":"2020","journal-title":"Electr. Power Compon. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/TSG.2011.2168244","article-title":"Agent-based electricity market simulation with demand response from commercial buildings","volume":"2","author":"Zhou","year":"2011","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"04022030","DOI":"10.1061\/(ASCE)EY.1943-7897.0000854","article-title":"Studying dynamic pricing in electrical power markets with distributed generation: Agent-based modeling and reinforcement-learning approach","volume":"148","author":"Ali","year":"2022","journal-title":"J. Energy Eng."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3756","DOI":"10.1049\/iet-gtd.2016.0075","article-title":"Markov game approach for multi-agent competitive bidding strategies in electricity market","volume":"10","author":"Rashedi","year":"2016","journal-title":"IET Gener. Transm. Distrib."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"106474","DOI":"10.1016\/j.epsr.2020.106474","article-title":"Distributed generation planning from the investor's viewpoint considering pool-based electricity markets","volume":"187","author":"Farshad","year":"2020","journal-title":"Electr. Power Syst. Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"107415","DOI":"10.1016\/j.ijepes.2021.107415","article-title":"Agent based simulation of centralized electricity transaction market using bi-level and Q-learning algorithm approach","volume":"134","author":"Namalomba","year":"2022","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"4180","DOI":"10.1109\/TPWRS.2020.2999536","article-title":"Agent-based modeling in electricity market using deep deterministic policy gradient algorithm","volume":"35","author":"Liang","year":"2020","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1343","DOI":"10.1109\/TSG.2019.2936142","article-title":"Deep reinforcement learning for strategic bidding in electricity markets","volume":"11","author":"Ye","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"119813","DOI":"10.1016\/j.apenergy.2022.119813","article-title":"The emerging threat of artificial intelligence on competition in liberalized electricity markets: A deep Q-network approach","volume":"325","author":"Chan","year":"2022","journal-title":"Appl. Energy"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"124657","DOI":"10.1016\/j.energy.2022.124657","article-title":"Strategic bidding in a competitive electricity market: An intelligent method using multi-agent transfer learning based on reinforcement learning","volume":"256","author":"Wu","year":"2022","journal-title":"Energy"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1125","DOI":"10.1007\/s11630-020-1308-0","article-title":"A deep reinforcement learning bidding algorithm on Electricity Market","volume":"29","author":"Jia","year":"2020","journal-title":"J. Therm. Sci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"534","DOI":"10.35833\/MPCE.2020.000502","article-title":"Approximating Nash equilibrium in day-ahead electricity market bidding with multi-agent deep reinforcement learning","volume":"9","author":"Du","year":"2021","journal-title":"J. Mod. Power Syst. Clean Energy"},{"key":"ref_37","unstructured":"Guangdong Power Exchange Center CO, LTD (2023, January 25). Annual Report of Guangdong Electricity Market in 2021. Available online: https:\/\/pm.gd.csg.cn\/views\/page\/downfile\/infoAdmin\/info\/20220208171001\/%E5%B9%BF%E4%B8%9C%E7%94%B5%E5%8A%9B%E5%B8%82%E5%9C%BA2021%E5%B9%B4%E5%B9%B4%E5%BA%A6%E6%8A%A5%E5%91%8A20220208171419.pdf."},{"key":"ref_38","first-page":"181","article-title":"Analysis of power purchase-sale strategy of power retail companies with different establishment qualifications","volume":"40","author":"Dou","year":"2020","journal-title":"Proc. CSEE"},{"key":"ref_39","first-page":"147","article-title":"Research on influencing factors of profitability of electricity sales companies under spread rebate mode","volume":"3","author":"Jiang","year":"2020","journal-title":"Price Theory Pract."},{"key":"ref_40","unstructured":"China\u2019s National Energy Administration (2023, January 20). Say Goodbye to Disorderly Competition Power Market Calls for an Era of Power Sales Specialization, Available online: http:\/\/www.nea.gov.cn\/2017-11\/03\/c_136725792.htm."},{"key":"ref_41","unstructured":"China Weekly (2023, January 20). Chaos of Electricity Retail Reform. Available online: http:\/\/www.inewsweek.cn\/finance\/2021-08-21\/13546.shtml."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"100055","DOI":"10.1016\/j.egycc.2021.100055","article-title":"Lessons learned from power industry reform and electricity spot market trial operation in Southern China","volume":"2","author":"Wang","year":"2021","journal-title":"Energy Clim. Change"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Lu, T., Zhang, W., Wang, Y., Xie, H., and Ding, X. (2022). Medium- and long-term trading strategies for large electricity retailers in China\u2019s electricity market. Energies, 15.","DOI":"10.3390\/en15093342"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2176","DOI":"10.1109\/TSG.2020.3027728","article-title":"A Reinforcement Learning-Based Decision System for Electricity Pricing Plan Selection by Smart Grid End Users","volume":"12","author":"Lu","year":"2021","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"2059","DOI":"10.1109\/TPWRS.2019.2955479","article-title":"Optimal Alliance Strategies Among Retailers Under Energy Deviation Settlement Mechanism in China's Forward Electricity Market","volume":"35","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"137063","DOI":"10.1109\/ACCESS.2020.3011594","article-title":"Prospect theory-based optimal bidding model of a prosumer in the power market","volume":"8","author":"Wang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_47","unstructured":"(2023, January 22). AEMO Energy Offers (Public). Available online: https:\/\/www.aemo.com.au\/energy-systems\/electricity\/national-electricity-market-nem\/data-nem\/market-data-nemweb."},{"key":"ref_48","unstructured":"(2022, December 15). PJM Manual 11: Energy & Ancillary Services Market Operations. Available online: http:\/\/www.pjm.com\/~\/media\/documents\/manuals\/m11.ashx."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1990","DOI":"10.1109\/TPWRS.2004.836192","article-title":"Short-term electricity market auction game analysis: Uniform and pay-as-bid pricing","volume":"19","author":"Son","year":"2004","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.tej.2006.11.011","article-title":"Why we need to stick with uniform-price auctions in electricity markets","volume":"20","author":"Cramton","year":"2007","journal-title":"Electr. J."},{"key":"ref_51","unstructured":"Independent System Operator New England (2022, December 15). The Benefits of Uniform Clearing-Price Auctions for Pricing Electricity: Why Pay-as-Bid Auctions Do Not Cost Less. ISONE Report 2006. Available online: https:\/\/www.iso-ne.com\/static-assets\/documents\/committees\/comm_wkgrps\/othr\/clg\/mtrls\/2009\/jul292009\/uniform_clearing_price_auctions_whitepaper.pdf."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"112482","DOI":"10.1016\/j.enpol.2021.112482","article-title":"Equilibrium strategy based policy shifts towards the integration of wind power in spot electricity markets: A perspective from China","volume":"157","author":"Liu","year":"2021","journal-title":"Energy Policy"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"120746","DOI":"10.1109\/ACCESS.2020.3005932","article-title":"Optimal bidding of price-maker retailers with demand price quota curves under price uncertainty","volume":"8","author":"Wang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Wang, Y., Tian, L., Xia, J., Zhang, W., and Zhang, K. (2020). Economic Assessment of the Peer-to-Peer Trading Policy of Distributed PV Electricity: A Case Study in China. Sustainability, 12.","DOI":"10.3390\/su12135235"},{"key":"ref_55","unstructured":"Guangdong Power Exchange Center (2023, January 20). Monthly Trading Data 2022. Available online: https:\/\/pm.gd.csg.cn\/views\/page\/market.html."}],"container-title":["Axioms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-1680\/12\/2\/142\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:19:58Z","timestamp":1760120398000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-1680\/12\/2\/142"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,30]]},"references-count":55,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["axioms12020142"],"URL":"https:\/\/doi.org\/10.3390\/axioms12020142","relation":{},"ISSN":["2075-1680"],"issn-type":[{"type":"electronic","value":"2075-1680"}],"subject":[],"published":{"date-parts":[[2023,1,30]]}}}