Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (6A): 230600179-8.doi: 10.11896/jsjkx.230600179

• Interdiscipline & Application • Previous Articles     Next Articles

Multi-agent Based Bidding Strategy Model Considering Wind Power

HUANG Feihu1,2, LI Peidong2, PENG Jian2, DONG Shilei1, ZHAO Honglei1, SONG Weiping1, LI Qiang3   

  1. 1 Aostar Information Technologies Co.,Ltd.,Chengdu 610000,China
    2 College of Computer Science,Sichuan University,Chengdu 610065,China
    3 State Grid Information & Telecommunication Group Co.,Ltd.,Beijing 102211,China
  • Published:2024-06-06
  • About author:HUANG Feihu,born in 1990,Ph.D,is a member of CCF(No.H7742G).His main research interests include reinforcement learning and power big data.
    PENG Jian,born in 1970,Ph.D,professor,Ph.D supervisor,is an outstanding member of CCF(No.22761S).His main research interests include big data and deep learning.
  • Supported by:
    Sichuan Science and Technology Program(22ZYFG0034),Intelligent Terminal Key Laboratory of Sichuan Pro-vince(SCITLAB-20001),Post doctoral Interdisciplinary Innovation Fund(10822041A2137) and Sichuan University and Yibin Cooperation Program(2020CDYB-30).

PDF (PC)

265

Abstract: Under the background of new power system,the pricing problem of new energy generators has been a research hotspot in the electricity spot market.Compared with traditional energy,wind power output is subject to more uncertain factors,which poses a challenge to wind power generators in finding the optimal bid.To address this issue,this paper proposes a pricing strategy model for generators that takes into account of wind power based on the multi-agent reinforcement learning algorithm named WoLF-PHC.In the model,the spot market includes wind power,thermal power,and hydropower,and each generator is abstracted as an intelligent agent,and a stochastic constrained planning algorithm is used to model the profit function of the wind power agent.For the pricing strategy model of the agents,the D3QN algorithm is combined with the WoLF-PHC algorithm,which enables the model to handle complex state spaces when bidding.In addition,to model the interactive environment,a DDPM diffusion model is proposed to generate wind power output data and optimize the simulation of wind power clearing scenarios.In this paper,simulation experiments are carried out based on a 3-node power simulation system.Experimental results show that the proposed wind power profit function modeling,WoLF-PHC improvement,wind power output generation,and other techniques are feasible,which can effectively solve the bidding pricing problem of wind power in the spot market,and learn better strategy after fewer iterations.

Key words: WoLF-PHC, Multi-agent reinforcement learning, Electricity spot market, Bidding strategy, Diffusion model

CLC Number: 

  • TP389
[1]WEN R,TAN L.Short-term Power Forecasting for Photovoltaic Generation Based on HS-ESN[J].Computer Science,2017,44(6):226-231,265.
[2]LU C,JIANG T,DENG H,et al.Bidding Strategy and ProfitDistribution of Power Generation Company with Clean Energy in Spot Market Based on Cooperative Game Theory[J].Electric Power Construction,2020,41(12):9.
[3]NING B.Analysis on generating cost of thermal p0wer plant and its pricing strategy[D].North China Electric Power University,2009.
[4]ZHANG L.Cost Analysis and Bidding Strategy Research ofPower Plants under Power Market Environment[D].North China Electric Power University,2010.
[5]WU J,WANG J,YAN Y.Agent-based dynamic simulation of an electricity market with multilateral bidding[J].International Journal of Modeling,Simulation,and Scientific Computing,2019,10(3):15-27.
[6]YANG M.Research on bidding strategy of electricity marketbased on current trading mode[D].Beijing University of Posts and Telecommunications,2019.
[7]WANG J,WU J,KONG X,Multi-agent simulation for strategic bidding in electricity markets using reinforcement learning[J].Power and Energy Systems,2020,9(3):1051-1065.
[8]CHEN L,GUO T,LIU Y T,et al.Survey of Multi-Agent Stra-tegy Based on Reinforcement Learning[C]//2020 Chinese Control and Decision Conference(CCDC).2020:604-609.
[9]DONG H,DONG H,DING Z,et al.Deep Reinforcement Lear-ning[M].Singapore:Springer,2020.
[10]KONG D,KONG X,XIAO J,et al.Dynamic pricing of demand response based on elasticity transfer and reinforcement learning[C]//2019 22nd International Conference on Electrical Machines and Systems(ICEMS).IEEE,2019:1-5.
[11]GAO Y,LI J,CAO R,et al.Simulation of Generators’ Bidding Behavior Based on Multi-agent Double DQN[J].Power System Technology,2020,44(11):4175-4183.
[12]LIU D,GAO Y,WANG W,et al.Research on bidding strategy of thermal power companies in electricity market based on Multi-Agent Deep Deterministic Policy Gradient[J].IEEE access,2021,9:81750-81764.
[13]TANG C,ZHANG L,LIU F,et al.Research on Pricing Mechanism of Electricity Spot Market Based on Multi-agent Reinforcement Learning(Part I):Bi-level Optimization Model for Generators Under Different Pricing Mechanisms[J].Proceedings of the CSEE,2021,41(2):536-552.
[14]JIANG T,WANG X,JIANG C,et al.Optimal Hybrid Stochastic Robust Bidding Strategy of Wind and Hydraulic Pumped Storage Jointly Participating in Day-ahead and Real-time Market Using Data-driven Method[J].Power System Technology,2022,46(2):481-495.
[15]DONG X,SUN Y,PU T.Day-ahead scenario generation of renewable energy based on conditional GAN[J].Proceedings of the CSEE,2020,40(17):5527-5536.
[16]CHEN Y,WANG Y,KIRSCHEN D S,et al.Model-free renewable scenario generation using generative adversarial networks[J].IEEE Transactions on Power Systems,2017,33(99):3265-3275.
[17]HO J,JAIN A,ABBEEL P.Denoising diffusion probabilisticmodels[J].Advances in Neural Information Processing Systems,2020,33:6840-6851.
[18]YUAN W,LUO J,LU L,et al.Methods in Adversarial Intelligent Game:A Holistic Comparative Analysis from Perspective of Game Theory and Reinforcement Learning[J].Computer Science,2022,49(8):191-204.
[19]QIN Z,LI N,LIU X,et al.Overview of Research on Model-free Reinforcement Learning[J].Computer Science,2021,48(3):180-187.
[20]ZHANG X,GAO W,ZHONG J.Decentralized Economic Dis-patching of Multi-Micro Grid Considering Wind Power and Photovoltaic Output Uncertainty[J].IEEE Access,2021,9(104):93-103.
[21]ZHANG L,TANG C,LIU F,et al.Research on Pricing Mechanism of Electricity Spot Market Based on Multi-Agent Reinforcement Learning(Part II):Decision-making Framework of Pricing Mechanism Combined With Theory and Simulation[J].Proceedings of the CSEE,2021,41(3):1004-1017.
[22]CHEN Y,WANG Y,KIRSCHEN D S,et al.Model-free renewable scenario generation using generative adversarial networks[J].IEEE Transactions on Power Systems,2017,33(99):3265-3275.
[23]JIANG C,MAO Y,CHAI Y,et al.Scenario generation for wind power using improved generative adversarial networks[J].IEEE Access,2018,6:62193-62203.
[24]LIANG J,TANG W.Wind power scenario generation for microgrid day-ahead scheduling using sequential generative adversarial networks[C]//2019 IEEE International Conference on Communications,Control,and Computing Technologies for Smart Grids.2019:1-6.
[25]WANG J,ZENG K,ZHANG J,et al.GAN-generated Face Detection Based on Space-Frequency Convolutional Neural Network[J].Computer Science,2023,50(6):216-224.
[26]FENG H,HUANG C,FENG S,et al.Bidding Model of Generator Considering the Medium and Long-Term contract[C]//Proceedings of the 2019 Annual Meeting of the Power Market Committee of the Chinese Society of Electrical Engineering.2019:336-342.
[27]XU B,ZHU F,ZHONG P,et al.Identifying long-term effects of using hydropower to complement wind power uncertainty through stochastic programming[J].Applied Energy.2019,253:113535.
[28]WANG D,LIU B,JIA H,et al.Peer-to-Peer Energy Transaction Decision of Prosumers Based on Reinforcement Learning[J].Automation of Electric Power Systems,2021,45(3):139-147.
[29]KONG D,KONG X,XIAO J,et al.Dynamic pricing of demand response based on elasticity transfer and reinforcement learning[C]//2019 22nd International Conference on Electrical Machines and Systems(ICEMS).2019:1-5.
[30]HAN D,HUANG W,YAN Z.Deep Reinforcement Learning for Virtual Bidding in Electricity Markets[J].Proceedings of the CSEE,2022,42(4):1443-1455.
[1] XIN Yuanxia, HUA Daoyang, ZHANG Li. Multi-agent Reinforcement Learning Algorithm Based on AI Planning [J]. Computer Science, 2024, 51(5): 179-192.
[2] SHI Dianxi, HU Haomeng, SONG Linna, YANG Huanhuan, OUYANG Qianying, TAN Jiefu , CHEN Ying. Multi-agent Reinforcement Learning Method Based on Observation Reconstruction [J]. Computer Science, 2024, 51(4): 280-290.
[3] GE Yinchi, ZHANG Hui, SUN Haohang. Differential Privacy Data Synthesis Method Based on Latent Diffusion Model [J]. Computer Science, 2024, 51(3): 30-38.
[4] YAN Zhihao, ZHOU Zhangbing, LI Xiaocui. Survey on Generative Diffusion Model [J]. Computer Science, 2024, 51(1): 273-283.
[5] LUO Ruiqing, ZENG Kun, ZHANG Xinjing. Curriculum Learning Framework Based on Reinforcement Learning in Sparse HeterogeneousMulti-agent Environments [J]. Computer Science, 2024, 51(1): 301-309.
[6] XIONG Liqin, CAO Lei, CHEN Xiliang, LAI Jun. Value Factorization Method Based on State Estimation [J]. Computer Science, 2023, 50(8): 202-208.
[7] LIN Xiangyang, XING Qinghua, XING Huaixi. Study on Intelligent Decision Making of Aerial Interception Combat of UAV Group Based onMADDPG [J]. Computer Science, 2023, 50(6A): 220700031-7.
[8] RONG Huan, QIAN Minfeng, MA Tinghuai, SUN Shengjie. Novel Class Reasoning Model Towards Covered Area in Given Image Based on InformedKnowledge Graph Reasoning and Multi-agent Collaboration [J]. Computer Science, 2023, 50(1): 243-252.
[9] SHI Dian-xi, ZHAO Chen-ran, ZHANG Yao-wen, YANG Shao-wu, ZHANG Yong-jun. Adaptive Reward Method for End-to-End Cooperation Based on Multi-agent Reinforcement Learning [J]. Computer Science, 2022, 49(8): 247-256.
[10] DU Wei, DING Shi-fei. Overview on Multi-agent Reinforcement Learning [J]. Computer Science, 2019, 46(8): 1-8.
[11] ZHENG Hong-bo, WU Bin, XU Fei, ZHANG Mei-yu, QIN Xu-jia. Visualization of Solid Waste Incineration Exhaust Emissions Based on Gaussian Diffusion Model [J]. Computer Science, 2019, 46(6A): 527-531.
[12] ZHAO Hai-yong and JIA Yang-li. Improved Anisotropic Diffusion Denosing Model [J]. Computer Science, 2013, 40(Z11): 147-149.
[13] . Research on Influence Maximization Problem Based on Dynamic Networks [J]. Computer Science, 2012, 39(6): 111-115.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.