计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (6A): 230600179-8.doi: 10.11896/jsjkx.230600179

• 交叉&应用 • 上一篇    下一篇

计及风电的发电商报价多智能体模型

黄飞虎1,2, 李沛东2, 彭舰2, 董石磊1, 赵红磊1, 宋卫平1, 李强3   

  1. 1 四川中电启明星信息技术有限公司 成都 610000
    2 四川大学计算机学院 成都 610065
    3 国网信息通信产业集团有限公司 北京 102211
  • 发布日期:2024-06-06
  • 通讯作者: 彭舰(jianpeng@scu.edu.cn)
  • 作者简介:(huangfh@scu.edu.cn)
  • 基金资助:
    四川省重点研发计划(2023YFG0112,22ZYFG0034);四川省重点实验室开放课题(SCITLAB-20001);四川大学博士后交叉学科基金(10822041A2137);四川大学宜宾市合作项目(2020CDYB-30)

Multi-agent Based Bidding Strategy Model Considering Wind Power

HUANG Feihu1,2, LI Peidong2, PENG Jian2, DONG Shilei1, ZHAO Honglei1, SONG Weiping1, LI Qiang3   

  1. 1 Aostar Information Technologies Co.,Ltd.,Chengdu 610000,China
    2 College of Computer Science,Sichuan University,Chengdu 610065,China
    3 State Grid Information & Telecommunication Group Co.,Ltd.,Beijing 102211,China
  • Published:2024-06-06
  • About author:HUANG Feihu,born in 1990,Ph.D,is a member of CCF(No.H7742G).His main research interests include reinforcement learning and power big data.
    PENG Jian,born in 1970,Ph.D,professor,Ph.D supervisor,is an outstanding member of CCF(No.22761S).His main research interests include big data and deep learning.
  • Supported by:
    Sichuan Science and Technology Program(22ZYFG0034),Intelligent Terminal Key Laboratory of Sichuan Pro-vince(SCITLAB-20001),Post doctoral Interdisciplinary Innovation Fund(10822041A2137) and Sichuan University and Yibin Cooperation Program(2020CDYB-30).

PDF (PC)

270

摘要： 新型电力系统背景下,新能源发电商的报价问题一直是电力现货市场中的研究热点。相比传统能源,风电出力受外界不确定性因素的影响较大,给风力发电商求解最优报价带来了挑战。为此,基于多智能体强化学习算法WoLF-PHC构建了计及风电的发电商报价策略模型。模型中,考虑了风电、火电和水电3种能源参与的现货市场,每一个发电商抽象为一个智能体,且基于随机约束规划算法建模风电智能体的收益函数;对于智能体的报价策略模型,将D3QN与WoLF-PHC算法结合,使模型能够满足报价时智能体状态空间复杂的情况;此外,对于交互环境的建模,提出利用DDPM扩散模型生成风电出力数据,优化风电出清场景的仿真。最后,基于3节点的电力仿真系统开展模拟实验,实验结果表明,提出的风电收益函数建模、WoLF-PHC改进、风电出力生成等技术是可行的,能有效解决风电参与竞价的现货市场报价问题,并且能够在较少的迭代次数后学习到较优的策略。

关键词: WoLF-PHC, 多智能体强化学习, 电力现货市场, 竞价策略, 扩散模型

Abstract: Under the background of new power system,the pricing problem of new energy generators has been a research hotspot in the electricity spot market.Compared with traditional energy,wind power output is subject to more uncertain factors,which poses a challenge to wind power generators in finding the optimal bid.To address this issue,this paper proposes a pricing strategy model for generators that takes into account of wind power based on the multi-agent reinforcement learning algorithm named WoLF-PHC.In the model,the spot market includes wind power,thermal power,and hydropower,and each generator is abstracted as an intelligent agent,and a stochastic constrained planning algorithm is used to model the profit function of the wind power agent.For the pricing strategy model of the agents,the D3QN algorithm is combined with the WoLF-PHC algorithm,which enables the model to handle complex state spaces when bidding.In addition,to model the interactive environment,a DDPM diffusion model is proposed to generate wind power output data and optimize the simulation of wind power clearing scenarios.In this paper,simulation experiments are carried out based on a 3-node power simulation system.Experimental results show that the proposed wind power profit function modeling,WoLF-PHC improvement,wind power output generation,and other techniques are feasible,which can effectively solve the bidding pricing problem of wind power in the spot market,and learn better strategy after fewer iterations.

Key words: WoLF-PHC, Multi-agent reinforcement learning, Electricity spot market, Bidding strategy, Diffusion model

中图分类号: 

  • TP389
[1]WEN R,TAN L.Short-term Power Forecasting for Photovoltaic Generation Based on HS-ESN[J].Computer Science,2017,44(6):226-231,265.
[2]LU C,JIANG T,DENG H,et al.Bidding Strategy and ProfitDistribution of Power Generation Company with Clean Energy in Spot Market Based on Cooperative Game Theory[J].Electric Power Construction,2020,41(12):9.
[3]NING B.Analysis on generating cost of thermal p0wer plant and its pricing strategy[D].North China Electric Power University,2009.
[4]ZHANG L.Cost Analysis and Bidding Strategy Research ofPower Plants under Power Market Environment[D].North China Electric Power University,2010.
[5]WU J,WANG J,YAN Y.Agent-based dynamic simulation of an electricity market with multilateral bidding[J].International Journal of Modeling,Simulation,and Scientific Computing,2019,10(3):15-27.
[6]YANG M.Research on bidding strategy of electricity marketbased on current trading mode[D].Beijing University of Posts and Telecommunications,2019.
[7]WANG J,WU J,KONG X,Multi-agent simulation for strategic bidding in electricity markets using reinforcement learning[J].Power and Energy Systems,2020,9(3):1051-1065.
[8]CHEN L,GUO T,LIU Y T,et al.Survey of Multi-Agent Stra-tegy Based on Reinforcement Learning[C]//2020 Chinese Control and Decision Conference(CCDC).2020:604-609.
[9]DONG H,DONG H,DING Z,et al.Deep Reinforcement Lear-ning[M].Singapore:Springer,2020.
[10]KONG D,KONG X,XIAO J,et al.Dynamic pricing of demand response based on elasticity transfer and reinforcement learning[C]//2019 22nd International Conference on Electrical Machines and Systems(ICEMS).IEEE,2019:1-5.
[11]GAO Y,LI J,CAO R,et al.Simulation of Generators’ Bidding Behavior Based on Multi-agent Double DQN[J].Power System Technology,2020,44(11):4175-4183.
[12]LIU D,GAO Y,WANG W,et al.Research on bidding strategy of thermal power companies in electricity market based on Multi-Agent Deep Deterministic Policy Gradient[J].IEEE access,2021,9:81750-81764.
[13]TANG C,ZHANG L,LIU F,et al.Research on Pricing Mechanism of Electricity Spot Market Based on Multi-agent Reinforcement Learning(Part I):Bi-level Optimization Model for Generators Under Different Pricing Mechanisms[J].Proceedings of the CSEE,2021,41(2):536-552.
[14]JIANG T,WANG X,JIANG C,et al.Optimal Hybrid Stochastic Robust Bidding Strategy of Wind and Hydraulic Pumped Storage Jointly Participating in Day-ahead and Real-time Market Using Data-driven Method[J].Power System Technology,2022,46(2):481-495.
[15]DONG X,SUN Y,PU T.Day-ahead scenario generation of renewable energy based on conditional GAN[J].Proceedings of the CSEE,2020,40(17):5527-5536.
[16]CHEN Y,WANG Y,KIRSCHEN D S,et al.Model-free renewable scenario generation using generative adversarial networks[J].IEEE Transactions on Power Systems,2017,33(99):3265-3275.
[17]HO J,JAIN A,ABBEEL P.Denoising diffusion probabilisticmodels[J].Advances in Neural Information Processing Systems,2020,33:6840-6851.
[18]YUAN W,LUO J,LU L,et al.Methods in Adversarial Intelligent Game:A Holistic Comparative Analysis from Perspective of Game Theory and Reinforcement Learning[J].Computer Science,2022,49(8):191-204.
[19]QIN Z,LI N,LIU X,et al.Overview of Research on Model-free Reinforcement Learning[J].Computer Science,2021,48(3):180-187.
[20]ZHANG X,GAO W,ZHONG J.Decentralized Economic Dis-patching of Multi-Micro Grid Considering Wind Power and Photovoltaic Output Uncertainty[J].IEEE Access,2021,9(104):93-103.
[21]ZHANG L,TANG C,LIU F,et al.Research on Pricing Mechanism of Electricity Spot Market Based on Multi-Agent Reinforcement Learning(Part II):Decision-making Framework of Pricing Mechanism Combined With Theory and Simulation[J].Proceedings of the CSEE,2021,41(3):1004-1017.
[22]CHEN Y,WANG Y,KIRSCHEN D S,et al.Model-free renewable scenario generation using generative adversarial networks[J].IEEE Transactions on Power Systems,2017,33(99):3265-3275.
[23]JIANG C,MAO Y,CHAI Y,et al.Scenario generation for wind power using improved generative adversarial networks[J].IEEE Access,2018,6:62193-62203.
[24]LIANG J,TANG W.Wind power scenario generation for microgrid day-ahead scheduling using sequential generative adversarial networks[C]//2019 IEEE International Conference on Communications,Control,and Computing Technologies for Smart Grids.2019:1-6.
[25]WANG J,ZENG K,ZHANG J,et al.GAN-generated Face Detection Based on Space-Frequency Convolutional Neural Network[J].Computer Science,2023,50(6):216-224.
[26]FENG H,HUANG C,FENG S,et al.Bidding Model of Generator Considering the Medium and Long-Term contract[C]//Proceedings of the 2019 Annual Meeting of the Power Market Committee of the Chinese Society of Electrical Engineering.2019:336-342.
[27]XU B,ZHU F,ZHONG P,et al.Identifying long-term effects of using hydropower to complement wind power uncertainty through stochastic programming[J].Applied Energy.2019,253:113535.
[28]WANG D,LIU B,JIA H,et al.Peer-to-Peer Energy Transaction Decision of Prosumers Based on Reinforcement Learning[J].Automation of Electric Power Systems,2021,45(3):139-147.
[29]KONG D,KONG X,XIAO J,et al.Dynamic pricing of demand response based on elasticity transfer and reinforcement learning[C]//2019 22nd International Conference on Electrical Machines and Systems(ICEMS).2019:1-5.
[30]HAN D,HUANG W,YAN Z.Deep Reinforcement Learning for Virtual Bidding in Electricity Markets[J].Proceedings of the CSEE,2022,42(4):1443-1455.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发