Computer Science

Computer Science ›› 2025, Vol. 52 ›› Issue (9): 337-345.doi: 10.11896/jsjkx.240700197

• Artificial Intelligence • Previous Articles     Next Articles

Dynamic Pricing and Energy Scheduling Strategy for Photovoltaic Storage Charging Stations Based on Multi-agent Deep Reinforcement Learning

CHEN Jintao1,3, LIN Bing2,3, LIN Song1, CHEN Jing3, CHEN Xing3   

  1. 1 College of Computer and Cyber Security,Fujian Normal University,Fuzhou 350117,China
    2 College of Physics and Energy,Fujian Normal University,Fuzhou 350117,China
    3 Fujian Key Laboratory of Network Computing and Intelligent Information Processing,Fuzhou 350116,China
  • Received:2024-07-30 Revised:2024-11-29 Online:2025-09-15 Published:2025-09-11
  • About author:CHEN Jintao,born in 2000,postgra-duate.His main research interests include resource scheduling and reinforcement learning.
    LIN Bing,born in 1986,Ph.D,associate professor,postgraduate supervisor,is a member of CCF(No.83773M).His main research interests include cloud computing technology and computationalintelligence.
  • Supported by:
    Natural Science Foundation of China(62072108), University-Industry Cooperation of Fujian Province(2022H6024,2021H6026),Fujian University Physics Union(FJPHYS-2022-B02),Special Funds for Promoting High-quality Development of Marine and Fishery Industries in Fujian Province(FJHYF-ZH-2023-02) and Fujian Key Technological Innovation and Industrialization Projects(2024XQ004).

PDF (PC)

12

Abstract: The improvement in the operational profits of photovoltaic storage charging stations(PSCSs) can enable charging station operators to increase their investment and deployment of PSCSs infrastructure,thereby alleviating the load pressure on the grid caused by the growing penetration of electric vehicles(EVs).To address the issue of improving PSCSs operational profits,a dynamic pricing and energy scheduling strategy based on multi-agent deep reinforcement learning(MADRL) is proposed to enhance the overall operational profits of PSCSs under a fully cooperative relationship.Firstly,aiming to maximize the total operational profits of all PSCSs,multiple PSCSs and EVs under a single PSCS operator are modeled as a Markov game.Secondly,the multi-agent twin delayed deep deterministic policy gradient(MATD3) algorithm is used to solve the model,setting the selling price of charging services and the charging and discharging strategies of the energy storage system(ESS) to achieve profit maximization.The cosine annealing method is employed to adjust the learning rate of the algorithm,improving its convergence rate and threshold.Finally,to prevent potential price monopolies under a fully cooperative relationship among multiple stations,an inverse demand function is introduced to constrain the selling price of charging services.Experimental results show that the proposed strategy improves the operational profits of charging stations by 4.17% to 66.67% compared to benchmark methods,and using the inverse demand function effectively prevents price monopolies among multiple stations.

Key words: Multi-agent deep reinforcement learning, Photovoltaic storage charging station, Energy scheduling, Dynamic pricing, Inverse demand function

CLC Number: 

  • TP391
[1]FANG Q.Energy storage operation strategy of optical storage charging station based on PPO algorithm[J].Electric Engineering,2024(2):97-100.
[2]LIU Y X,ZHANG S,GUO L,et al.A coordinated optimalscheduling method of distribution grid and photovoltaic storage charging station taking into account electric energy-standby coupling[J].Power System Technology,2024,48(8):3175-3185.
[3]LEE S,CHOI D H.Dynamic pricing and energy management for profit maximization in multiple smart electric vehicle charging stations:A privacy-preserving deep reinforcement learning approach[J].Applied Energy,2021,304:117754.
[4]AFFOLABI L,SHAHIDEHPOUR M,GAN W,et al.Optimal transactive energy trading of electric vehicle charging stations with on-site PV generation in constrained power distribution networks[J].IEEE Transactions on Smart Grid,2021,13(2):1427-1440.
[5]WANG R S,CHEN Z,XING Q,et al.A Modified Rainbow-Based Deep Reinforcement Learning Method for Optimal Scheduling of Charging Station[J].Sustainability,2022,14(3):1884.
[6]ZHANG S,LIU J J,SU Y T.Research on real-time controlstrategy of optical storage charging station based on PSO-DDPG algorithm[J].Mechanical and Electrical Information,2023(17):5-9.
[7]YANG X Y,CUI T X,WANG H R,et al.Multiagent Deep Reinforcement Learning for Electric Vehicle Fast Charging Station Pricing Game in Electricity-Transportation Nexus[J].IEEE Transactions on Industrial Informatics,2024 20(4):6345-6355.
[8]QIAN T,SHAO C,LI X,et al.Multi-agent deep reinforcement learning method for EV charging station game[J].IEEE Transactions on Power Systems,2021,37(3):1682-1694.
[9]ACKERMANN J,GABLER V,OSA T,et al.Reducing overestimation bias in multi-agent domains using double centralized critics[J].arXiv:1910.01465,2019.
[10]HUANG Z Q,LIN B,LU Y,et al.A charging station siting andcapacity determination method for multi-objective optimization[J].Journal of Fujian Normal University(Natural Science Edition),2024,40(2):23-35.
[11]QUE H K,FENG X F,GUO W C,et al.Charging station layout model based on fuzzy bi-objective planning[J].Computer Science,2022,39(3):751-757.
[12]WANG S,BI S,ZHANG Y A.Reinforcement learning for real-time pricing and scheduling control in EV charging stations[J].IEEE Transactions on Industrial Informatics,2019,17(2):849-859.
[13]XU H,WU Q,WEN J,et al.Joint bidding and pricing for electricity retailers based on multi-task deep reinforcement learning[J].International Journal of Electrical Power & Energy Systems,2022,138:107897.
[14]SHIN M J,CHOI D H,KIM J.Cooperative management forPV/ESS-enabled electric vehicle charging stations:A multiagent deep reinforcement learning approach[J].IEEE Transactions on Industrial Informatics,2019,16(5):3493-3503.
[15]KABIR M E,ASSIC C,TUSHAR M H K,et al.Optimal scheduling of EV charging at a solar power-based charging station[J].IEEE Systems Journal,2020,14(3):4221-4231.
[16]ZHOU X Y,QIAN L P,HUANG Y P,et al.An Optimization Method for Electric Vehicle Charging Scheduling Based on Ant Colony Algorithm[J].Computer Science.2020,47(11):280-285.
[17]WU W T,LIN Y,LIU R H,et al.Online EV charge scheduling based on time-of-use pricing and peak load minimization:Properties and efficient algorithms[J].IEEE Transactions on Intelligent Transportation Systems,2020,23(1):572-586.
[18]YU W W,LIU S L,CHEN Q G,et al.Multi-objective optimal scheduling of photovoltaic microgrids considering electric vehicle charging and demand-side response[J].Proceedings of the CSU-EPSA,2018,30(1):88-97.
[19]HAO Y.Energy optimization management of new energy charging station based on genetic algorithm[J].Telecom Power Technologies,2019,36(11):27-28,31.
[20]SU L,JIANG X C,WANG W,et al.Optimized energy management of microgrids taking into account electric vehicles and photovoltaic energy storage[J].Automation of Electric Power Systems,2015,39(9):164-171.
[21]LYU C,ZHAN S,ZHANG Y,et al.Synergistic two-stage optimization for multi-objective energy management strategy of integrated photovoltaic-storage charging stations[J].Journal of Energy Storage,2024,89:111665.
[22]MURIITHI G,CHOWDHURY S.Optimal energy management of a grid-tied solar pv-battery microgrid:A reinforcement learning approach[J].Energies,2021,14(9):2700.
[23]CUI L,WANG Q,QU H,et al.Dynamic pricing for fast charging stations with deep reinforcement learning[J].Applied Energy,2023,346:121334.
[24]LEE S,CHOI D H.Three-Stage Deep Reinforcement Learning for Privacy-and Safety-Aware Smart Electric Vehicle Charging Station Scheduling and Volt/VAR Control[J].IEEE Internet of Things Journal,2023,11(5):8578-8589.
[25]QIAN T,SHAO C,LI X,et al.Multi-agent deep reinforcement learning method for EV charging station game[J].IEEE Transactions on Power Systems,2021,37(3):1682-1694.
[26]VARIAN H R.Intermediate microeconomics with calculus:amodern approach[M].New York:W.W.NORTON & Company,2014:114-115.
[27]LOWE R,WU Y I,TAMAR A,et al.Multi-agent actor-critic for mixed cooperative-competitive environments[C]//Advances in Neural Information Processing Systems.2017.
[28]ZHANG Y,YANG Q,AN D,et al.Multistep multiagent reinforcement learning for optimal energy schedule strategy of charging stations in smart grid[J].IEEE Transactions on Cyber-netics,2022,53(7):4292-4305.
[1] GAO Yuzhao, NIE Yiming. Survey of Multi-agent Deep Reinforcement Learning Based on Value Function Factorization [J]. Computer Science, 2024, 51(6A): 230300170-9.
[2] Cui ZHANG, En WANG, Funing YANG, Yong jian YANG , Nan JIANG. UAV Frequency-based Crowdsensing Using Grouping Multi-agentDeep Reinforcement Learning [J]. Computer Science, 2023, 50(2): 57-68.
[3] SUN Hui-ting, FAN Yan-fang, MA Meng-xiao, CHEN Ruo-yu, CAI Ying. Dynamic Pricing-based Vehicle Collaborative Computation Offloading Scheme in VEC [J]. Computer Science, 2022, 49(9): 242-248.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.