Computer Science ›› 2022, Vol. 49 ›› Issue (4): 263-268.doi: 10.11896/jsjkx.210300155

• Artificial Intelligence • Previous Articles     Next Articles

Heating Strategy Optimization Method Based on Deep Learning

LI Peng1,2, YI Xiu-wen2, QI De-kang1,2, DUAN Zhe-wen2,3, LI Tian-rui1   

  1. 1 School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China;
    2 JD Intelligent Cities Research, Beijing 100176, China;
    3 School of Computer Science and Technology, Xidian University, Xi'an 710071, China
  • Received:2021-03-15 Revised:2021-07-25 Published:2022-04-01
  • About author:LI Peng,born in 1996,postgraduate.His main research interests include deep learning and deep reinforcement learning.YI Xiu-wen,born in 1991,Ph.D,data scientist,researcher,is a member of China Computer Federation.His main research interests include spatio-temporal data mining and deep learning.
  • Supported by:
    This work was supported by the National Key R&D Program of China(2019YFB2101801) and National Natural Science Foundation of China(61773324).

Abstract: Typically, the strategy of central heating for buildings in winter is climate compensator.However, this strategy heavily relies on manual experience with a relatively simple regulation.Therefore, how to optimize the heating control strategy is very important to keep the indoor temperature stable and comfortable.For this task, this paper proposes a heating strategy optimization method based on deep learning and deep reinforcement learning, which can optimize the original control strategy based on real historical data.The paper first develops a deep MTDN (Multiple Time Difference Network) as the simulator to predict the next time slot's room temperature.By learning the thermodynamic law of indoor temperature change, the network has high accuracy and confirms the physical laws.After that, the SAC (Soft Actor-Critic) algorithm based on maximum entropy reinforcement learning is employed as the strategy optimizer to interact with the simulator.Here, we use the evaluation index of the human body's thermal response as the reward to train and optimize the heating control strategy.Based on the real data of a heat exchange station in Tianjin, we evaluate the predictive ability of the simulator and the control ability of the strategy optimizer, respectively.The results verify that, compared with other types of prediction simulators, this simulator not only has high prediction accuracy but also conforms to physical laws.At the same time, compared with the original strategy, the strategy learned by the strategy optimizer can ensure that the indoor temperature is more stable and comfortable in multiple time periods of random sampling.

Key words: Central heating, Deep learning, Deep reinforcement learning, Heating optimization, Urban computing

CLC Number: 

  • TP399
[1] CHENG L.Application of climate compensator in heating system[J].Building Science,2010,26(10):42-46.
[2] CRAWLEY D B,LAWRIE L K,WINKELMANN F C,et al.EnergyPlus:creating a new-generation building energy simulation program[J].Energy and buildings,2001,33(4):319-331.
[3] LI Y,ANG K H,CHONG G C Y.PID control system analysis and design[J].IEEE Control Systems Magazine,2006,26(1):32-41.
[4] HINTON G E,SALAKHUTDINOV R R.Reducing the dimen-sionality of data with neural networks[J].Science,2006,313(5786):504-507.
[5] SILVER D,HUANG A,MADDISON C J,et al.Mastering the game of Go with deep neural networks and tree search[J].Nature,2016,529(7587):484-489.
[6] SCHULMAN J,WOLSKI F,DHARIWAL P,et al.Proximalpolicy optimization algorithms[J].arXiv:1707.06347,2017.
[7] LILLICRAP T P,HUNT J J,PRITZEL A,et al.Continuouscontrol with deep reinforcement learning[J].arXiv:1509.02971,2015.
[8] HAARNOJA T,ZHOU A,ABBEEL P,et al.Soft actor-critic:Off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//International Conference on Machine Learning.PMLR,2018:1861-1870.
[9] DEAR R D,BRAGER G.Developing an adaptive model of thermal comfort and preference[J].Ashrae Trans,1998,104(1):73-81.
[10] FAZLOLLAHI S,BECKER G,MARECHAL F.Multi-objec-tives,multi-period optimization of district energy systems:III.Distribution networks[J].Computers & Chemical Engineering,2014,66(4):82-97.
[11] LI S Q,JIANG Z J.Heating load forecasting model based on Neural Network[J].District Heating,2018,(4):42-46.
[12] BAI H,WANG Y,FAN W Q,et al.Backwater Temperature Control System of Heat Network Based on PID[J].District Heating,2019,(3):132-136.
[13] WU J X,ZHAO T,LIU L S,et al.Research on Heat-exchange Station Operation Based on Flowmaster Simulation[J].District Heating,2019,(4):144-150.
[14] LI Q,HAN B C.Optimal Control of Primary Side of Thermal Power Station Based on Deep Deterministic Policy Gradient[J].Science Technology and Engineering,2019,19(29):193-200.
[15] ZHANG C,KUPPANNAGARI S R,KANNAN R,et al.Buil-ding HVAC scheduling using reinforcement learning via neural network based model approximation[C]//Proceedings of the 6th ACM International Conference on Systems for Energy-efficient Buildings,Cities,and Transportation.2019:287-296.
[16] ZHANG Z,CHONG A,PAN Y,et al.Whole building energy model for HVAC optimal control:A practical framework based on deep reinforcement learning[J].Energy and Buildings,2019,199:472-490.
[17] WEI T,WANG Y,ZHU Q.Deep reinforcement learning forbuilding HVAC control[C]//Proceedings of the 54th Annual Design Automation Conference 2017.2017:1-6.
[18] BROCKMAN G,CHEUNG V,PETTERSSON L,et al.Openai gym[J].arXiv:1606.01540,2016.
[19] TARTARINI F,SCHIAVON S.pythermalcomfort:A Pythonpackage for thermal comfort research[J].SoftwareX,2020,12:100578.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[9] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[10] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[11] YU Bin, LI Xue-hua, PAN Chun-yu, LI Na. Edge-Cloud Collaborative Resource Allocation Algorithm Based on Deep Reinforcement Learning [J]. Computer Science, 2022, 49(7): 248-253.
[12] LI Meng-fei, MAO Ying-chi, TU Zi-jian, WANG Xuan, XU Shu-fang. Server-reliability Task Offloading Strategy Based on Deep Deterministic Policy Gradient [J]. Computer Science, 2022, 49(7): 271-279.
[13] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[14] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[15] WANG Jun-feng, LIU Fan, YANG Sai, LYU Tan-yue, CHEN Zhi-yu, XU Feng. Dam Crack Detection Based on Multi-source Transfer Learning [J]. Computer Science, 2022, 49(6A): 319-324.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!