Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (4): 344-352.doi: 10.11896/jsjkx.230100048

• Artificial Intelligence • Previous Articles     Next Articles

Deep Reinforcement Learning Portfolio Model Based on Dynamic Selectors

ZHAO Miao1, XIE Liang1, LIN Wenjing1, XU Haijiao2   

  1. 1 College of Science,Wuhan University of Technology,Wuhan 430070,China
    2 School of Computer Science,Guangdong University of Education,Guangzhou 510303,China
  • Received:2023-01-10 Revised:2023-05-30 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    Natural Science Foundation of Guangdong Province,China(2020A1515011208),Basic and Applied Basic Research Project of Guangzhou Basic Research Program(202102080353) and Characteristic Innovation Project of Natural Science in General Colleges and Universities in Guangdong Province(2019KTSCX117).

PDF (PC)

418

Abstract: In recent years,portfolio management problems have been extensively studied in the field of artificial intelligence,but there are some improvements in the existing quantitative trading methods based on deep learning.First of all,the prediction model of stocks is single,usually a model only trains a trading expert,and the decision of trading is only based on the prediction results of the model.Secondly,the data source used in the model is relatively single,only considering the stock's own data,ignoring the impact of the entire market risk on the stock.Aiming at the above problems,a reinforcement learning model based on dynamic selection predictor(DSDRL) is proposed.The model is divided into three parts.Firstly,the characteristics of stock data are extracted and introduced into multiple predictors.Multiple prediction models are trained for different investment strategies,and the current optimal prediction results are obtained by dynamic selector.Secondly,the market environment evaluation module is used to quantify the current market risk and obtain the appropriate proportion of investment amount.Finally,based on the first two mo-dules,a deep reinforcement learning model is established to simulate the real trading environment,and the actual portfolio strategy is obtained based on the predicted results and the proportion of investment amount.In this paper,the daily k-line data of China Securities 500 and S & P 500 are used for test verification.The results show that the proposed model is superior to other refe-rence models in Sharpe rate and other indicators.

Key words: Reinforcement learning, LSTM, Investment portfolio, Stock market forecast, Neural networks

CLC Number: 

  • TP391
[1]QIN Y,SONG D,CHEN H,et al.A dual-stage attention-based recurrent neural network for time series prediction[J].arXiv:1704.02971,2017.
[2]XIE C,RAJAN D,CHAI Q.An interpretable Neural FuzzyHammerstein-Wiener network for stock price prediction[J].Information Sciences,2021,577:324-335.
[3]TAKEUCHI L,LEE Y Y.Applying deep learning to enhance momentum trading strategies in stocks[R].Technical Report.Stanford,CA,USA:Stanford University,2013.
[4]RUNDO F.Deep LSTM with reinforcement learning layer forfinancial trend prediction in FX high frequency trading systems[J].Applied Sciences,2019,9(20):4460.
[5]SEZER O B,OZBAYOGLU A M.Algorithmic Financial Tra-ding with Deep Convolutional Neural Networks:Time Series to Image Conversion Approach[J].Applied Soft Computing,2018,70:525-538.
[6]YANG H,LIU X Y,ZHONG S,et al.Deep reinforcement lear-ning for automated stock trading:an ensemble strategy[C]//Proceedings of the first ACM International Conference on AI in Finance.2020:1-8.
[7]LIN H,ZHOU D,LIU W,et al.Learning multiple stock trading patterns with temporal routing adaptor and optimal transport[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining.2021:1017-1026.
[8]MILLER C,ARJUNAN P,KATHIRGAMANATHAN A,et al.The ASHRAE great energy predictor III competition:Overview and results[J].Science and Technology for the Built Environment,2020,26(10):1427-1447.
[9]LIANG Y,KE S,ZHANG J,et al.Geoman:Multi-level attention networks for geo-sensory time series prediction[C]//IJCAI.2018:3428-3434.
[10]GILBERT K.An ARIMA supply chain model[J].Management Science,2005,51(2):305-310.
[11]DURBIN J,KOOPMAN S J.Time series analysis by state space methods[M].OUP Oxford,2012.
[12]CAETANO M A L,YONEYAMA T.Characterizing abruptchanges in the stock prices using a wavelet decomposition me-thod[J].Physica A:Statistical Mechanics and its Applications,2007,383(2):519-526.
[13]DING Q,WU S,SUN H,et al.Hierarchical Multi-Scale Gaus-sian Transformer for Stock Movement Prediction[C]//IJCAI.2020:4640-4646.
[14]KVAMME H,SELLEREITE N,AAS K,et al.Predicting mortgage default using convolutional neural networks[J].Expert Systems with Applications,2018,102:207-217.
[15]CHEN W,YEO C K,LAU C T,et al.Leveraging social media news to predict stock index movement using RNN-boost[J].Data & Knowledge Engineering,2018,118:14-24.
[16]PIAO G,BRESLIN J G.Financial aspect and sentiment predictions with deep neural networks:an ensemble approach[C]//Companion Proceedings of the The Web Conference 2018.2018:1973-1977.
[17]SPILAK B.Deep neural networks for cryptocurrencies price prediction[D].Berlin:Humboldt-Universität zu Berlin,2018.
[18]SI W,LI J,DING P,et al.A multi-objective deep reinforcement learning approach for stock index future's intraday trading[C]//2017 10th International Symposium on Computational Intelligence and Design(ISCID).IEEE,2017:431-436.
[19]GLASSERMAN P,KRSTOVSKI K,LALIBERTE P,et al.Choosing news topics to explain stock market returns[C]//Proceedings of the First ACM International Conference on AI in Finance.2020:1-8.
[20]BRAHIMI B,TOUAHRIA M,TARI A.Improving sentimentanalysis in Arabic:A combined approach[J].Journal of King Saud University－Computer and Information Sciences,2021,33(10):1242-1250.
[21]AZHIKODAN A R,BHAT A G K,JADHAV M V.Stocktrading bot using deep reinforcement learning[C]//Innovations in Computer Science and Engineering:Proceedings of the Fifth ICICSE 2017.Singapore:Springer,2019:41-49.
[22]GRACE A.Can deep learning techniques improve the risk adjusted returns from enhanced indexing investment strategies[J/OL].https://arrow.tudublin.ie/cgi/viewcontent.cgi?article=1129&context=scschcomdis.
[23]FU X Y,DU J H,GUO Y F,et al.A machine learning framework for stock selection[J].arXiv:1806.01743,2018.
[24]CHEN L,GAO Q.Application of deep reinforcement learning on automated stock trading[C]//2019 IEEE 10th International Conference on Software Engineering and Service Science(ICSESS).IEEE,2019:29-33.
[25]PUN C S,WANG L,WONG H Y.Financial thought experiment:a gan-based ap-proach to vast robust portfolio selection[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence(IJCAI'20).2020:451-458.
[26]NEUNEIER R.Optimal asset allocation using adaptive dynamic programming[J].Advances in Neural Information Processing Systems,1995,8:952-958.
[27]MOODY J,SAFFELL M.Reinforcement learning for trading[J].Advances in Neural Information Processing Systems,1998,11:917-923.
[28]DENG Y,BAO F,KONG Y,et al.Deep direct reinforcement learning for financial signal representation and trading[J]. IEEE transactions on Neural Networks and Learning Systems,2016,28(3):653-664.
[29]YU P,LEE J S,KULYATIN I,et al.Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization[J].ar-Xiv:1901.08740,2019.
[30]LIU X Y,YANG H Y,CHEN Q,et al.Finrl:A deep reinforcement learning library forautomated stock trading in quantitative finance[J].arXiv:2011.09607,2020.
[31]WANG Z C,HUANG B W,TU S,et al.Deeptrader:a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions embedding[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2021:643-650.
[1] WANG Xu, LIU Changhong, LI Shengchun, LIU Shuang, ZHAO Kangting, CHEN Liang. Study on Manufacturing Company Automated Chart Analysis Method Based on Natural LanguageGeneration [J]. Computer Science, 2024, 51(4): 174-181.
[2] WANG Ruiping, WU Shihong, ZHANG Meihang, WANG Xiaoping. Review of Vision-based Neural Network 3D Dynamic Gesture Recognition Methods [J]. Computer Science, 2024, 51(4): 193-208.
[3] SHI Dianxi, HU Haomeng, SONG Linna, YANG Huanhuan, OUYANG Qianying, TAN Jiefu , CHEN Ying. Multi-agent Reinforcement Learning Method Based on Observation Reconstruction [J]. Computer Science, 2024, 51(4): 280-290.
[4] HE Jiaojun, CAI Manchun, LU Tianliang. Android Malware Detection Method Based on GCN and BiLSTM [J]. Computer Science, 2024, 51(4): 388-395.
[5] WANG Yao, LUO Junren, ZHOU Yanzhong, GU Xueqiang, ZHANG Wanpeng. Review of Reinforcement Learning and Evolutionary Computation Methods for StrategyExploration [J]. Computer Science, 2024, 51(3): 183-197.
[6] ZHENG Cheng, SHI Jingwei, WEI Suhua, CHENG Jiaming. Dual Feature Adaptive Fusion Network Based on Dependency Type Pruning for Aspect-basedSentiment Analysis [J]. Computer Science, 2024, 51(3): 205-213.
[7] WANG Yan, WANG Tianjing, SHEN Hang, BAI Guangwei. Optimal Penetration Path Generation Based on Maximum Entropy Reinforcement Learning [J]. Computer Science, 2024, 51(3): 360-367.
[8] XU Tianyue, LIU Xianhui, ZHAO Weidong. Knowledge Graph and User Interest Based Recommendation Algorithm [J]. Computer Science, 2024, 51(2): 55-62.
[9] LI Junwei, LIU Quan, XU Yapeng. Option-Critic Algorithm Based on Mutual Information Optimization [J]. Computer Science, 2024, 51(2): 252-258.
[10] SHI Dianxi, PENG Yingxuan, YANG Huanhuan, OUYANG Qianying, ZHANG Yuhui, HAO Feng. DQN-based Multi-agent Motion Planning Method with Deep Reinforcement Learning [J]. Computer Science, 2024, 51(2): 268-277.
[11] WANG Yangmin, HU Chengyu, YAN Xuesong, ZENG Deze. Study on Deep Reinforcement Learning for Energy-aware Virtual Machine Scheduling [J]. Computer Science, 2024, 51(2): 293-299.
[12] ZHAO Xiaoyan, ZHAO Bin, ZHANG Junna, YUAN Peiyan. Study on Cache-oriented Dynamic Collaborative Task Migration Technology [J]. Computer Science, 2024, 51(2): 300-310.
[13] WNAG Yuzhen, ZONG Guoxiao, WEI Qiang. SGPot:A Reinforcement Learning-based Honeypot Framework for Smart Grid [J]. Computer Science, 2024, 51(2): 359-370.
[14] LUO Ruiqing, ZENG Kun, ZHANG Xinjing. Curriculum Learning Framework Based on Reinforcement Learning in Sparse HeterogeneousMulti-agent Environments [J]. Computer Science, 2024, 51(1): 301-309.
[15] GUO Yuxing, YAO Kaixuan, WANG Zhiqiang, WEN Liangliang, LIANG Jiye. Black-box Graph Adversarial Attacks Based on Topology and Feature Fusion [J]. Computer Science, 2024, 51(1): 355-362.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.