Computer Science ›› 2026, Vol. 53 ›› Issue (10): 268-276.doi: 10.11896/jsjkx.251200192
• Artificial Intelligence • Previous Articles Next Articles
SHANG Bingnan
CLC Number:
| [1] PATIL R,GUDIVADA V.A review of current trends,tech-niques,and challenges in large language models(llms)[J].Applied Sciences,2024,14(5):2074. [2] ZHAO Y,HE J W,ZHU S C,et al.Security of Large Language Models:Current Status and Challenges[J].Computer Science,2024,51(1):68-71. [3] KALYAN K S.A survey of GPT-3 family large language models including ChatGPT and GPT-4[J].Natural Language Proces-sing Journal,2024,6:100048. [4] GRATTAFIORI A,DUBEY A,JAUHRI A,et al.The llama 3 herd of models[J].arXiv:2407.21783,2024. [5] LIU A,FENG B,XUE B,et al.Deepseek-v3 technical report[J].arXiv:2412.19437,2024. [6] ERNST D,LOUETTE A,FONTENEAU R.Introduction to reinforcement learning[EB/OL].https://damien-ernst.be/wp-content/uploads/2025/04/introduction_to_reinforcement_learning_v250407.pdf. [7] HAN G J,ZHU S C,LIN C,et al.Multi-Agent Reinforcement Learning:From Foundational Theory to Cutting-Edge Algorithms[J].Acta Electronica Sinica,2025,53(12):4756-4786. [8] AGGARWAL P,WELLECK S.L1:Controlling how long a reasoning model thinks with reinforcement learning[J].arXiv:2503.04697,2025. [9] WANG Z,BI B,PENTYALA S K,et al.A comprehensive survey of llm alignment techniques:Rlhf,rlaif,ppo,dpo and more[J].arXiv:2407.16216,2024. [10] JI J,QIU T,CHEN B,et al.AI alignment:A comprehensive survey[J].arXiv:2310.19852,2023. [11] SHEN T,JIN R,HUANG Y,et al.Large language model alignment:A survey[J]. arXiv:2309.15025,2023. [12] HARMON M E,HARMON S S.Reinforcement Learning:ATutorial[R].Wright Laboratory,Avionics Directorate,1997. [13] MOUSAVI S S,SCHUKAT M,HOWLEY E.Deep reinforcement learning:an overview[C]//Proceedings of SAI Intelligent Systems Conference.Cham:Springer,2016:426-440. [14] ZAMBALDI V,RAPOSO D,SANTORO A,et al.Relationaldeep reinforcement learning[J].arXiv:1806.01830,2018. [15] VINYALS O,BABUSCHKIN I,CZARNECKI W M,et al.Grandmaster level in StarCraft II using multi-agent reinforcement learning[J].Nature,2019,575(7782):350-354. [16] BERNER C,BROCKMAN G,CHAN B,et al.Dota 2 with large scale deep reinforcement learning[J].arXiv:1912.06680,2019. [17] SILVER D,HUANG A.MADDISON C,et al.Mastering thegame of Go with deep neural networks and tree search[J].Nature,2016,529:484-489. [18] BYEON H.Advances in value-based,policy-based,and deeplearning-based reinforcement learning[J].International Journal of Advanced Computer Science and Applications,2023,14(8):348-354. [19] RIS-ALA R.Q-Learning Algorithm[M]//Fundamentals of Reinforcement Learning.Cham:Springer,2023:31-55. [20] JIA Y,ZHOU X Y.q-Learning in continuous time[J].Journal of Machine Learning Research,2023,24(161):1-61. [21] MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning[J].Nature,2015,518(7540):529-533. [22] OSBAND I,BLUNDELL C,PRITZEL A,et al.Deep explora-tion via bootstrapped DQN [J].Advances in Neural Information Processing Systems,2016,29:4033-4041. [23] ZHAO H,TANG W,YAO D.Policy optimization for continuous reinforcement learning[J].Advances in Neural Information Processing Systems,2023,36:13637-13663. [24] STÅHLBERG S,BONET B,GEFFNER H.Learning generalpolicies with policy gradient methods[C]//Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning.2023:647-657. [25] WILLIAMS R J.Simple statistical gradient-following algorithms for connectionist reinforcement learning[J].Machine Learning,1992,8(3):229-256. [26] MNIH V,BADIA A P,MIRZA M,et al.Asynchronous methods for deep reinforcement learning[C]//International Conference on Machine Learning.PMLR,2016:1928-1937. [27] HAARNOJA T,ZHOU A,ABBEEL P,et al.Soft actor-critic:Off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//International Conference on Machine Learning.PMLR,2018:1861-1870. [28] SEWAK M.Actor-critic models and the A3C:The asynchronous advantage actor-critic model[M]//Deep reinforcement learning:frontiers of artificial intelligence.Singapore:Springer,2019:141-152. [29] ANDRYCHOWICZ M,RAICHUK A,STAŃCZYK P,et al.What matters in on-policy reinforcement learning? a large-scale empirical study[J].arXiv:2006.05990,2020. [30] MUNOS R,STEPLETON T,HARUTYUNYAN A,et al.Safe and efficient off-policy reinforcement learning[C]//Advances in Neural Information Processing Systems.2016. [31] MOERLAND T M,BROEKENS J,PLAAT A,et al.Model-based reinforcement learning:A survey[J].Foundations and Trends® in Machine Learning,2023,16(1):1-118. [32] WANG D,GAO N,LIU D,et al.Recent progress in reinforcement learning and adaptive dynamic programming for advanced control applications[J].IEEE/CAA Journal of Automatica Sinica,2023,11(1):18-36. [33] ŚWIECHOWSKI M,GODLEWSKI K,SAWICKI B,et al.Monte Carlo tree search:A review of recent modifications and applications[J].Artificial Intelligence Review,2023,56(3):2497-2562. [34] JAMES S,KONIDARIS G,ROSMAN B.An analysis of monte carlo tree search[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2017. [35] CHASLOT G M J,WINANDS M H M,HERIK H J,et al.Progressive strategies for Monte-Carlo tree search[J].New Mathematics and Natural Computation,2008,4(3):343-357. [36] WINANDS M H M,BJÖRNSSON Y,SAITO J T.Monte-Carlo tree search solver[C]//International Conference on Computers and Games.Berlin:Springer,2008:25-36. [37] ZHANG D,HUANG X,ZHOU D,et al.Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B:A Technical Report[J].arXiv:2406.07394,2024. [38] WU J,FENG M,ZHANG S,et al.Beyond examples:High-level automated reasoning paradigm in in-context learning via mcts[J].arXiv:2411.18478,2024. [39] KARTAL B,NUNES E,GODOY J,et al.Monte Carlo treesearch for multi-robot task allocation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2016. [40] LI W,LIU Y,MA Y,et al.A self-learning Monte Carlo treesearch algorithm for robot path planning[J].Frontiers in Neurorobotics,2023,17:1039644. [41] MO S,PEI X,WU C.Safe reinforcement learning for autonomous vehicle using monte carlo tree search[J].IEEE Transactions on Intelligent Transportation Systems,2021,23(7):6766-6773. [42] MCKINNEY L,ZHANG Y.Training language models to follow instructions with human feedback[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems(NIPS’22).2023:27730-27744. [43] WU X.A Comprehensive Survey on Learning from Rewards for Large Language Models:Reward Models and Learning Strategies[C]//Findings of the Association for Computational Linguistics(EMNLP 2025).2025:17847-17875. [44] RAFAILOV R,SHARMA A,MITCHELL E,et al.Direct preference optimization:Your language model is secretly a reward model[J].Advances in Neural Information Processing Systems,2023,36:53728-53741. [45] ETHAYARAJH K,XU W,MUENNIGHOFF N,et al.Kto:Model alignment as prospect theoretic optimization[J].arXiv:2402.01306,2024. [46] CHAUDHARI S,AGGARWAL P,MURAHARI V,et al.Rlhf deciphered:A critical analysis of reinforcement learning from human feedback for llms[J].ACM Computing Surveys,2025,58(2):1-37. [47] DONG H,XIONG W,PANG B,et al.Rlhf workflow:From reward modeling to online rlhf[J].arXiv:2405.07863,2024. [48] OUYANG L,WU J,JIANG X,et al.Training language models to follow instructions with human feedback[J].Advances in Neural Information Processing Systems,2022,35:27730-27744. [49] WANG Y,KORDI Y,MISHRA S,et al.Self-instruct:Aligning language models with self-generated instructions[C]//Procee-dings of the 61st Annual Meeting of the Association for Computational Linguistics.2023:13484-13508. [50] XU C,SUN Q,ZHENG K,et al.Wizardlm:Empowering large language models to follow complex instructions[J].arXiv:2304.12244,2023. [51] ZHENG R,DOU S,GAO S,et al.Secrets of rlhf in large language models part i:Ppo[J].arXiv:2307.04964,2023. [52] SCHULMAN J,WOLSKI F,DHARIWAL P,et al.Proximalpolicy optimization algorithms[J].arXiv:1707.06347,2017. [53] HERSHEY J R,OLSEN P A.Approximating the KullbackLeibler divergence between Gaussian mixture models[C]//2007 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP’07).IEEE,2007. [54] PANG X,TANG S,YE R,et al.Self-alignment of large lan-guage models via multi-agent social simulation[C]//ICLR 2024 Workshop on Large Language Model(LLM) Agents.2024. [55] LEVY J S.An introduction to prospect theory[J].Political Psychology,1992,13(2):171-186. [56] SRIVASTAVA S S,AGGARWAL V.A technical survey of reinforcement learning techniques for large language models[J].arXiv:2507.04136,2025. [57] UESATO J,KUSHMAN N,KUMAR R,et al.Solving mathword problems with process-and outcome-based feedback[J].arXiv:2211.14275,2022. [58] LIU S,CAO J,WANG Y,et al.Self-play reinforcement learning with comprehensive critic in computer games[J].Neurocompu-ting,2021,449:207-213. [59] ARNESEN S,REIN D,MICHAEL J.Training language models to win debates with self-play improves judge accuracy[J].ar-Xiv:2409.16636,2024. [60] WANG Y,LI Z,ZANG Y,et al.Unified multimodal chain-of-thought reward model through reinforcement fine-tuning[J].arXiv:2505.03318,2025. [61] SONG Y,KEMPE J,MUNOS R.Outcome-based exploration for llm reasoning[J].arXiv:2509.06941,2025. [62] ZHANG W,LI X,DONG K,et al.Process vs. Outcome Re-ward:Which is Better for Agentic RAG Reinforcement Learning[J].arXiv:2505.14069,2025. [63] DONG Y,JIANG X,TAO Y,et al.Countering capability boun-dary collapse of llms in reinforcement learning with hybrid-policy optimization[J].arXiv:2508.00222,2025. [64] SHINN N,CASSANO F,GOPINATH A,et al.Reflexion:Language agents with verbal reinforcement learning[J].Advances in Neural Information Processing Systems,2023,36:8634-8652. [65] CAO Y,ZHAO H,CHENG Y,et al.Survey on large language model-enhanced reinforcement learning:Concept,taxonomy,and methods[J].IEEE Transactions on Neural Networks and Lear-ning Systems,2025,36(6):9737-9757. [66] LIU K,YANG D,QIAN Z,et al.Reinforcement learning meets large language models:A survey of advancements and applications across the llm lifecycle[J].arXiv:2509.16679,2025. [67] ZHOU X,CAO L,CHEN X l,et al.A Review of Research on Large Language Model-Empowered Reinforcement Learning Agents[J/OL].Computer Science,1-28.[2026-03-04].https://link.cnki.net/urlid/50.1075.tp.20251222.0819.002. [68] CAO Y Q,CHEN X L,DONG H Y,et al.A Survey on Large Model-Assisted Reward Design Methods for Reinforcement Learning[J].Computer Engineering and Applications,2026,62(9):83-107. |
| [1] | HE Jiaojun, LI Xin. Review of Graph Learning Based on Large Language Models:Methods,Benchmarks and Advances [J]. Computer Science, 2026, 53(9): 1-15. |
| [2] | LI Luozheng, LI Lingbo, YUAN Quan. Survey of Chinese Datasets for LLM Safety Alignment:Landscape and Prospects [J]. Computer Science, 2026, 53(9): 16-23. |
| [3] | AN Ran, CHENG Kai, SHAO Tianhao, LI Changyuan, CHEN Yan. Framework,Techniques and Challenges of Task Planning Agent Based on Large Language Model [J]. Computer Science, 2026, 53(9): 24-37. |
| [4] | XIA Yi, ZHOU Gang, ZHANG Kaixiang, LAN Mingjing, LI Zhufeng, SU Benrong, HE Haofeng, FENG Shizhong. Survey on Collaborative-driven Knowledge Reasoning Technologies Integrating Large Language Models and Knowledge Graphs [J]. Computer Science, 2026, 53(9): 38-54. |
| [5] | ZHANG Rongjie, PANG Xiongwen, WANG Fengling. Review of Large Language Model-based Time Series Modeling via Fine-tuning and AgentArchitecture [J]. Computer Science, 2026, 53(9): 55-70. |
| [6] | CHEN Hai, FAN Changjun, SUN Boliang, CHENG Jie, KONG Junfeng, YUAN Yuchen. Order-based,LLM-augmented Kill-chain for Time-sensitive Target Engagement [J]. Computer Science, 2026, 53(9): 71-81. |
| [7] | GUO Yuyang, SHI Lei, LIU Huan, DONG Yixiang, LI Rui. Domain-adapted and Dynamically Retrieval-augmented Approach for Large-scale History Discipline Model Construction [J]. Computer Science, 2026, 53(9): 92-100. |
| [8] | LEI Kunhan, TAO Jiuyang, LI Chenyu, ZHONG Qi, YE Linfa. Method for Generating Operational Action Plans Based on Large Language Model Guided byChain-of-Thought [J]. Computer Science, 2026, 53(9): 101-109. |
| [9] | WEI Debin, WANG Xinrui, YANG Li, PAN Chengsheng. DTD3-AQM:Satellite Network Queue Management Algorithm Based on Information Timeliness [J]. Computer Science, 2026, 53(9): 375-384. |
| [10] | LI Zhennan, QIAN Jiayan, WANG Xinzhi, ZHANG Hui. Technology Risk Structure Recognition Based on Multi-granularity Semantic Dual Reflection [J]. Computer Science, 2026, 53(9): 395-404. |
| [11] | XIA Bin, SU Jinya, CAO Junmin, XIAO Junwen, SUN Guozi. Research on Log Anomaly Prediction Method for Log Sequence Graph Construction Guided byReinforcement Learning [J]. Computer Science, 2026, 53(9): 405-413. |
| [12] | HE Yulin, XIAO Youqi, YANG Zhenyu, HUANG Zhexue, CUI Laizhong. Adaptive Frequency Tuning Approach for Spark Clusters [J]. Computer Science, 2026, 53(8): 71-84. |
| [13] | JIA Zishuo, ZHANG Jian’ge, HE Haofeng, FENG Shizhong, LIU Yilin. Survey on Mutually Augmenting Technologies and Applications of Large Models and KnowledgeGraphs [J]. Computer Science, 2026, 53(8): 219-228. |
| [14] | LIU Jing. Review of Music Artificial Intelligence Driven by Large Language Models [J]. Computer Science, 2026, 53(8): 229-244. |
| [15] | ZHANG Haoran, HAO Wenning, JIN Dawei, CHENG Kai, LIU Junyang. Agentic Retrieval Augmented Generation Framework Based on Retrieval Task Planning and Reflection Mechanism [J]. Computer Science, 2026, 53(8): 285-297. |
|
||
Home