计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (12): 286-292.doi: 10.11896/jsjkx.240300104

• 人工智能 • 上一篇    下一篇

基于大语言模型的电力知识库智能问答系统构建与评价

张金营1, 王天堃1, 么长英2, 谢华2, 柴林政3, 刘书恺3, 李彤亮4, 李舟军3   

  1. 1 国电电力发展股份有限公司 辽宁 大连 116000
    2 国能信控互联技术有限公司 北京 100039
    3 北京航空航天大学计算机学院 北京 100191
    4 北京信息科技大学计算机学院 北京 102206
  • 收稿日期:2024-03-14 修回日期:2024-05-15 出版日期:2024-12-15 发布日期:2024-12-10
  • 通讯作者: 李舟军(lizj@buaa.edu.cn)
  • 作者简介:jinying.zhang@chnenergy.com.cn
  • 基金资助:
    国家自然科学基金(62276017);软件开发环境国家重点实验室自主研究项目(SKLSDE-2021ZX-18)

Construction and Evaluation of Intelligent Question Answering System for Electric Power Knowledge Base Based on Large Language Model

ZHANG Jinying1, WANG Tiankun1, YAO Changying2, XIE Hua2, CHAI Linzheng3, LIU Shukai3, LI Tongliang4, LI Zhoujun3   

  1. 1 Guodian Power Development Co., Ltd., Dalian, Liaoning 116000, China
    2 Guoneng Xinkong Internet Technology Co., Ltd., Beijing 100039, China
    3 School of Computer Science and Engineering, Beihang University, Beijing 100191, China
    4 Computer School, Beijing Information Science and Technology University, Beijing 102206, China
  • Received:2024-03-14 Revised:2024-05-15 Online:2024-12-15 Published:2024-12-10
  • About author:ZHANG Jinying,born in 1984,Ph.D.His main research interests include artificial intelligence,distributed control systems,and fieldbus control systems.
    LI Zhoujun,born in 1963,Ph.D,professor,is a member of CCF(No.06618S).His main research interests include artificial intelligence,natural language processing,network and information secu-rity.
  • Supported by:
    Natural Science Foundation of China(62276017) and State Key Laboratory of Complex& Critical Software Environment(SKLSDE-2021ZX-18).

PDF (PC)

388

摘要： 大语言模型是近年来自然语言处理领域的一个重大突破,已成为该领域研究的一种新范式。在金融、法律等垂直领域,基于FinGPT,ChatLaw等垂直领域大模型的智能问答系统,促进了大模型技术在相关领域的学术研究与应用落地。然而,由于电力领域缺乏相关的高质量数据,相关的大模型问答系统的构建工作遇到了较大阻碍。为了构建电力领域的智能问答系统,提出了基于大语言模型的电力知识库智能问答系统 ChatPower。为了确保问答效果,ChatPower充分利用了电力管理各环节的数据。通过语义化理解,梳理和整合了大量的电力专业知识,精心设计和构建了一个较大规模的电力系统知识库。该知识库覆盖电力相关规章制度、安全生产管理体系以及发电设备故障知识等方面的内容。此外,通过参考检索到的电力知识,ChatPower显著缓解了问答中存在的模型幻觉问题,并在检索系统中引入了BM25检索、向量库检索与重排相结合的方法,有效降低了单纯依赖向量库检索的不准确性。同时,ChatPower结合基于大模型的提示工程技术,提升了对于规章制度类型问题生成回复的条理性。为了对问答系统进行评价,构建了一个电力知识问答的测试数据集,并对其进行了测试验证,测试结果表明:基于大语言模型的电力知识库问答系统ChatPower能够有效提升电力相关知识的检索和问答的准确性。

关键词: 大语言模型, 知识库问答系统, 信息检索, 自然语言生成

Abstract: Large language model is a major breakthrough in the field of natural language processing in recent years and have become a new paradigm for research in this field.In vertical fields such as finance and law,intelligent question and answering systems based on large models in vertical fields such as FinGPT and ChatLaw have promoted the academic research and application of large model technology in related fields.However,due to the lack of relevant high-quality data in the electric power field,the construction of related large-model question answering systems has encountered great obstacles.In order to build an intelligent question and answering system in the electric power field,an intelligent question and answering system for electric power know-ledge base ChatPower based on a large language model is proposed.In order to ensure the Q&A effect,ChatPower fully utilizes data from all aspects of power management,sorts out and integrates a large amount of power professional knowledge through semantic understanding,and carefully designs and constructs a large-scale power system knowledge base.The knowledge base co-vers power-related rules and regulations,production safety management systems,and power generation equipment failure know-ledge.In addition,by referring to the retrieved electricity knowledge,ChatPower significantly reduces the problem of model illusion in question and answering,and introduces a method that combines BM25 retrieval,dense retrieval and rerank in the retrieval system,effectively reducing the the inaccuracy of relying solely on vector library retrieval.At the same time,ChatPower combines prompt engineering technology based on large models to improve the orderliness of generating responses to rules and regulations type questions.In order to evaluate the Q&A system,a test data set for electric power knowledge question and answering is constructed,and ChatPower is tested and verified.The test results show that the electric power knowledge base question and answe-ring system ChatPower based on a large language model can effectively improve the accuracy of retrieval of power-related know-ledge and Q&A.

Key words: Large language model, Knowledge base question answering system, Information retrieval, Natural language generation

中图分类号: 

  • TP391
[1]MOLLA D,VICEDO J L.Question Answering in Restricted Domains:An Overview[J].Computational Linguistics,2007,33(1):41-61.
[2]BOLLACKER K,EVANS C,PARITOSH P,et al.Freebase:A collaboratively created graph database for structuring human knowledge[C]//Proceedings of the ACM SIGMOD Interna-tional Conference on Management of Data(SIGMOD).Canada:ACM,2008:1247-1250.
[3]LEHMANN J,ISELE R,JAKOB M,et al.DBpediaa large-scale,multilingual knowledge base extracted from Wikipedia[J].Semantic Web,2015,6(2):167-195.
[4]PELLISSIER T T,VELIČKOVIĆ D,SCHAFFERT S,et al.From Freebase to Wikidata:The Great Migration[C]//Procee-dings of the International World Wide Web Conference(WWW).Canada:ACM,2016:1419-1428.
[5]NOY N F,GAO Y,JAIN A,et al.Industry-scale KnowledgeGraphs:Lessons and Challenges[J].ACM Queue,2019,17(2):20.
[6]ARUN K.Making search easier:How Amazon's Product Graph is helping customers find products more easily.Amazon Blog.[EB/OL].https://blog.aboutamazon.com/innovation/making-search-easier.
[7]SAURABH S.Bring rich knowledge of people,places,thingsand local businesses to your apps[EB/OL].https://blogs.bing.com/search-quality-insights/2017-07/bring-rich-knowledge-of-people-places-things-andlocal-businesses-to-your-apps.
[8]ABUJABAL A, YAHYA M, RIEDEWALD M,et al.Automated template generation for question answering over knowledge graphs[C]//Proceedings of the International World Wide Web Conference(WWW).Perth:ACM,2017:1191-1200.
[9]LIU A,HUANG Z,LU H,et al.BB-KBQA:BERT-basedknowledge base question answering[C]//Proceedings of the 18th China National Conference on Chinese Computational Linguistics.Kunming:Springer,2019:81-92.
[10]CHEN B, SUN L, HAN X P.Sequence-to-Action:End-to-end semantic graph generation for semantic parsing[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Melbourne:ACL, 2018:766-777.
[11]BORDES A, CHOPRA S, WESTON J.Question answeringwith subgraph embeddings[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.Doha:ACL,2014:615-620.
[12]YANG H,LIU X Y,DAN W C.FinGPT:Open-Source FinancialLarge Language Models[C]//FinLLM Symposium at IJCAI 2023.2023.
[13]CUI J,LI Z,YAN Y,et al.ChatLaw:Open-Source Legal Large Language Model with Integrated External Knowledge Bases[J].arXiv:2306.16092,2023.
[14]DEVLIN J,CHANG M, LEE K,et al.BERT:Pre-training ofDeep bidirectional Transformers for Language Understanding[J].Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2019,1(2):4171-4186.
[15]RADFORD A,WU J,CHILD R,et al.Language models are unsupervised multitask learners[EB/OL].https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf.
[16]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isAll You Need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems(NIPS).Montreal,Canada:NIPS,2017:5998-6008.
[17]OpenAI.Our approach to alignment research[EB/OL].https://openai.com/blog/our-approach-to-alignment-research.
[18]OPENA I,JOSH A,STEVEN A,et al.GPT-4 Technical Report[J].arXiv:2303.08774,2023.
[19]TOUVRON H,LOUIS M,KEVIN S,et al.Llama 2:Openfoundation and fine-tuned chat models[J].arXiv:2307.09288,2023.
[20]ZENG A,LIU X,DU Z X,et al.Glm-130b:An open bilingual pre-trained model[J].arXiv:2210.02414,2022.
[21]WEI X, CUI X Y, CHENG N,et al.Zero-shot information extraction via chatting with chatgpt[J].arXiv:2302.10205,2023.
[22]BROWNT B, MANNB,RYDERN,et al.Language models are few-shot learners[C]//Advances in Neural Information Processing Systems.Curran Associates,Inc,2020:1877-1901.
[23]WEI J,WANG X Z,SCHUURMANS D,et al.Chain-of-thought prompting elicits reasoning in large language models[C]//Advances in Neural Information Processing Systems.Curran Associates,Inc,2022:24824-24837.
[24]WELLECK S,KULIKOV I,ROLLER S, et al.Neural text ge-neration with unlikelihood training[C]//International Confe-rence on Learning Representations.Addis Ababa:OpenReview.net:2020.
[25]YAO S Y,YU D,ZHAO J,et al.Tree of thoughts:Deliberate problem solving with large language models[J].arXiv:2305.10601,2023.
[26]YAO Y,LI Z C,ZHAO H.Beyond Chain-of-Thought,Effective Graph-of-Thought Reasoning in Large Language Models[J].arXiv:2305.16582,2023.
[27]CHEN W H,MA X G,WANG X Y,et al.Program of thoughts prompting:Disentangling computation from reasoning for numerical reasoning tasks[J].arXiv:2211.12588,2022.
[28]LUO K,LIN F,LUO X,et al.Knowledge base question answe-ring via encoding of complex query graphs[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels:Association for Computational Linguistics,2018:2185-2194.
[29]ZHU S,CHENG X,SU S.Knowledge-based question answering by tree-to-sequence learning[J].Neurocomputing,2020,372:64-72.
[30]MILLER A H,FISCH A,DODGE J,et al.Key-value memorynetworks for directly reading documents[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Austin:The Association for Computational Linguistics,2016:1400-1409.
[31]SHI J X,CAO S L,HOU L,et al.TransferNet:An effective and transparent framework for multi-hop question answering over relation graph[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.Punta Cana:The Association for Computational Linguistics,2021:4143-4158.
[32]LUO H R,EH H,TANG Z C,et al.ChatKBQA:A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models[J].arXiv:2310.08975,2023.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发