Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (12): 286-292.doi: 10.11896/jsjkx.240300104

• Artificial Intelligence • Previous Articles     Next Articles

Construction and Evaluation of Intelligent Question Answering System for Electric Power Knowledge Base Based on Large Language Model

ZHANG Jinying1, WANG Tiankun1, YAO Changying2, XIE Hua2, CHAI Linzheng3, LIU Shukai3, LI Tongliang4, LI Zhoujun3   

  1. 1 Guodian Power Development Co., Ltd., Dalian, Liaoning 116000, China
    2 Guoneng Xinkong Internet Technology Co., Ltd., Beijing 100039, China
    3 School of Computer Science and Engineering, Beihang University, Beijing 100191, China
    4 Computer School, Beijing Information Science and Technology University, Beijing 102206, China
  • Received:2024-03-14 Revised:2024-05-15 Online:2024-12-15 Published:2024-12-10
  • About author:ZHANG Jinying,born in 1984,Ph.D.His main research interests include artificial intelligence,distributed control systems,and fieldbus control systems.
    LI Zhoujun,born in 1963,Ph.D,professor,is a member of CCF(No.06618S).His main research interests include artificial intelligence,natural language processing,network and information secu-rity.
  • Supported by:
    Natural Science Foundation of China(62276017) and State Key Laboratory of Complex& Critical Software Environment(SKLSDE-2021ZX-18).

PDF (PC)

388

Abstract: Large language model is a major breakthrough in the field of natural language processing in recent years and have become a new paradigm for research in this field.In vertical fields such as finance and law,intelligent question and answering systems based on large models in vertical fields such as FinGPT and ChatLaw have promoted the academic research and application of large model technology in related fields.However,due to the lack of relevant high-quality data in the electric power field,the construction of related large-model question answering systems has encountered great obstacles.In order to build an intelligent question and answering system in the electric power field,an intelligent question and answering system for electric power know-ledge base ChatPower based on a large language model is proposed.In order to ensure the Q&A effect,ChatPower fully utilizes data from all aspects of power management,sorts out and integrates a large amount of power professional knowledge through semantic understanding,and carefully designs and constructs a large-scale power system knowledge base.The knowledge base co-vers power-related rules and regulations,production safety management systems,and power generation equipment failure know-ledge.In addition,by referring to the retrieved electricity knowledge,ChatPower significantly reduces the problem of model illusion in question and answering,and introduces a method that combines BM25 retrieval,dense retrieval and rerank in the retrieval system,effectively reducing the the inaccuracy of relying solely on vector library retrieval.At the same time,ChatPower combines prompt engineering technology based on large models to improve the orderliness of generating responses to rules and regulations type questions.In order to evaluate the Q&A system,a test data set for electric power knowledge question and answering is constructed,and ChatPower is tested and verified.The test results show that the electric power knowledge base question and answe-ring system ChatPower based on a large language model can effectively improve the accuracy of retrieval of power-related know-ledge and Q&A.

Key words: Large language model, Knowledge base question answering system, Information retrieval, Natural language generation

CLC Number: 

  • TP391
[1]MOLLA D,VICEDO J L.Question Answering in Restricted Domains:An Overview[J].Computational Linguistics,2007,33(1):41-61.
[2]BOLLACKER K,EVANS C,PARITOSH P,et al.Freebase:A collaboratively created graph database for structuring human knowledge[C]//Proceedings of the ACM SIGMOD Interna-tional Conference on Management of Data(SIGMOD).Canada:ACM,2008:1247-1250.
[3]LEHMANN J,ISELE R,JAKOB M,et al.DBpediaa large-scale,multilingual knowledge base extracted from Wikipedia[J].Semantic Web,2015,6(2):167-195.
[4]PELLISSIER T T,VELIČKOVIĆ D,SCHAFFERT S,et al.From Freebase to Wikidata:The Great Migration[C]//Procee-dings of the International World Wide Web Conference(WWW).Canada:ACM,2016:1419-1428.
[5]NOY N F,GAO Y,JAIN A,et al.Industry-scale KnowledgeGraphs:Lessons and Challenges[J].ACM Queue,2019,17(2):20.
[6]ARUN K.Making search easier:How Amazon's Product Graph is helping customers find products more easily.Amazon Blog.[EB/OL].https://blog.aboutamazon.com/innovation/making-search-easier.
[7]SAURABH S.Bring rich knowledge of people,places,thingsand local businesses to your apps[EB/OL].https://blogs.bing.com/search-quality-insights/2017-07/bring-rich-knowledge-of-people-places-things-andlocal-businesses-to-your-apps.
[8]ABUJABAL A, YAHYA M, RIEDEWALD M,et al.Automated template generation for question answering over knowledge graphs[C]//Proceedings of the International World Wide Web Conference(WWW).Perth:ACM,2017:1191-1200.
[9]LIU A,HUANG Z,LU H,et al.BB-KBQA:BERT-basedknowledge base question answering[C]//Proceedings of the 18th China National Conference on Chinese Computational Linguistics.Kunming:Springer,2019:81-92.
[10]CHEN B, SUN L, HAN X P.Sequence-to-Action:End-to-end semantic graph generation for semantic parsing[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Melbourne:ACL, 2018:766-777.
[11]BORDES A, CHOPRA S, WESTON J.Question answeringwith subgraph embeddings[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.Doha:ACL,2014:615-620.
[12]YANG H,LIU X Y,DAN W C.FinGPT:Open-Source FinancialLarge Language Models[C]//FinLLM Symposium at IJCAI 2023.2023.
[13]CUI J,LI Z,YAN Y,et al.ChatLaw:Open-Source Legal Large Language Model with Integrated External Knowledge Bases[J].arXiv:2306.16092,2023.
[14]DEVLIN J,CHANG M, LEE K,et al.BERT:Pre-training ofDeep bidirectional Transformers for Language Understanding[J].Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2019,1(2):4171-4186.
[15]RADFORD A,WU J,CHILD R,et al.Language models are unsupervised multitask learners[EB/OL].https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf.
[16]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isAll You Need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems(NIPS).Montreal,Canada:NIPS,2017:5998-6008.
[17]OpenAI.Our approach to alignment research[EB/OL].https://openai.com/blog/our-approach-to-alignment-research.
[18]OPENA I,JOSH A,STEVEN A,et al.GPT-4 Technical Report[J].arXiv:2303.08774,2023.
[19]TOUVRON H,LOUIS M,KEVIN S,et al.Llama 2:Openfoundation and fine-tuned chat models[J].arXiv:2307.09288,2023.
[20]ZENG A,LIU X,DU Z X,et al.Glm-130b:An open bilingual pre-trained model[J].arXiv:2210.02414,2022.
[21]WEI X, CUI X Y, CHENG N,et al.Zero-shot information extraction via chatting with chatgpt[J].arXiv:2302.10205,2023.
[22]BROWNT B, MANNB,RYDERN,et al.Language models are few-shot learners[C]//Advances in Neural Information Processing Systems.Curran Associates,Inc,2020:1877-1901.
[23]WEI J,WANG X Z,SCHUURMANS D,et al.Chain-of-thought prompting elicits reasoning in large language models[C]//Advances in Neural Information Processing Systems.Curran Associates,Inc,2022:24824-24837.
[24]WELLECK S,KULIKOV I,ROLLER S, et al.Neural text ge-neration with unlikelihood training[C]//International Confe-rence on Learning Representations.Addis Ababa:OpenReview.net:2020.
[25]YAO S Y,YU D,ZHAO J,et al.Tree of thoughts:Deliberate problem solving with large language models[J].arXiv:2305.10601,2023.
[26]YAO Y,LI Z C,ZHAO H.Beyond Chain-of-Thought,Effective Graph-of-Thought Reasoning in Large Language Models[J].arXiv:2305.16582,2023.
[27]CHEN W H,MA X G,WANG X Y,et al.Program of thoughts prompting:Disentangling computation from reasoning for numerical reasoning tasks[J].arXiv:2211.12588,2022.
[28]LUO K,LIN F,LUO X,et al.Knowledge base question answe-ring via encoding of complex query graphs[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels:Association for Computational Linguistics,2018:2185-2194.
[29]ZHU S,CHENG X,SU S.Knowledge-based question answering by tree-to-sequence learning[J].Neurocomputing,2020,372:64-72.
[30]MILLER A H,FISCH A,DODGE J,et al.Key-value memorynetworks for directly reading documents[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Austin:The Association for Computational Linguistics,2016:1400-1409.
[31]SHI J X,CAO S L,HOU L,et al.TransferNet:An effective and transparent framework for multi-hop question answering over relation graph[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.Punta Cana:The Association for Computational Linguistics,2021:4143-4158.
[32]LUO H R,EH H,TANG Z C,et al.ChatKBQA:A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models[J].arXiv:2310.08975,2023.
[1] LIU Yumeng, ZHAO Yijing, WANG Bicong, WANG Chao, ZHANG Baomin. Advances in SQL Intelligent Synthesis Technology [J]. Computer Science, 2024, 51(7): 40-48.
[2] WANG Zhaodan, ZOU Weiqin, LIU Wenjie. Buggy File Identification Based on Recommendation Lists [J]. Computer Science, 2024, 51(6A): 230600088-8.
[3] XU Yiran, ZHOU Yu. Prompt Learning Based Parameter-efficient Code Generation [J]. Computer Science, 2024, 51(6): 61-67.
[4] WANG Xu, LIU Changhong, LI Shengchun, LIU Shuang, ZHAO Kangting, CHEN Liang. Study on Manufacturing Company Automated Chart Analysis Method Based on Natural LanguageGeneration [J]. Computer Science, 2024, 51(4): 174-181.
[5] LI Zhanqi, WU Xinwei, ZHANG Lei, LIU Quanzhou, XIE Hui, XIONG Deyi. Automatic Test Case Generation Method for Automotive Electronic Control System Verification [J]. Computer Science, 2024, 51(12): 63-70.
[6] BAI Jianghao, PIAO Yong. Ensemble Learning Based Open Source License Detection and Compatibility Assessment [J]. Computer Science, 2024, 51(12): 79-86.
[7] ZHU Yangfu, LI Meiling, TAN Jiachen, WU Bin. Study on Text-based Personality Detection－A Review [J]. Computer Science, 2024, 51(12): 209-222.
[8] MA Qimin, LI Xiangmin, ZHOU Yaqian. Large Language Model-based Method for Mobile App Accessibility Enhancement [J]. Computer Science, 2024, 51(12): 223-233.
[9] KA Zuming, ZHAO Peng, ZHANG Bo, FU Xiaoning. Survey of Recommender Systems for Large Language Models [J]. Computer Science, 2024, 51(11A): 240800111-11.
[10] CHEN Jing, ZHOU Gang, LI Shunhang, ZHENG Jiali, LU Jicang, HAO Yaohui. Review of Fake News Detection on Social Media [J]. Computer Science, 2024, 51(11): 1-14.
[11] ZHAO Yue, HE Jinwen, ZHU Shenchen, LI Congyi, ZHANG Yingjie, CHEN Kai. Security of Large Language Models:Current Status and Challenges [J]. Computer Science, 2024, 51(1): 68-71.
[12] LIANG Mingxuan, WANG Shi, ZHU Junwu, LI Yang, GAO Xiang, JIAO Zhixiang. Survey of Knowledge-enhanced Natural Language Generation Research [J]. Computer Science, 2023, 50(6A): 220200120-8.
[13] MA Tinghuai, SUN Shengjie, RONG Huan, QIAN Minfeng. Knowledge Graph-to-Text Model Based on Dynamic Memory and Two-layer Reconstruction Reinforcement [J]. Computer Science, 2023, 50(3): 12-22.
[14] MENG Yiyue, PENG Rong, LYU Qibiao. Text Material Recommendation Method Combining Label Classification and Semantic QueryExpansion [J]. Computer Science, 2023, 50(1): 76-86.
[15] HAN Hong-qi, RAN Ya-xin, ZHANG Yun-liang, GUI Jie, GAO Xiong, YI Meng-lin. Study on Cross-media Information Retrieval Based on Common Subspace Classification Learning [J]. Computer Science, 2022, 49(5): 33-42.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.