计算机科学 ›› 2022, Vol. 49 ›› Issue (1): 140-145.doi: 10.11896/jsjkx.210100177

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于多头注意力机制的用户窃电行为检测

肖丁, 张玙璠, 纪厚业   

  1. 北京邮电大学计算机学院 北京100876
  • 收稿日期:2021-01-22 修回日期:2021-07-23 出版日期:2022-01-15 发布日期:2022-01-18
  • 通讯作者: 张玙璠(zhangyufan@bupt.edu.cn)
  • 作者简介:dxiao@bupt.edu.cn
  • 基金资助:
    国家自然科学基金(U20B2045,61772082)

Electricity Theft Detection Based on Multi-head Attention Mechanism

XIAO Ding, ZHANG Yu-fan, JI Hou-ye   

  1. School of Computer Science,Beijing University of Posts and Telecommunications,Beijing 100876,China
  • Received:2021-01-22 Revised:2021-07-23 Online:2022-01-15 Published:2022-01-18
  • About author:XIAO Ding,born in 1966,lecture.His main research interests include software engineering and artificial intelligence.
    ZHANG Yu-fan,born in 1996,postgra-duate.Her main research interests include power systems and artificial intelligence.
  • Supported by:
    National Natural Science Foundation of China(U20B2045,61772082)

摘要: 窃电对社会和经济发展造成了重大损害。如何基于电力大数据来检测用户恶意窃电行为,已受到学术界和工业界的广泛关注。针对传统方法依赖于手工特征、行为序列表征不足和检测精度差等问题,提出了一种基于多头注意力机制的窃电检测模型(Electricity Theft Detection Based on Multi-Head Attention,ETD-MHA)。该模型基于双向门控循环神经网络(Bidirectional Gated Recurrent Unit,BiGRU)充分捕获用户用电行为序列的时序特征,引入多头注意力机制来进一步增强关键特征的区分度,并通过加深网络来提高学习效果。在爱尔兰和中国国家电网智能电表数据集上进行了大量的实验,结果表明,相比传统的逻辑回归(Linear Regression,LR)、支持向量机(Support Vector Machine,SVM)、随机森林(Random Forest,RF)等多种算法,所提模型展现出了明显的优势。例如,在爱尔兰智能电表数据集上,其AUC值相比LR算法最高提升了34.6%。

关键词: 多头注意力机制, 门控循环神经网络, 窃电检测, 深度学习, 智能电网

Abstract: Electricity theft causes significant damage to social and economic development.How to detect malicious electricity theft based on power big data has been widely concerned by academia and industry.Aiming at the problems of traditional methods relying on manual features,insufficient behavior sequence representation,poor detection accuracy,etc.,this paper proposes an electricity theft detection model based on multi-head attention mechanism (ETD-MHA).The bidirectional gated recurrent unit is used to fully capture the time features of the electricity consumption behavior sequence,and the distinction of key features is gradually enhanced in the multi-head attention mechanism,and finally,the learning effect is improved by deepening the networks.Extended experiments are conducted on the smart meter datasets of Ireland and China State Grid.The results show that the proposed method achieves better performance compared with the linear regression (LR),support vector machine (SVM),random forest (RF),and other traditional algorithms.For example,the AUC value of the proposed model is improved by up to 34.6%compared to the LR algorithm.

Key words: Deep learning, Electricity theft detection, Gated recurrent neural network, Multi-head attention mechanism, Smart grids

中图分类号: 

  • TP183
[1]DEPURU S S S R,WANG L,DEVABHAKTUNI V.Electricity theft:Overview,issues,prevention and a smart meter based approach to control theft[J].Energy Policy,2011,39(2):1007-1015.
[2]NAVANI J P,SHARMA N K,SAPRA S.Technical and Non-Technical Losses in Power System and Its Economic Consequence in Indian Economy[J].International Journal of Electro-nics & Computer Science Engineering,2012,1(2):757-761.
[3]AKOUEMO H N,POVINELLI R J.Probabilistic anomaly detection in natural gas time series data[J].International Journal of Forecasting,2016,32(3):948-956.
[4]NIZAR A H,DONG Z Y,WANG Y.Power Utility Nontechnical Loss Analysis With Extreme Learning Machine Method[J].IEEE Transactions on Power Systems,2008,23(3):946-955.
[5]NAGI J,YAP K S,TIONG S K,et al.Nontechnical Loss Detection for Metered Customers in Power Utility Using Support Vector Machines[J].IEEE Transactions on Power Delivery,2010,25(2):1162-1171.
[6]DEPURU S S S R,WANG L,DEVABHAKTUNI V,et al.A hybrid neural network model and encoding technique for enhanced classification of energy consumption data[C]//2011 IEEE Power and Energy Society General Meeting.IEEE,2011:1-8.
[7]DEPURU S S S R,WANG L,DEVABHAKTUNI V.Supportvector machine based data classification for detection of electricity theft[C]//2011 IEEE/PES Power Systems Conference and Exposition.IEEE,2011:1-8.
[8]NAGI J,YAP K S,TIONG S K,et al.Improving SVM-BasedNontechnical Loss Detection in Power Utility Using the Fuzzy Inference System[J].IEEE Transactions on Power Delivery,2011,26(2):1284-1285.
[9]JOKAR P,ARIANPOO N,LEUNG V C.Electricity Theft Detection in AMI Using Customers' Consumption Patterns[J].IEEE Transactions on Smart Grid,2017,7(1):216-226.
[10]LUONG M T,PHAM H,MANNING C D.Effective approaches to attention-based neural machine translation[J].arXiv:1508.04025,2015.
[11]CHEN P,LIU S,SHI C,et al.NeuCast:Seasonal Neural Forecast of Power Grid Time Series[C]//Twenty-Seventh International Joint Conference on Artificial Intelligence IJCAI-18.International Joint Conferences on Artificial Intelligence Organization.2018:3315-3321.
[12]ZHUANG S J,YU Z Y,GUO W Z,et al.Short Term Load Forecasting via Zoneout-based Multi-time Scale Recurrent Neural Network[J].Computer Science,2020,47(9):105-109.
[13]HU T Y,GUO Q L,SUN H B.Nontechnical loss detectionbased on stacked uncorrelating autoencoder and support vector machine[J].Automation of Electric Power Systems,2019,43(1):119-127.
[14]MENG Z,XU X.A Hybrid Short-Term Load ForecastingFramework with an Attention-Based Encoder-Decoder Network Based on Seasonal and Trend Adjustment[J].Energies,2019,12(24):4612.
[15]ZHENG Z,YANG Y,NIU X,et al.Wide and Deep Convolutio-nal Neural Networks for Electricity-Theft Detection to Secure Smart Grids[J].IEEE Transactions on Industrial Informatics,2017:14(4):1606-1615.
[16]NABIL M,MAHMOUD M,ISMAIL M,et al.Deep recurrentelectricity theft detection in AMI networks with evolutionary hyper-parameter tuning[C]//2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber,Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData).IEEE,2019:1002-1008.
[17]CHEN Z,MENG D,ZHANG Y,et al.Electricity theft detection using deep bidirectional recurrent neural network[C]//2020 22nd International Conference on Advanced Communication Technology (ICACT).IEEE,2020:401-406.
[18]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[J].arXiv:1409.0473,2014.
[19]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[J].arXiv:1706.03762,2017.
[20]ISSDA.Data from the commission for energy regulation[EB/OL].[2019-07-01].http://www.ucd.ie/issda/data/commissionforenergyregulationcer/.
[1] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[2] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[3] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[4] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[5] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[6] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[7] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[8] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[9] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[10] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[11] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[12] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[13] 王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰.
基于多源迁移学习的大坝裂缝检测
Dam Crack Detection Based on Multi-source Transfer Learning
计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[14] 楚玉春, 龚航, 王学芳, 刘培顺.
基于YOLOv4的目标检测知识蒸馏算法研究
Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4
计算机科学, 2022, 49(6A): 337-344. https://doi.org/10.11896/jsjkx.210600204
[15] 祝文韬, 兰先超, 罗唤霖, 岳彬, 汪洋.
改进Faster R-CNN的光学遥感飞机目标检测
Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN
计算机科学, 2022, 49(6A): 378-383. https://doi.org/10.11896/jsjkx.210300121
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!