计算机科学 ›› 2023, Vol. 50 ›› Issue (1): 334-341.doi: 10.11896/jsjkx.211100001

• 信息安全 • 上一篇    下一篇

一种基于强化学习的口令猜解模型

李小玲1, 吴昊天1, 周涛1, 鲁辉2   

  1. 1 华南理工大学计算机科学与工程学院 广州 510006
    2 广州大学网络空间先进技术研究院 广州 510006
  • 收稿日期:2021-11-01 修回日期:2022-03-27 出版日期:2023-01-15 发布日期:2023-01-09
  • 通讯作者: 吴昊天(wuht@scut.edu.cn)
  • 作者简介:202021044839@mail.scut.edu.cn
  • 基金资助:
    广东省重点领域研发计划(2019B010137004);广东省自然科学基金面上项目(2021A1515011798)

Password Guessing Model Based on Reinforcement Learning

LI Xiaoling1, WU Haotian1, ZHOU Tao1, LU Hui2   

  1. 1 School of Computer Science and Engineering,South China University of Technology,Guangzhou 510006,China
    2 Cyberspace Institute of Advanced Technology,Guangzhou University,Guangzhou 510006,China
  • Received:2021-11-01 Revised:2022-03-27 Online:2023-01-15 Published:2023-01-09
  • About author:LI Xiaoling,born in 1998,postgra-duate.Her main research interests include deep learning based password guessing and so on.
    WU Haotian,born in 1980,Ph.D,asso-ciate professor.His main research intere-sts include information hiding,privacy preservation,password guessing and blockchain.
  • Supported by:
    R & D Project in Key Areas of Guangdong Province,China(2019B010137004) and Natural Science Foundation of Guangdong Province,China(2021A1515011798).

摘要: 口令猜解是口令安全研究的重要方向之一。基于生成式对抗网络(Generative Adversarial Network,GAN)的口令猜解是近几年提出的一种新方法,其通过判别器对生成口令的评判结果来指导生成器的更新,进而生成口令猜测集。然而由于判别器对生成器的指导不足,现有的基于GAN的口令猜解模型的猜解效率较低。针对这个问题,提出了一种基于强化学习Actor-Critic算法改进的GAN口令猜解模型AC-Pass。AC-Pass模型通过Critic网络和判别器输出的奖赏共同指导Actor网络每一时间步生成策略的更新,实现了对口令序列生成过程的强化指导。将AC-Pass模型应用到RockYou,LinkedIn和CSDN口令集进行实验,并与PCFG模型、已有基于GAN的口令猜解模型PassGAN和seqGAN进行比较。实验结果表明,无论是同源测试集还是异源测试集,AC-Pass模型在9×108猜测集上的口令破解率均高于PassGAN和seqGAN;且当测试集与训练集之间的口令空间分布差异较大时,AC-Pass表现出了优于PCFG的口令猜解性能;另外,AC-Pass模型有较大的口令输出空间,其破解率随着口令猜测集的增大而提高。

关键词: 口令猜解, 深度学习, 强化学习, Actor-Critic算法, 生成式对抗网络

Abstract: Password guessing is an important research direction in password security.Password guessing based on generative adversarial network(GAN) is a new method proposed in recent years,which guides the update of the generator according to evaluation results on passwords generated by the discriminator.Consequently,password guessing sets can be generated with trained GANs.However,the existing GAN-based password guessing models have low efficiency due to inadequate guidance of the discriminator to the generator.To solve this problem,an improved GAN password guessing model AC-Pass based on reinforcement learning Actor-Critic algorithm is proposed.The AC-Pass model guides the update of the generation strategy of the Actor network at each time step through the output rewards of the discriminator and the Critic network,and realizes the reinforce guidance of password sequence generation process.The proposed AC-Pass model is implemented on RockYou,LinkedIn and CSDN data sets and compared with PCFG model and the existing GANs-based password guessing models such as PassGAN and seqGAN.Results on homologous testing sets and heterologous testing sets indicate that password cracking rate of AC-Pass model on the guessing set is higher than that of PassGAN and seqGAN.Moreover,AC-Pass shows better guessing performance than PCFG when the password spatial distribution between the testing set and the training set is significant.In addition,the AC-Pass model has a large password output space.As the size of password guessing set increases,the cracking rate continues to rise.

Key words: Password guessing, Deep learning, Reinforcement learning, Actor-Critic algorithm, Generative adversarial network

中图分类号: 

  • TP309
[1]HAN W L,YUAN L,LI S S,et al.An Efficient Algorithm to Generate Password Sets Based on Samples[J].Chinese Journal of Computers,2017,40(5):1151-1167.
[2]LIU G S,QIU W D,MENG K,et al.Password Vulnerability Assessment and Recovery Based on Ruels Mined from Large-Scale Real Data[J].Chinese Journal of Computers,2016,39(3):454-467.
[3]XIE Z J,ZHANG M,LI Z H,et al.Analysis of Large-scale Real User Password Data Based on Cracking Algorithms[J].Computer Science,2020,47(11):48-54.
[4]WANG D,ZOU Y K,TAO Y,et al.Password Guessing Model Based on Recurrent Neural Networks and Generative Adversa-rial Networks[J].Chinese Journal of Computers,2021,44(8):1519-1534.
[5]YU L,ZHANG W,WANG J,et al.Seqgan:Sequence generative adversarial nets with policy gradient[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2017,31(1),2852-2858.
[6]NARAYANAN A,SHMATIKOV V.Fast dictionary attacks on passwords using time-space tradeoff[C]//Proceedings of the 12th ACM Conference on Computer and communications security.2005:364-372.
[7]WEIR M,AGGARWAL S,DE MEDEIROS B,et al.Password cracking using probabilistic context-free grammars[C]//2009 30th IEEE Symposium on Security and Privacy.IEEE,2009:391-405.
[8]TANSEY W.Improved models for password guessing [EB/OL].https://www.semanticscholar.org/paper/ImprovedMo-dels-for-Password-Guessing-Tansey/3451ac7f102da12e1197c681b77d368ba3b19ac9.
[9]DÜRMUTH M,ANGELSTORF F,CASTELLUCCIA C,et al.OMEN:Faster password guessing using an ordered markov enumerator[C]//International Symposium on Engineering Secure Software and Systems.Cham:Springer,2015:119-132.
[10]HOUSHMAND S,AGGARWAL S,FLOOD R.Next gen PCFG password cracking [J].IEEE Transactions on Information Forensics and Security,2015,10(8):1776-1791.
[11]WANG D,WANG P.The emperor's new password creationpolicies[C]//European Symposium on Research in Computer Security.Cham:Springer,2015:456-477.
[12]LI Y,WANG H,SUN K.A study of personal information in human-chosen passwords and its security implications[C]//IEEE INFOCOM 2016-the 35th Annual IEEE International Confe-rence on Computer Communications.IEEE,2016:1-9.
[13]WANG D,ZHANG Z,WANG P,et al.Targeted online password guessing:An underestimated threat[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security.2016:1242-1254.
[14]MELICHER W,UR B,SEGRETI S M,et al.Fast,lean,and accurate:Modeling password guessability using neural networks[C]//25th {USENIX} Security Symposium({USENIX} Security 16).2016:175-191.
[15]XU L,GE C,QIU W,et al.Password guessing based on LSTM recurrent neural networks[C]//2017 IEEE International Conference on Computational Science and Engineering(CSE) and IEEE International Conference on Embedded and Ubiquitous Computing(EUC).IEEE,2017:785-788.
[16]XIA Z Y,YI P,LIU Y,et al.GENPass:A multi-source deeplearning model for password guessing[J].IEEE Transactions on Multimedia,2019,22(5):1323-1332.
[17]HITAJ B,GASTI P,ATENIESE G,et al.Passgan:A deeplearning approach for password guessing[C]//International Conference on Applied Cryptography and Network Security.Cham:Springer,2019:217-237.
[18]GULRAJANI I,AHMED F,ARJOVSKY M,et al.Improvedtraining of wasserstein gans [J].arXiv:1704.00028,2017.
[19]NAM S,JEON S,KIM H,et al.Recurrent gans password cra-cker for iot password security enhancement [J].Sensors,2020,20(11):3106.
[20]PASQUINI D,GANGWAL A,ATENIESE G,et al.Improving password guessing via representation learning[C]//2021 IEEE Symposium on Security and Privacy(SP).IEEE,2021:1382-1399.
[21]MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning [J].Nature,2015,518(7540):529-533.
[22]SILVER D,LEVER G,HEESS N,et al.Deterministic policygradient algorithms[C]//International Conference on Machine Learning.PMLR,2014:387-395.
[23]KONDA V R,TSITSIKLIS J N.Actor-critic algorithms[C]//Advances in Neural Information Processing Systems.2000:1008-1014.
[24]LILLICRAP T P,HUNT J J,PRITZEL A,et al.Continuouscontrol with deep reinforcement learning [J].arXiv:1509.02971,2015.
[25]MNIH V,BADIA A P,MIRZA M,et al.Asynchronous methodsfor deep reinforcement learning[C]//International Conference on Machine Learning.PMLR,2016:1928-1937.
[26]YANG S M,SHAN Z,DING Y,et al.Survey of Research on Deep Reinforcement Learning[J].Computer Engineering,2021,47(12):19-29.
[27]LIN K,LI D,HE X,et al.Adversarial ranking for language ge-neration [J].arXiv:1705.11001,2017.
[28]FEDUS W,GOODFELLOW I,DAI A M.Maskgan:better text generation via filling in the_ [J].arXiv:1801.07736,2018.
[29]ZHANG X,LECUN Y.Text understanding from scratch [J].arXiv:1502.01710,2015.
[1] 蔡肖, 陈志华, 盛斌.
基于移位窗口金字塔Transformer的遥感图像目标检测
SPT:Swin Pyramid Transformer for Object Detection of Remote Sensing
计算机科学, 2023, 50(1): 105-113. https://doi.org/10.11896/jsjkx.211100208
[2] 王斌, 梁宇栋, 刘哲, 张超, 李德玉.
亮度自调节的无监督图像去雾与低光图像增强算法研究
Study on Unsupervised Image Dehazing and Low-light Image Enhancement Algorithms Based on Luminance Adjustment
计算机科学, 2023, 50(1): 123-130. https://doi.org/10.11896/jsjkx.211100058
[3] 李雪辉, 张拥军, 史殿习, 徐化池, 史燕燕.
融合注意力特征的无锚框视觉目标跟踪方法
AFTM:Anchor-free Object Tracking Method with Attention Features
计算机科学, 2023, 50(1): 138-146. https://doi.org/10.11896/jsjkx.211000083
[4] 孙凯丽, 罗旭东, 罗有容.
预训练语言模型的应用综述
Survey of Applications of Pretrained Language Models
计算机科学, 2023, 50(1): 176-184. https://doi.org/10.11896/jsjkx.220800223
[5] 黄昱洲, 王立松, 秦小麟.
一种基于深度强化学习的无人小车双层路径规划方法
Bi-level Path Planning Method for Unmanned Vehicle Based on Deep Reinforcement Learning
计算机科学, 2023, 50(1): 194-204. https://doi.org/10.11896/jsjkx.220500241
[6] 郑诚, 梅亮, 赵伊研, 张苏航.
基于双向注意力机制和门控图卷积网络的文本分类方法
Text Classification Method Based on Bidirectional Attention and Gated Graph Convolutional Networks
计算机科学, 2023, 50(1): 221-228. https://doi.org/10.11896/jsjkx.211100095
[7] 荣欢, 钱敏峰, 马廷淮, 孙圣杰.
基于先验知识图谱的多代理被遮挡目标类别推理模型
Novel Class Reasoning Model Towards Covered Area in Given Image Based on InformedKnowledge Graph Reasoning and Multi-agent Collaboration
计算机科学, 2023, 50(1): 243-252. https://doi.org/10.11896/jsjkx.220700112
[8] 徐平安, 刘全.
基于相似度约束的双策略蒸馏深度强化学习方法
Deep Reinforcement Learning Based on Similarity Constrained Dual Policy Distillation
计算机科学, 2023, 50(1): 253-261. https://doi.org/10.11896/jsjkx.211100167
[9] 张启阳, 陈希亮, 张巧.
基于轨迹感知的稀疏奖励探索方法
Sparse Reward Exploration Method Based on Trajectory Perception
计算机科学, 2023, 50(1): 262-269. https://doi.org/10.11896/jsjkx.220700010
[10] 魏楠, 魏祥麟, 范建华, 薛羽, 胡永扬.
面向频谱接入深度强化学习模型的后门攻击方法
Backdoor Attack Against Deep Reinforcement Learning-based Spectrum Access Model
计算机科学, 2023, 50(1): 351-361. https://doi.org/10.11896/jsjkx.220800269
[11] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[12] 熊丽琴, 曹雷, 赖俊, 陈希亮.
基于值分解的多智能体深度强化学习综述
Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization
计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[13] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[14] 刘兴光, 周力, 刘琰, 张晓瀛, 谭翔, 魏急波.
基于边缘智能的频谱地图构建与分发方法
Construction and Distribution Method of REM Based on Edge Intelligence
计算机科学, 2022, 49(9): 236-241. https://doi.org/10.11896/jsjkx.220400148
[15] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!