计算机科学 ›› 2024, Vol. 51 ›› Issue (11A): 240300191-6.doi: 10.11896/jsjkx.240300191
卓明松, 莫凌飞
ZHUO Mingsong, MO Lingfei
摘要： 脉冲神经网络(Spiking Neural Network,SNN)被认为是最符合生物大脑机制的类脑计算模型,凭借其事件驱动、高能效、可解释等特点吸引了越来越多的研究关注。然而,由于脉冲的二值输出与不可微分性,SNN的训练方法仍存在一定空缺。于是借鉴皮层记忆单元通过局部网络存储记忆信息的方式,提出一种基于多子网络预训练的脉冲神经网络分类方法。该方法使用样本标签信息优化了脉冲序列特征提取过程,采用改进的脉冲时间依赖可塑性学习规则预训练多个单类别特征提取子网络,并将预训练后的子网络进行无监督特征融合,有效提高了网络的特征分类能力。此外,在权重可视化与t-SNE可视化工具的帮助下,分析了方法的有效性。所提方法在MNIST与Fashion-MNIST数据集上分别取得了97.40%与88.81%的分类准确度。
中图分类号:
| [1]KRIZHEVSKY A,SUTSKEVER I,HINTONG E.Imagenetclassification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90. [2]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A Convolutional Neural Network for Modelling Sentences[C]//Annual meeting of the Association for Computational Linguistics.2014. [3]SILVER D,SCHRITTWIESER J,SIMONYAN K,et al.Mastering the game of go without human knowledge[J].Nature,2017,550(7676):354-359. [4]WANG X Q,ZENG Y H,HAN D M,et al.Brain like computing based on spiking neural networks [J].Journal of Beijing Institute of Technology,2019,45(12):1277-1286. [5]MAASS W.Networks of spiking neurons:the third generation of neural network models[J].Neural Networks,1997,10(9):1659-1671. [6]CAPORALE N,DAN Y.Spike timing-dependent plasticity:aHebbian learning rule[J].Annu.Rev.Neurosci.,2008,31:25-46. [7]LESTER D,FURBER S.Spinnaker:Distributed computer engineering for neuromorphics[M]//Neural Nets WIRN11.IOS Press,2011:324-331. [8]PEI J,DENG L,SONG S,et al.Towards artificial general intelligence with hybrid Tianjic chip architecture[J].Nature,2019,572(7767):106-111. [9]DAVIES M,SRINIVASA N,LIN T H,et al.Loihi:A neuromorphic manycore processor with on-chip learning[J].IEEE Micro,2018,38(1):82-99. [10]MA D,SHEN J,GU Z,et al.Darwin:A neuromorphic hardware co-processor based on spiking neural networks[J].Journal of Systems Architecture,2017,77:43-51. [11]BOUVIER M,VALENTIAN A,MESQUIDA T,et al.Spiking neural networks hardware implementations and challenges:A survey[J].ACM Journal on Emerging Technologies in Computing Systems(JETC),2019,15(2):1-35. [12]DIEHL P U,COOK M.Unsupervised learning of digit recognition using spike-timing-dependent plasticity[J].Frontiers in Computational Neuroscience,2015,9:99. [13]KOHONEN T.The self-organizing map[J].Proceedings of the IEEE,1990,78(9):1464-1480. [14]TEYLER T J,RUDY J W.The hippocampal indexing theoryand episodic memory:updating the index[J].Hippocampus,2007,17(12):1158-1169. [15]PONULAK F.ReSuMe-new supervised learning method for spiking neural networks.Institute of Control and Information Engineering,Poznoń University of Technology[R].Tech.rep,2005. [16]ZHANG T,ZENG Y,ZHAO D,et al.A plasticity-centric ap-proach to train the non-differential spiking neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018. [17]KOCH C,SEGEV I.Methods in neuronal modeling:from ions to networks[M].MIT press,1998. [18]HODGKIN A L,HUXLEYA F.A quantitative description ofmembrane current and its application to conduction and excitation in nerve[J].The Journal of Physiology,1952,117(4):500. [19]KRUG M,MÜLLER-WELDE P,WAGNERM,et al.Functional plasticity in two afferent systems of the granule cells in the rat dentate area:frequency-related changes,long-term potentiation and heterosynaptic depression[J].Brain Research,1985,360(1/2):264-272. [20]QUERLIOZ D,BICHLER O,DOLLFUS P,et al.Immunity to device variations in a spiking neural network with memristive nanodevices[J].IEEE Transactions on Nanotechnology,2013,12(3):288-295. [21]HAO Y,HUANG X,DONG M,et al.A biologically plausiblesupervised learning method for spiking neural networks using the symmetric STDP rule[J].Neural Networks,2020,121:387-395. [22]MO L,WANG G,LONG E,et al.ALSA:Associative Learning Based Supervised Learning Algorithm for SNN[J].Frontiers in Neuroscience,2022,16:838832. [23]LIU F,ZHAO W,CHEN Y,et al.SSTDP:Supervised spike ti-ming dependent plasticity for efficient spiking neural network training[J].Frontiers in Neuroscience,2021,15:756876. [24]TAVANAEI A,MAIDA A.BP-STDP:Approximating back-propagation using spike timing dependent plasticity[J].Neurocomputing,2019,330:39-47. [25]PUTRA R V W,SHAFIQUE M.Fspinn:An optimizationframework for memory-efficient and energy-efficient spiking neural networks[J].IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems,2020,39(11):3601-3613. [26]RASTOGI M,LU S,ISLAM N,et al.On the self-repair role of astrocytes in STDP enabled unsupervised SNNs[J].Frontiers in Neuroscience,2021,14:603796. [27]SHI M,ZHANG T,ZENG Y.A curiosity-based learning method for spiking neural networks[J].Frontiers in Computational Neuroscience,2020,14:7. [28]ZHAO D,ZENG Y,ZHANG T,et al.GLSNN:A multi-layer spiking neural network based on global feedback alignment and local STDP plasticity[J].Frontiers in Computational Neuroscience,2020,14:576841. |
|
||
首页