基于多子网络预训练的脉冲神经网络分类模型

doi:10.11896/jsjkx.240300191

Abstract

Abstract: Spiking neural network(SNN) is widely regarded as the most biologically plausible model,aligning closely with the mechanisms of the biological brain.It has garnered increasing research attention due to its event-driven nature,high energy efficiency,and interpretability.However,the training methods of SNN still have some limitations due to the binary output and non-differentiability of the spike.This paper proposes a SNN classification method based on multi-subnetwork pre-training,which draws inspiration from the way cortical memory units store memory information through local networks.This approach leverages sample label information to optimize the feature extraction process,employs enhanced spike-timing-dependent-plasticity learning rules for pre-training multiple single-class feature extraction subnetworks,and conducts unsupervised feature fusion on these pre-trained subnetworks to effectively enhance the network's feature classification capability.Furthermore,the effectiveness of this method is analyzed through weight visualization and t-SNE visualization tools.Finally,the classification accuracy of 97.40% and 88.81% is achieved on MNIST and Fashion MNIST datasets,respectively.

Key words: Spiking neural network, Spike-timing-dependent-plasticity, Single-class feature extraction subnetworks, Unsupervised feature fusion, Brain-like computing

CLC Number:

TP389.1

ZHUO Mingsong, MO Lingfei. Spiking Neural Network Classification Model Based on Multi-subnetwork Pre-training[J].Computer Science, 2024, 51(11A): 240300191-6.

References

[1]KRIZHEVSKY A,SUTSKEVER I,HINTONG E.Imagenetclassification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[2]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A Convolutional Neural Network for Modelling Sentences[C]//Annual meeting of the Association for Computational Linguistics.2014.
[3]SILVER D,SCHRITTWIESER J,SIMONYAN K,et al.Mastering the game of go without human knowledge[J].Nature,2017,550(7676):354-359.
[4]WANG X Q,ZENG Y H,HAN D M,et al.Brain like computing based on spiking neural networks [J].Journal of Beijing Institute of Technology,2019,45(12):1277-1286.
[5]MAASS W.Networks of spiking neurons:the third generation of neural network models[J].Neural Networks,1997,10(9):1659-1671.
[6]CAPORALE N,DAN Y.Spike timing-dependent plasticity:aHebbian learning rule[J].Annu.Rev.Neurosci.,2008,31:25-46.
[7]LESTER D,FURBER S.Spinnaker:Distributed computer engineering for neuromorphics[M]//Neural Nets WIRN11.IOS Press,2011:324-331.
[8]PEI J,DENG L,SONG S,et al.Towards artificial general intelligence with hybrid Tianjic chip architecture[J].Nature,2019,572(7767):106-111.
[9]DAVIES M,SRINIVASA N,LIN T H,et al.Loihi:A neuromorphic manycore processor with on-chip learning[J].IEEE Micro,2018,38(1):82-99.
[10]MA D,SHEN J,GU Z,et al.Darwin:A neuromorphic hardware co-processor based on spiking neural networks[J].Journal of Systems Architecture,2017,77:43-51.
[11]BOUVIER M,VALENTIAN A,MESQUIDA T,et al.Spiking neural networks hardware implementations and challenges:A survey[J].ACM Journal on Emerging Technologies in Computing Systems(JETC),2019,15(2):1-35.
[12]DIEHL P U,COOK M.Unsupervised learning of digit recognition using spike-timing-dependent plasticity[J].Frontiers in Computational Neuroscience,2015,9:99.
[13]KOHONEN T.The self-organizing map[J].Proceedings of the IEEE,1990,78(9):1464-1480.
[14]TEYLER T J,RUDY J W.The hippocampal indexing theoryand episodic memory:updating the index[J].Hippocampus,2007,17(12):1158-1169.
[15]PONULAK F.ReSuMe-new supervised learning method for spiking neural networks.Institute of Control and Information Engineering,Poznoń University of Technology[R].Tech.rep,2005.
[16]ZHANG T,ZENG Y,ZHAO D,et al.A plasticity-centric ap-proach to train the non-differential spiking neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[17]KOCH C,SEGEV I.Methods in neuronal modeling:from ions to networks[M].MIT press,1998.
[18]HODGKIN A L,HUXLEYA F.A quantitative description ofmembrane current and its application to conduction and excitation in nerve[J].The Journal of Physiology,1952,117(4):500.
[19]KRUG M,MÜLLER-WELDE P,WAGNERM,et al.Functional plasticity in two afferent systems of the granule cells in the rat dentate area:frequency-related changes,long-term potentiation and heterosynaptic depression[J].Brain Research,1985,360(1/2):264-272.
[20]QUERLIOZ D,BICHLER O,DOLLFUS P,et al.Immunity to device variations in a spiking neural network with memristive nanodevices[J].IEEE Transactions on Nanotechnology,2013,12(3):288-295.
[21]HAO Y,HUANG X,DONG M,et al.A biologically plausiblesupervised learning method for spiking neural networks using the symmetric STDP rule[J].Neural Networks,2020,121:387-395.
[22]MO L,WANG G,LONG E,et al.ALSA:Associative Learning Based Supervised Learning Algorithm for SNN[J].Frontiers in Neuroscience,2022,16:838832.
[23]LIU F,ZHAO W,CHEN Y,et al.SSTDP:Supervised spike ti-ming dependent plasticity for efficient spiking neural network training[J].Frontiers in Neuroscience,2021,15:756876.
[24]TAVANAEI A,MAIDA A.BP-STDP:Approximating back-propagation using spike timing dependent plasticity[J].Neurocomputing,2019,330:39-47.
[25]PUTRA R V W,SHAFIQUE M.Fspinn:An optimizationframework for memory-efficient and energy-efficient spiking neural networks[J].IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems,2020,39(11):3601-3613.
[26]RASTOGI M,LU S,ISLAM N,et al.On the self-repair role of astrocytes in STDP enabled unsupervised SNNs[J].Frontiers in Neuroscience,2021,14:603796.
[27]SHI M,ZHANG T,ZENG Y.A curiosity-based learning method for spiking neural networks[J].Frontiers in Computational Neuroscience,2020,14:7.
[28]ZHAO D,ZENG Y,ZHANG T,et al.GLSNN:A multi-layer spiking neural network based on global feedback alignment and local STDP plasticity[J].Frontiers in Computational Neuroscience,2020,14:576841.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Spiking Neural Network Classification Model Based on Multi-subnetwork Pre-training

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 6

Metrics

Comments

Recommended 0

[1]	FENG Ren, CHEN Yunhua, XIONG Zhimin, CHEN Pinghua. Self-calibrating First Spike Temporal Encoding Neuron Model [J]. Computer Science, 2024, 51(3): 244-250.
[2]	AN Yang, WANG Xiuqing, ZHAO Minghua. Mobile Robots' Path Planning Method Based on Policy Fusion and Spiking Deep ReinforcementLearning [J]. Computer Science, 2024, 51(11A): 240100211-11.
[3]	MA Weiwei, ZHENG Qinhong, LIU Shanshan. Study and Evaluation of Spiking Neural Network Model Based on Bee Colony Optimization [J]. Computer Science, 2023, 50(8): 221-225.
[4]	MA Weiqi, YUAN Jiabin, ZHA Keke, FAN Lili. Onboard Rock Detection Algorithm Based on Spiking Neural Network [J]. Computer Science, 2023, 50(1): 98-104.
[5]	HUANG Zenan, LIU Xiaojie, ZHAO Chenhui, DENG Yabin, GUO Donghui. Spiking Neural Network Model for Brain-like Computing and Progress of Its Learning Algorithm [J]. Computer Science, 2023, 50(1): 229-242.
[6]	LIU Yang, TU Chun-long and ZHENG Feng-bin. Research of Neural Cognitive Computing Model for Visual and Auditory Cross-media Retrieval [J]. Computer Science, 2015, 42(3): 19-25.