计算机科学 ›› 2025, Vol. 52 ›› Issue (3): 260-267.doi: 10.11896/jsjkx.240100195
宁黎苗1, 王自铭2, 林志诚1, 彭舰1, 唐华锦2
NING Limiao1, WANG Ziming2, LIN Zhicheng1, PENG Jian1, TANG Huajin2
摘要: 由于脉冲神经元和突触复杂的时空动力学特性,训练脉冲神经网络比较困难,目前尚不存在公认的核心训练算法与技术。为此,提出一种基于直接反馈对齐(DFA)的精确脉冲时间(PREST-DFA)学习规则。受脉冲分层误差再分配(SLAYER)学习算法的启发,PREST-DFA使用基于脉冲卷积差的误差信号,输出层通过迭代方式计算出误差值,利用基于DFA的误差传输机制,将误差广播至隐藏层神经元,最后实现突触权值更新。仿真实验表明,实现了时间驱动的PREST-DFA学习算法具有精确脉冲时间学习能力。根据文献查询结果,这是首次验证基于DFA机制的学习算法可以在深层网络中控制脉冲的精确发放时间,说明DFA机制可以应用于基于脉冲时间的算法设计。另外还进行了学习性能和训练速度的比较,实验结果表明PREST-DFA能在较低的推理延迟下实现良好的学习性能,与采用相同学习规则使用反向传播训练的学习算法相比,能够加快训练速度。
中图分类号:
[1]LIAN S,SHEN J,LIU Q,et al.Learnable Surrogate Gradient for Direct Training Spiking Neural Networks[C]//International Joint Conferences on Artificial Intelligence Organization.2023. [2]WU Y,DENG L,LI G,et al.Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks [J].Frontiers in Neuroscience,2018,12:331. [3]HAN J,WANG Z,SHEN J,et al.Symmetric-threshold ReLU for Fast and Nearly Lossless ANN-SNN Conversion [J].Machine Intelligence Research,2023,20(3):435-446. [4]BU T,DING J H,YU Z F,et al.Optimized Potential Initialization for Low-Latency Spiking Neural Networks [J].arXiv:2202.01440,2022. [5]NING L,DONG J,XIAO R,et al.Event-driven spiking neural networks with spike-based learning [J].Memetic Computing,2023,15(2):205-217. [6]LIU F,ZHAO W,CHEN Y,et al.SSTDP:Supervised SpikeTiming Dependent Plasticity for Efficient Spiking Neural Network Training [J].Frontiers in Neuroscience,2021,15:756876. [7]SHRESTHA S B,ORCHARD G.SLAYER:Spike Layer Error Reassignment in Time[C]//proceedings of the NeurIPS.2018. [8]LILLICRAP T P,COWNDEN D,TWEED D B,et al.Random synaptic feedback weights support error backpropagation for deep learning [J].Nature Communications,2016,7:13276. [9]LILLICRAP T P,SANTORO A,MARRIS L,et al.Backpropagation and the brain [J].Nature Reviews Neuroscience,2020,21(6):335-346. [10]NØKLAND A.Direct Feedback Alignment Provides Learning in Deep Neural Networks[C]//Proceedings of the NIPS.2016. [11]LAUNAY J,POLI I,BONIFACE F C,et al.Direct FeedbackAlignment Scales to Modern Deep Learning Tasks and Architectures[C]//Proceedings of the NeurIPS.2020. [12]NEFTCI E O,AUGUSTINE C,PAUL S,et al.Event-DrivenRandom Back-Propagation:Enabling Neuromorphic Deep Learning Machines [J].Frontiers in Neuroscience,2017,11:324. [13]ZHAO D,ZENG Y,ZHANG T,et al.GLSNN:A Multi-Layer Spiking Neural Network Based on Global Feedback Alignment and Local STDP Plasticity [J].Frontiers Computational Neuroscience,2020,14:576841. [14]LEE J,ZHANG R,ZHANG W,et al.Spike-Train Level Direct Feedback Alignment:Sidestepping Backpropagation for On-Chip Training of Spiking Neural Nets [J].Frontiers in Neuroscience,2020,14:143. [15]SHI C,WANG T,HE J,et al.DeepTempo:A Hardware-Friend-ly Direct Feedback Alignment Multi-Layer Tempotron Learning Rule for Deep Spiking Neural Networks [J].IEEE Transactions on Circuits and Systems II:Express Briefs,2021,68(5):1581-1585. [16]KANG W M,KWON D,WOO S Y,et al.Hardware-Based Spiking Neural Network Using a TFT-Type AND Flash Memory Array Architecture Based on Direct Feedback Alignment [J].IEEE Access,2021,9:73121-73132. [17]BANG S,LEW D,CHOI S,et al.An Energy-Efficient SNN Processor Design based on Sparse Direct Feedback and Spike Prediction[C]//Proceedings of the 2021 International Joint Confe-rence on Neural Networks (IJCNN).2021. [18]TAVANAEI A,MAIDA A.BP-STDP:Approximating back-propagation using spike timing dependent plasticity [J].Neurocomputing,2019,330:39-47. [19]FANG W,YU Z,CHEN Y,et al.Deep Residual Learning in Spiking Neural Networks[C]//Proceedings of the Advances in Neural Information Processing Systems.2021. [20]FANG W,YU Z,CHEN Y,et al.Incorporating Learnable Membrane Time Constant To Enhance Learning of Spiking Neural Networks[C]//Proceedings of the ICCV.2021. [21]KAISER J,FRIEDRICH A,TIECK J C V,et al.Embodied Neuromorphic Vision with Event-Driven Random Backpropagation [J].arXiv1904,04805,2019. [22]XU Q,QI Y,YU H,et al.CSNN:An Augmented Spiking based Framework with Perceptron-Inception[C]//Proceedings of the IJCAI.2018. [23]DING J,YU Z,TIAN Y,et al.Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networks[C]//Proceedings of the IJCAI-21.2021. |
|