- Fully neuromorphic vision and control for autonomous drone flight 时间：2023年03月15日 第一作者：Federico Paredes-Vall'es 链接.
摘要：生物感知和处理是异步和稀疏的，导致低延迟和高效的感知和行动。在机器人技术中，用于基于事件的视觉和脉冲神经网络的神经形态硬件有望表现出类似的特征。然而，由于当前嵌入式神经形态处理器中的网络大小有限以及训练脉冲神经网络的困难，机器人的实现仅限于具有低维感觉输入和运动动作的基本任务。在这里，我们提出了第一个完全神经形态的视觉控制管道，用于控制自由飞行的无人机。具体来说，我们训练一个脉冲神经网络，该网络接受基于高维原始事件的相机数据，并输出低级别的控制动作，以执行基于自主视觉的飞行。该网络的视觉部分由五层和28.8k个神经元组成，将传入的原始事件映射到自我运动估计，并在真实环境中进行自我监督学习训练
英文摘要
Biological sensing and processing is asynchronous and sparse, leading to low-latency and energy-efficient perception and action. In robotics, neuromorphic hardware for event-based vision and spiking neural networks promises to exhibit similar characteristics. However, robotic implementations have been limited to basic tasks with low-dimensional sensory inputs and motor actions due to the restricted network size in current embedded neuromorphic processors and the difficulties of training spiking neural networks. Here, we present the first fully neuromorphic vision-to-control pipeline for controlling a freely flying drone. Specifically, we train a spiking neural network that accepts high-dimensional raw event-based camera data and outputs low-level control actions for performing autonomous vision-based flight. The vision part of the network, consisting of five layers and 28.8k neurons, maps incoming raw events to ego-motion estimates and is trained with self-supervised learning on real event data. The control part consists of a single decoding layer and is learned with an evolutionary algorithm in a drone simulator. Robotic experiments show a successful sim-to-real transfer of the fully learned neuromorphic pipeline. The drone can accurately follow different ego-motion setpoints, allowing for hovering, landing, and maneuvering sideways$\unicode{x2014}$even while yawing at the same time. The neuromorphic pipeline runs on board on Intel's Loihi neuromorphic processor with an execution frequency of 200 Hz, spending only 27 $\unicode{x00b5}$J per inference. These results illustrate the potential of neuromorphic sensing and processing for enabling smaller, more intelligent robots.邮件日期
2023年03月16日- Training Robust Spiking Neural Networks with ViewPoint Transform and SpatioTemporal Stretching 时间：2023年03月14日 第一作者：Haibo Shen 链接.
摘要：神经形态视觉传感器（事件摄像机）模拟生物视觉感知系统，具有时间分辨率高、数据冗余少、功耗低、动态范围大等优点。由于事件和脉冲都是根据神经信号建模的，因此事件摄像机天生适合脉冲神经网络（SNN），SNN被认为是人工智能（AI）和理论神经科学的有前途的模型。然而，这些相机的非常规视觉信号对脉冲神经网络的鲁棒性提出了巨大挑战。在本文中，我们提出了一种新的数据增强方法，视点变换和时空拉伸（VPT-STS）。它通过变换时空域中的旋转中心和角度来生成来自不同视点的样本，从而提高了SNN的鲁棒性。此外，我们引入了时空拉伸，以避免视点转换中潜在的信息丢失。广泛的实验
英文摘要
Neuromorphic vision sensors (event cameras) simulate biological visual perception systems and have the advantages of high temporal resolution, less data redundancy, low power consumption, and large dynamic range. Since both events and spikes are modeled from neural signals, event cameras are inherently suitable for spiking neural networks (SNNs), which are considered promising models for artificial intelligence (AI) and theoretical neuroscience. However, the unconventional visual signals of these cameras pose a great challenge to the robustness of spiking neural networks. In this paper, we propose a novel data augmentation method, ViewPoint Transform and SpatioTemporal Stretching (VPT-STS). It improves the robustness of SNNs by transforming the rotation centers and angles in the spatiotemporal domain to generate samples from different viewpoints. Furthermore, we introduce the spatiotemporal stretching to avoid potential information loss in viewpoint transformation. Extensive experiments on prevailing neuromorphic datasets demonstrate that VPT-STS is broadly effective on multi-event representations and significantly outperforms pure spatial geometric transformations. Notably, the SNNs model with VPT-STS achieves a state-of-the-art accuracy of 84.4\% on the DVS-CIFAR10 dataset.注释
Accepted by ICASSP 2023. arXiv admin note: text overlap with arXiv:2207.11659邮件日期
2023年03月15日- Emergent Bio-Functional Similarities in a Cortical-Spike-Train-Decoding Spiking Neural Network Facilitate Predictions of Neural Computation 时间：2023年03月14日 第一作者：Tengjun Liu 链接.
摘要：尽管目标驱动的脉冲神经网络（SNN）具有更好的生物合理性，但它在分类生物脉冲序列方面并没有达到适用的性能，并且与传统的人工神经网络相比，几乎没有表现出生物功能的相似性。在这项研究中，我们提出了运动SRNN，这是一种受灵长类动物神经运动回路拓扑启发的递归SNN。通过使用motorSRNN解码猴子初级运动皮层的脉冲序列，我们在分类准确性和能量消耗之间实现了良好的平衡。运动SRNN通过捕捉和培养更多的余弦调谐来与输入进行通信，这是运动皮层神经元的一个基本特性，并在训练过程中保持其稳定性。在我们的猴子身上也观察到了这种训练诱导的余弦调谐的培养和持续性。此外，马达SRNN在单个神经元、群体和电路水平上产生了额外的生物功能相似性，证明了生物
英文摘要
Despite its better bio-plausibility, goal-driven spiking neural network (SNN) has not achieved applicable performance for classifying biological spike trains, and showed little bio-functional similarities compared to traditional artificial neural networks. In this study, we proposed the motorSRNN, a recurrent SNN topologically inspired by the neural motor circuit of primates. By employing the motorSRNN in decoding spike trains from the primary motor cortex of monkeys, we achieved a good balance between classification accuracy and energy consumption. The motorSRNN communicated with the input by capturing and cultivating more cosine-tuning, an essential property of neurons in the motor cortex, and maintained its stability during training. Such training-induced cultivation and persistency of cosine-tuning was also observed in our monkeys. Moreover, the motorSRNN produced additional bio-functional similarities at the single-neuron, population, and circuit levels, demonstrating biological authenticity. Thereby, ablation studies on motorSRNN have suggested long-term stable feedback synapses contribute to the training-induced cultivation in the motor cortex. Besides these novel findings and predictions, we offer a new framework for building authentic models of neural computation.邮件日期
2023年03月15日- Training Robust Spiking Neural Networks on Neuromorphic Data with Spatiotemporal Fragments 时间：2023年03月14日 第一作者：Haibo Shen 链接.
注释
Accepted by ICASSP 2023邮件日期
2023年03月15日- Adaptive-SpikeNet: Event-based Optical Flow Estimation using Spiking Neural Networks with Learnable Neuronal Dynamics 时间：2023年03月14日 第一作者：Adarsh Kumar Kosta 链接.
邮件日期
2023年03月15日- Training Stronger Spiking Neural Networks with Biomimetic Adaptive Internal Association Neurons 时间：2023年03月14日 第一作者：Haibo Shen 链接.
注释
Accepted by ICASSP 2023邮件日期
2023年03月15日- Dynamic Event-based Optical Flow Identification and Communication 时间：2023年03月13日 第一作者：Axel von Arnim 链接.
摘要：光学识别通常通过空间或时间视觉模式识别和定位来完成。时间模式识别，取决于技术，涉及通信频率、范围和精确跟踪之间的权衡。我们提出了一种带有发光信标的解决方案，该解决方案通过利用基于事件的快速摄像机和使用脉冲神经元计算的稀疏神经形态光流来跟踪来改善这种权衡。在一个资产监控用例中，我们证明了嵌入模拟无人机的系统对相对运动具有鲁棒性，并能够与多个移动信标同时通信和跟踪。最后，在一个硬件实验室原型中，我们实现了最先进的光学相机通信频率，达到了kHz量级。
英文摘要
Optical identification is often done with spatial or temporal visual pattern recognition and localization. Temporal pattern recognition, depending on the technology, involves a trade-off between communication frequency, range and accurate tracking. We propose a solution with light-emitting beacons that improves this trade-off by exploiting fast event-based cameras and, for tracking, sparse neuromorphic optical flow computed with spiking neurons. In an asset monitoring use case, we demonstrate that the system, embedded in a simulated drone, is robust to relative movements and enables simultaneous communication with, and tracking of, multiple moving beacons. Finally, in a hardware lab prototype, we achieve state-of-the-art optical camera communication frequencies in the kHz magnitude.注释
5 pages, 6 figures and 1 table邮件日期
2023年03月14日- Fusarium head blight detection, spikelet estimation, and severity assessment in wheat using 3D convolutional neural networks 时间：2023年03月10日 第一作者：Oumaima Hamila 链接.
摘要：镰刀菌头疫病（FHB）是影响全世界小麦和其他小谷物的最重要疾病之一。抗性品种的开发需要田间和温室表型的艰苦工作。在这项工作中考虑的应用是小麦植株上表达的FHB疾病症状的自动检测，小麦头上小穗总数和受感染小穗总数的自动估计，以及受感染小麦中FHB严重程度的自动评估。用于生成结果的数据是三维（3D）多光谱点云（PC），它是点的3D集合，每个点都与红、绿、蓝（RGB）和近红外（NIR）测量值相关。使用多光谱3D扫描仪收集了300多幅小麦植物图像，并创建了标记的UW-MRDC 3D小麦数据集。这些数据被用于开发用于FHB检测的新颖高效的3D卷积神经网络（CNN）模型
英文摘要
Fusarium head blight (FHB) is one of the most significant diseases affecting wheat and other small grain cereals worldwide. The development of resistant varieties requires the laborious task of field and greenhouse phenotyping. The applications considered in this work are the automated detection of FHB disease symptoms expressed on a wheat plant, the automated estimation of the total number of spikelets and the total number of infected spikelets on a wheat head, and the automated assessment of the FHB severity in infected wheat. The data used to generate the results are 3-dimensional (3D) multispectral point clouds (PC), which are 3D collections of points - each associated with a red, green, blue (RGB), and near-infrared (NIR) measurement. Over 300 wheat plant images were collected using a multispectral 3D scanner, and the labelled UW-MRDC 3D wheat dataset was created. The data was used to develop novel and efficient 3D convolutional neural network (CNN) models for FHB detection, which achieved 100% accuracy. The influence of the multispectral information on performance was evaluated, and our results showed the dominance of the RGB channels over both the NIR and the NIR plus RGB channels combined. Furthermore, novel and efficient 3D CNNs were created to estimate the total number of spikelets and the total number of infected spikelets on a wheat head, and our best models achieved mean absolute errors (MAE) of 1.13 and 1.56, respectively. Moreover, 3D CNN models for FHB severity estimation were created, and our best model achieved 8.6 MAE. A linear regression analysis between the visual FHB severity assessment and the FHB severity predicted by our 3D CNN was performed, and the results showed a significant correlation between the two variables with a 0.0001 P-value and 0.94 R-squared.邮件日期
2023年03月13日- Advancing Spiking Neural Networks towards Deep Residual Learning 时间：2023年03月10日 第一作者：Yifan Hu 链接.
邮件日期
2023年03月13日- Exact Gradient Computation for Spiking Neural Networks Through Forward Propagation 时间：2023年03月10日 第一作者：Jane H. Lee 链接.
邮件日期
2023年03月13日- Deep Spiking Neural Networks with High Representation Similarity Model Visual Pathways of Macaque and Mouse 时间：2023年03月09日 第一作者：Liwei Huang 链接.
摘要：深度人工神经网络（ANN）在灵长类和啮齿类动物的视觉路径建模中发挥着重要作用。然而，与生物对应物相比，它们大大简化了神经元的计算财产。相反，刺突神经网络（SNN）是更具生物学意义的模型，因为刺突神经元与生物神经元一样，用刺突时间序列编码信息。在这项研究中，我们首次使用深度SNN对视觉皮层进行建模，并使用各种最先进的深度CNN和ViT进行比较。使用三个相似性度量，我们在三种类型的刺激下对从两个物种收集的三个神经数据集进行了神经表示相似性实验。基于广泛的相似性分析，我们进一步研究了跨物种的功能层次和机制。几乎所有SNN的相似性得分都高于其计数
英文摘要
Deep artificial neural networks (ANNs) play a major role in modeling the visual pathways of primate and rodent. However, they highly simplify the computational properties of neurons compared to their biological counterparts. Instead, Spiking Neural Networks (SNNs) are more biologically plausible models since spiking neurons encode information with time sequences of spikes, just like biological neurons do. However, there is a lack of studies on visual pathways with deep SNNs models. In this study, we model the visual cortex with deep SNNs for the first time, and also with a wide range of state-of-the-art deep CNNs and ViTs for comparison. Using three similarity metrics, we conduct neural representation similarity experiments on three neural datasets collected from two species under three types of stimuli. Based on extensive similarity analyses, we further investigate the functional hierarchy and mechanisms across species. Almost all similarity scores of SNNs are higher than their counterparts of CNNs with an average of 6.6%. Depths of the layers with the highest similarity scores exhibit little differences across mouse cortical regions, but vary significantly across macaque regions, suggesting that the visual processing structure of mice is more regionally homogeneous than that of macaques. Besides, the multi-branch structures observed in some top mouse brain-like neural networks provide computational evidence of parallel processing streams in mice, and the different performance in fitting macaque neural representations under different stimuli exhibits the functional specialization of information processing in macaques. Taken together, our study demonstrates that SNNs could serve as promising candidates to better model and explain the functional hierarchy and mechanisms of the visual system.注释
Accepted by Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI-23)邮件日期
2023年03月13日- Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks 时间：2023年03月08日 第一作者：Tong Bu 链接.
摘要：脉冲神经网络（SNN）因其独特的低功耗财产和对神经形态硬件的快速推理而受到广泛关注。作为获得深度SNN的最有效方法，ANN-SNN转换在大规模数据集上取得了与ANN相当的性能。尽管如此，要使SNN的发射速率与ANN的激活相匹配，还需要很长的时间。结果，转换后的SNN在短时间步长下遭受严重的性能退化问题，这阻碍了SNN的实际应用。在本文中，我们从理论上分析了ANN-SNN转换误差，并推导了SNN的估计激活函数。然后，我们提出了量化剪辑地板移位激活函数来代替源ANN中的ReLU激活函数，这可以更好地逼近SNN的激活函数。我们证明了SNN和ANN之间的预期转换误差为零，使我们能够实现高精度和超低延迟的SNN。
英文摘要
Spiking Neural Networks (SNNs) have gained great attraction due to their distinctive properties of low power consumption and fast inference on neuromorphic hardware. As the most effective method to get deep SNNs, ANN-SNN conversion has achieved comparable performance as ANNs on large-scale datasets. Despite this, it requires long time-steps to match the firing rates of SNNs to the activation of ANNs. As a result, the converted SNN suffers severe performance degradation problems with short time-steps, which hamper the practical application of SNNs. In this paper, we theoretically analyze ANN-SNN conversion error and derive the estimated activation function of SNNs. Then we propose the quantization clip-floor-shift activation function to replace the ReLU activation function in source ANNs, which can better approximate the activation function of SNNs. We prove that the expected conversion error between SNNs and ANNs is zero, enabling us to achieve high-accuracy and ultra-low-latency SNNs. We evaluate our method on CIFAR-10/100 and ImageNet datasets, and show that it outperforms the state-of-the-art ANN-SNN and directly trained SNNs in both accuracy and time-steps. To the best of our knowledge, this is the first time to explore high-performance ANN-SNN conversion with ultra-low latency (4 time-steps). Code is available at https://github.com/putshua/SNN\_conversion\_QCFS邮件日期
2023年03月09日- Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks 时间：2023年03月07日 第一作者：Qingyan Meng 链接.
邮件日期
2023年03月08日- Emerging AI Technologies Inspiring the Next Generation of E-textiles 时间：2023年03月06日 第一作者：Frances Cleary 链接.
摘要：智能纺织品和可穿戴设备行业正在寻求先进的技术，以满足快速发展的纺织行业中的工业、消费者和新兴创新纺织品应用需求。同时，基于人脑生物神经工作原理的灵感正在推动下一代人工智能。模拟神经网络和人类神经系统的处理能力和财产的人工智能启发的硬件（神经形态计算）和软件模块正在形成。纺织行业需要积极研究这些新兴和新技术，从其工作和加工方法中汲取灵感，以促进电子织物世界中新的和创新的嵌入式智能进步。这一新兴的下一代人工智能（AI）正在各个行业（纺织、医疗、汽车、航空航天、军事）迅速获得兴趣。这些财产如何激发
英文摘要
The smart textile and wearables sector is looking towards advancing technologies to meet both industry, consumer and new emerging innovative textile application demands, within a fast paced textile industry. In parallel inspiration based on the biological neural workings of the human brain is driving the next generation of artificial intelligence. Artificial intelligence inspired hardware (neuromorphic computing) and software modules mimicking the processing capabilities and properties of neural networks and the human nervous system are taking shape. The textile sector needs to actively look at such emerging and new technologies taking inspiration from their workings and processing methods in order to stimulate new and innovative embedded intelligence advancements in the etextile world. This emerging next generation of Artificial intelligence(AI) is rapidly gaining interest across varying industries (textile, medical, automotive, aerospace, military). How such properties can inspire and drive advancements within the etextiles sector needs to be considered. This paper will provide an insight into current nanotechnology and artificial intelligence advancements in the etextiles domain before focusing specifically on the future vision and direction around the potential application of neuromorphic computing and spiking neural network inspired AI technologies within the textile sector. We investigate the core architectural elements of artificial neural networks, neuromorphic computing and how such neuroscience inspired technologies could impact and inspire change and new research developments within the e-textile sector.注释
14 pages, 8 figures, 2 tables邮件日期
2023年03月07日- Ensembles of Compact, Region-specific & Regularized Spiking Neural Networks for Scalable Place Recognition 时间：2023年03月06日 第一作者：Somayeh Hussaini 链接.
注释
8 pages, 6 figures, accepted to the IEEE International Conference on Robotics and Automation (ICRA) 2023邮件日期
2023年03月07日- TopSpark: A Timestep Optimization Methodology for Energy-Efficient Spiking Neural Networks on Autonomous Mobile Agents 时间：2023年03月03日 第一作者：Rachmad Vidya Wicaksana Putra 链接.
摘要：自主移动代理需要低功耗/节能的机器学习（ML）算法来完成其基于ML的任务，同时适应不同的环境，因为移动代理通常由电池供电。Spiking神经网络（SNN）可以满足这些要求，因为它们提供了低功耗/能量处理，这是因为它们的计算稀疏和高效的在线学习，具有生物启发的学习机制，可以适应不同的环境。最近的研究表明，可以通过减少每个神经元处理一系列脉冲（时间步长）的计算时间来优化SNN的能量消耗。然而，最先进的技术依赖于密集的设计搜索来确定仅用于推断的固定时间步长设置，从而阻碍SNN在训练和推断中实现进一步的能量效率提高。这些技术也限制了SNN在运行时执行有效的在线学习。为此，我们建议TopSpark
英文摘要
Autonomous mobile agents require low-power/energy-efficient machine learning (ML) algorithms to complete their ML-based tasks while adapting to diverse environments, as mobile agents are usually powered by batteries. These requirements can be fulfilled by Spiking Neural Networks (SNNs) as they offer low power/energy processing due to their sparse computations and efficient online learning with bio-inspired learning mechanisms for adapting to different environments. Recent works studied that the energy consumption of SNNs can be optimized by reducing the computation time of each neuron for processing a sequence of spikes (timestep). However, state-of-the-art techniques rely on intensive design searches to determine fixed timestep settings for only inference, thereby hindering SNNs from achieving further energy efficiency gains in both training and inference. These techniques also restrict SNNs from performing efficient online learning at run time. Toward this, we propose TopSpark, a novel methodology that leverages adaptive timestep reduction to enable energy-efficient SNN processing in both training and inference, while keeping its accuracy close to the accuracy of SNNs without timestep reduction. The ideas of TopSpark include analyzing the impact of different timesteps on the accuracy; identifying neuron parameters that have a significant impact on accuracy in different timesteps; employing parameter enhancements that make SNNs effectively perform learning and inference using less spiking activity; and developing a strategy to trade-off accuracy, latency, and energy to meet the design requirements. The results show that, TopSpark saves the SNN latency by 3.9x as well as energy consumption by 3.5x for training and 3.3x for inference on average, across different network sizes, learning rules, and workloads, while maintaining the accuracy within 2% of SNNs without timestep reduction.注释
This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible邮件日期
2023年03月06日- Hyperdimensional Computing with Spiking-Phasor Neurons 时间：2023年02月28日 第一作者：Jeff Orchard 链接.
摘要：矢量符号体系结构（VSA）是表示组合推理的强大框架。它们有助于神经网络的实现，使我们能够创建能够执行认知功能的神经网络，如空间推理、算术、符号绑定和逻辑。但是所涉及的向量可能非常大，因此可以选择超维（HD）计算。神经形态硬件的进步有望将神经网络的运行时间和能量足迹减少几个数量级。在这篇论文中，我们扩展了一些开创性的工作，以在可以在神经形态硬件上高效运行的脉冲神经元基底上运行VSA算法。
英文摘要
Vector Symbolic Architectures (VSAs) are a powerful framework for representing compositional reasoning. They lend themselves to neural-network implementations, allowing us to create neural networks that can perform cognitive functions, like spatial reasoning, arithmetic, symbol binding, and logic. But the vectors involved can be quite large, hence the alternative label Hyperdimensional (HD) computing. Advances in neuromorphic hardware hold the promise of reducing the running time and energy footprint of neural networks by orders of magnitude. In this paper, we extend some pioneering work to run VSA algorithms on a substrate of spiking neurons that could be run efficiently on neuromorphic hardware.邮件日期
2023年03月02日- Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks 时间：2023年02月28日 第一作者：Qingyan Meng 链接.
摘要：Spiking神经网络（SNN）是神经形态计算的高效节能模型。为了训练不可微SNN模型，通过时间反向传播（BPTT）和替代梯度（SG）方法已经获得了高性能。然而，这种方法在训练期间存在相当大的记忆成本和训练时间。在本文中，我们提出了通过时间的空间学习（SLTT）方法，与BPTT相比，该方法可以实现高性能，同时大大提高训练效率。首先，我们表明SNN通过时域的反向传播对最终计算的梯度仅有一点点贡献。因此，我们建议在反向传播期间忽略计算图中不重要的路由。所提出的方法减少了标量乘法的数量，并实现了独立于总时间步长的小内存占用。此外，我们提出了一种SLTT变体，称为SLTT-K，它仅允许在
英文摘要
Spiking Neural Networks (SNNs) are promising energy-efficient models for neuromorphic computing. For training the non-differentiable SNN models, the backpropagation through time (BPTT) with surrogate gradients (SG) method has achieved high performance. However, this method suffers from considerable memory cost and training time during training. In this paper, we propose the Spatial Learning Through Time (SLTT) method that can achieve high performance while greatly improving training efficiency compared with BPTT. First, we show that the backpropagation of SNNs through the temporal domain contributes just a little to the final calculated gradients. Thus, we propose to ignore the unimportant routes in the computational graph during backpropagation. The proposed method reduces the number of scalar multiplications and achieves a small memory occupation that is independent of the total time steps. Furthermore, we propose a variant of SLTT, called SLTT-K, that allows backpropagation only at K time steps, then the required number of scalar multiplications is further reduced and is independent of the total time steps. Experiments on both static and neuromorphic datasets demonstrate superior training efficiency and performance of our SLTT. In particular, our method achieves state-of-the-art accuracy on ImageNet, while the memory cost and training time are reduced by more than 70% and 50%, respectively, compared with BPTT.邮件日期
2023年03月01日- SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks 时间：2023年02月28日 第一作者：Rui-Jie Zhu 链接.
邮件日期
2023年03月01日- Communication Trade-offs in Federated Learning of Spiking Neural Networks 时间：2023年02月27日 第一作者：Soumi Chaki 链接.
摘要：Spiking神经网络（SNN）是传统人工神经网络（ANN）的生物替代品。尽管取得了有希望的初步结果，但在分布式方案中训练SNN时的权衡还没有得到很好的理解。这里，我们考虑联合学习环境中的SNN，其中通过聚合来自客户端的多个本地模型来创建高质量的全局模型，而不共享任何数据。当两种机制降低了上行链路通信成本时，我们研究了用于在客户端训练多个SNN的联合学习：i）随机屏蔽从客户端发送到服务器的模型更新；以及ii）一些客户端不向服务器发送其更新的客户端退出。我们使用Spiking Heidelberg数字（SHD）数据集的子集评估了SNN的性能。结果表明，随机屏蔽和客户端丢弃概率之间的权衡对于在固定数量的客户端上获得满意的性能至关重要。
英文摘要
Spiking Neural Networks (SNNs) are biologically inspired alternatives to conventional Artificial Neural Networks (ANNs). Despite promising preliminary results, the trade-offs in the training of SNNs in a distributed scheme are not well understood. Here, we consider SNNs in a federated learning setting where a high-quality global model is created by aggregating multiple local models from the clients without sharing any data. We investigate federated learning for training multiple SNNs at clients when two mechanisms reduce the uplink communication cost: i) random masking of the model updates sent from the clients to the server; and ii) client dropouts where some clients do not send their updates to the server. We evaluated the performance of the SNNs using a subset of the Spiking Heidelberg digits (SHD) dataset. The results show that a trade-off between the random masking and the client drop probabilities is crucial to obtain a satisfactory performance for a fixed number of clients.注释
5 pages, 7 figures, 1 algorithm, 1 table邮件日期
2023年03月03日- SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks 时间：2023年02月27日 第一作者：Rui-Jie Zhu 链接.
摘要：随着大型语言模型的规模不断扩大，运行它所需的计算资源也在不断增加。Spiking神经网络（SNN）已成为一种高效的深度学习方法，它利用稀疏和事件驱动的激活来减少与模型推理相关的计算开销。尽管SNN在许多计算机视觉任务上已经与非扣球模型竞争，但SNN的训练也被证明更具挑战性。因此，它们的表现落后于现代深度学习，我们还没有看到SNN在语言生成中的有效性。在本文中，我们成功地实现了“SpikeGPT”，这是一个具有纯二进制、事件驱动的脉冲激活单元的生成语言模型。我们在三个模型变量上训练所提出的模型：45M、125M和260M参数。据我们所知，这是迄今为止任何功能性反向训练SNN的4倍。我们通过修改变压器块以替换多头来实现这一点
英文摘要
As the size of large language models continue to scale, so does the computational resources required to run it. Spiking neural networks (SNNs) have emerged as an energy-efficient approach to deep learning that leverage sparse and event-driven activations to reduce the computational overhead associated with model inference. While they have become competitive with non-spiking models on many computer vision tasks, SNNs have also proven to be more challenging to train. As a result, their performance lags behind modern deep learning, and we are yet to see the effectiveness of SNNs in language generation. In this paper, we successfully implement `SpikeGPT', a generative language model with pure binary, event-driven spiking activation units. We train the proposed model on three model variants: 45M, 125M and 260M parameters. To the best of our knowledge, this is 4x larger than any functional backprop-trained SNN to date. We achieve this by modifying the transformer block to replace multi-head self attention to reduce quadratic computational complexity to linear with increasing sequence length. Input tokens are instead streamed in sequentially to our attention mechanism (as with typical SNNs). Our preliminary experiments show that SpikeGPT remains competitive with non-spiking models on tested benchmarks, while maintaining 5x less energy consumption when processed on neuromorphic hardware that can leverage sparse, event-driven activations. Our code implementation is available at https://github.com/ridgerchu/SpikeGPT.邮件日期
2023年02月28日- AutoML for neuromorphic computing and application-driven co-design: asynchronous, massively parallel optimization of spiking architectures 时间：2023年02月26日 第一作者：Angel Yanguas-Gil 链接.
摘要：在这项工作中，我们将AutoML启发的方法扩展到神经形态结构的探索和优化。通过将基于模型的并行异步搜索方法与模拟脉冲架构的模拟框架相集成，我们能够有效地探索神经形态架构的配置空间，并识别导致目标应用中最高性能的条件子集。我们已经在实时芯片上学习应用的示例案例中演示了这种方法。我们的结果表明，我们可以有效地使用优化方法来优化复杂的架构，从而为应用程序驱动的代码设计提供了一条可行的途径。
英文摘要
In this work we have extended AutoML inspired approaches to the exploration and optimization of neuromorphic architectures. Through the integration of a parallel asynchronous model-based search approach with a simulation framework to simulate spiking architectures, we are able to efficiently explore the configuration space of neuromorphic architectures and identify the subset of conditions leading to the highest performance in a targeted application. We have demonstrated this approach on an exemplar case of real time, on-chip learning application. Our results indicate that we can effectively use optimization approaches to optimize complex architectures, therefore providing a viable pathway towards application-driven codesign.邮件日期
2023年02月28日- A Unified Framework for Soft Threshold Pruning 时间：2023年02月25日 第一作者：Yanqi Chen 链接.
摘要：软阈值修剪是最先进的修剪方法之一，具有最先进的性能。然而，以前的方法要么在阈值调度器上执行无目标搜索，要么简单地设置阈值可训练，缺乏统一角度的理论解释。在这项工作中，我们将软阈值修剪重新表述为使用迭代收缩阈值算法（ISTA）解决的隐式优化问题，该算法是稀疏恢复和压缩感知领域的经典方法。在这个理论框架下，先前软阈值修剪研究中提出的所有阈值调整策略都被归纳为不同类型的调整$L_1$正则化项。基于我们的框架，通过对阈值调度的深入研究，我们进一步导出了一个最优阈值调度器。该调度器保持$L_1$-正则化系数稳定，这意味着从优化的角度来看是一个时间不变的目标函数。原则上
英文摘要
Soft threshold pruning is among the cutting-edge pruning methods with state-of-the-art performance. However, previous methods either perform aimless searching on the threshold scheduler or simply set the threshold trainable, lacking theoretical explanation from a unified perspective. In this work, we reformulate soft threshold pruning as an implicit optimization problem solved using the Iterative Shrinkage-Thresholding Algorithm (ISTA), a classic method from the fields of sparse recovery and compressed sensing. Under this theoretical framework, all threshold tuning strategies proposed in previous studies of soft threshold pruning are concluded as different styles of tuning注释
To appear in the 11th International Conference on Learning Representations (ICLR 2023)邮件日期
2023年02月28日- Heterogeneous Neuronal and Synaptic Dynamics for Spike-Efficient Unsupervised Learning: Theory and Design Principles 时间：2023年02月22日 第一作者：Biswadeep Chakraborty 链接.
摘要：本文表明，神经元和突触动力学的异质性降低了递归脉冲神经网络（RSNN）的脉冲活动，同时提高了预测性能，实现了脉冲高效（无监督）学习。我们分析表明，神经元整合/松弛动力学的多样性提高了RSNN学习更多不同输入模式的能力（更高的记忆容量），从而提高了分类和预测性能。我们进一步证明，突触的异质性脉冲时间依赖可塑性（STDP）动力学降低了脉冲活动，但保留了记忆能力。分析结果促使异质RSNN设计使用贝叶斯优化来确定神经元和突触中的异质性，以提高数学上的｛E｝$，定义为脉冲活动和记忆容量的比率。对时间序列分类和预测任务的实证结果表明，优化的HRNN提高了性能并减少了脉冲活动
英文摘要
This paper shows that the heterogeneity in neuronal and synaptic dynamics reduces the spiking activity of a Recurrent Spiking Neural Network (RSNN) while improving prediction performance, enabling spike-efficient (unsupervised) learning. We analytically show that the diversity in neurons' integration/relaxation dynamics improves an RSNN's ability to learn more distinct input patterns (higher memory capacity), leading to improved classification and prediction performance. We further prove that heterogeneous Spike-Timing-Dependent-Plasticity (STDP) dynamics of synapses reduce spiking activity but preserve memory capacity. The analytical results motivate Heterogeneous RSNN design using Bayesian optimization to determine heterogeneity in neurons and synapses to improve注释
Paper Published in ICLR 2023 (https://openreview.net/forum?id=QIRtAqoXwj) Journal-ref: The Eleventh International Conference on Learning Representations 2023邮件日期
2023年02月24日- Bridging the Gap between ANNs and SNNs by Calibrating Offset Spikes 时间：2023年02月21日 第一作者：Zecheng Hao 链接.
摘要：Spiking神经网络（SNN）由于其低功耗和时间信息处理的独特特性而引起了极大的关注。ANN-SNN转换作为应用SNN最常用的训练方法，可以确保转换后的SNN在大规模数据集上实现与ANN相当的性能。然而，在低数量的时间步长下，性能严重下降，这阻碍了SNN在神经形态芯片上的实际应用。在本文中，我们不是评估不同的转换误差，然后消除这些误差，而是定义一个偏移脉冲来测量实际和期望的SNN发射率之间的偏差程度。我们对偏移脉冲进行了详细分析，并注意到额外（或更少）脉冲的激发是转换错误的主要原因。基于此，我们提出了一种基于移动初始膜电势的优化策略，并从理论上证明了相应的优化
英文摘要
Spiking Neural Networks (SNNs) have attracted great attention due to their distinctive characteristics of low power consumption and temporal information processing. ANN-SNN conversion, as the most commonly used training method for applying SNNs, can ensure that converted SNNs achieve comparable performance to ANNs on large-scale datasets. However, the performance degrades severely under low quantities of time-steps, which hampers the practical applications of SNNs to neuromorphic chips. In this paper, instead of evaluating different conversion errors and then eliminating these errors, we define an offset spike to measure the degree of deviation between actual and desired SNN firing rates. We perform a detailed analysis of offset spike and note that the firing of one additional (or one less) spike is the main cause of conversion errors. Based on this, we propose an optimization strategy based on shifting the initial membrane potential and we theoretically prove the corresponding optimal shifting distance for calibrating the spike. In addition, we also note that our method has a unique iterative property that enables further reduction of conversion errors. The experimental results show that our proposed method achieves state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet datasets. For example, we reach a top-1 accuracy of 67.12% on ImageNet when using 6 time-steps. To the best of our knowledge, this is the first time an ANN-SNN conversion has been shown to simultaneously achieve high accuracy and ultralow latency on complex datasets. Code is available at https://github.com/hzc1208/ANN2SNN_COS.注释
Accepted by ICLR 2023邮件日期
2023年02月22日- NU-AIR -- A Neuromorphic Urban Aerial Dataset for Detection and Localization of Pedestrians and Vehicles 时间：2023年02月18日 第一作者：Craig Iaboni 链接.
摘要：捕捉城市环境中行人和车辆的注释图像可用于训练神经网络（NN）以执行机器视觉任务。本文介绍了第一个开源的空中神经形态数据集，该数据集捕捉了在城市环境中移动的行人和车辆。该数据集名为NU-AIR，包含70.75分钟的事件镜头，这些镜头由安装在城市环境中运行的四旋翼上的640 x 480分辨率神经形态传感器采集。在不同的海拔和照明条件下，可以捕捉到繁忙的城市十字路口的行人、不同类型的车辆和街景。记录中包含的车辆和行人的手动边界框注释以30 Hz的频率提供，总共产生93204个标签。通过训练三个Spiking神经网络（SNN）和十个Deep神经网络（DNN）来评估数据集的保真度。tes的平均精度（mAP）结果
英文摘要
Annotated imagery capturing pedestrians and vehicles in an urban environment can be used to train Neural Networks (NNs) for machine vision tasks. This paper presents the first open-source aerial neuromorphic dataset that captures pedestrians and vehicles moving in an urban environment. The dataset, titled NU-AIR, features 70.75 minutes of event footage acquired with a 640 x 480 resolution neuromorphic sensor mounted on a quadrotor operating in an urban environment. Crowds of pedestrians, different types of vehicles, and street scenes at a busy urban intersection are captured at different elevations and illumination conditions. Manual bounding box annotations of vehicles and pedestrians contained in the recordings are provided at a frequency of 30 Hz, yielding 93,204 labels in total. Evaluation of the dataset's fidelity is performed by training three Spiking Neural Networks (SNNs) and ten Deep Neural Networks (DNNs). The mean average precision (mAP) accuracy results achieved for the testing set evaluations are on-par with results reported for similar SNNs and DNNs on established neuromorphic benchmark datasets. All data and Python code to voxelize the data and subsequently train SNNs/DNNs has been open-sourced.注释
10 pages, 4 figures邮件日期
2023年02月21日- KLIF: An optimized spiking neuron unit for tuning surrogate gradient slope and membrane potential 时间：2023年02月18日 第一作者：Chunming Jiang 链接.
摘要：Spiking神经网络（SNN）由于其处理时间信息的能力、低功耗和更高的生物合理性而备受关注。然而，为SNN开发高效和高性能的学习算法仍然具有挑战性。像人工神经网络（ANN）到SNN转换这样的方法可以将ANN转换为SNN，但性能损失很小，但需要长时间的模拟来近似速率编码。通过基于脉冲的反向传播（BP）（如替代梯度近似）直接训练SNN更灵活。然而现在，SNN的性能与ANN相比没有竞争力。在本文中，我们提出了一种新的基于k的漏积分和火（KLIF）神经元模型，以提高SNN的学习能力。与流行的泄漏积分和火灾（LIF）模型相比，KLIF增加了一个可学习的缩放因子，以在训练期间动态更新替代梯度曲线的斜率和宽度，并结合了ReLU激活因子
英文摘要
Spiking neural networks (SNNs) have attracted much attention due to their ability to process temporal information, low power consumption, and higher biological plausibility. However, it is still challenging to develop efficient and high-performing learning algorithms for SNNs. Methods like artificial neural network (ANN)-to-SNN conversion can transform ANNs to SNNs with slight performance loss, but it needs a long simulation to approximate the rate coding. Directly training SNN by spike-based backpropagation (BP) such as surrogate gradient approximation is more flexible. Yet now, the performance of SNNs is not competitive compared with ANNs. In this paper, we propose a novel k-based leaky Integrate-and-Fire (KLIF) neuron model to improve the learning ability of SNNs. Compared with the popular leaky integrate-and-fire (LIF) model, KLIF adds a learnable scaling factor to dynamically update the slope and width of the surrogate gradient curve during training and incorporates a ReLU activation function that selectively delivers membrane potential to spike firing and resetting. The proposed spiking unit is evaluated on both static MNIST, Fashion-MNIST, CIFAR-10 datasets, as well as neuromorphic N-MNIST, CIFAR10-DVS, and DVS128-Gesture datasets. Experiments indicate that KLIF performs much better than LIF without introducing additional computational cost and achieves state-of-the-art performance on these datasets with few time steps. Also, KLIF is believed to be more biological plausible than LIF. The good performance of KLIF can make it completely replace the role of LIF in SNN for various tasks.邮件日期
2023年02月21日- Adaptive Axonal Delays in feedforward spiking neural networks for accurate spoken word recognition 时间：2023年02月16日 第一作者：Pengfei Sun 链接.
摘要：Spiking神经网络（SNN）是构建准确、高效的自动语音识别系统的一种很有前途的研究途径。音频到脉冲编码和训练算法的最新进展使SNN能够应用于实际任务。受生物学启发的SNN使用稀疏异步事件进行通信。因此，脉冲时间对SNN性能至关重要。在这方面，大多数工作集中于训练突触权重，很少有人考虑事件传递中的延迟，即轴突延迟。在这项工作中，我们考虑了上限为最大值的可学习轴突延迟，其可以根据每个网络层中的轴突延迟分布进行调整。我们表明，我们提出的方法实现了SHD数据集（92.45%）和NTIDIGITS数据集（95.09%）上报告的最佳分类结果。我们的工作说明了训练具有复杂时间结构的任务的轴突延迟的潜力。
英文摘要
Spiking neural networks (SNN) are a promising research avenue for building accurate and efficient automatic speech recognition systems. Recent advances in audio-to-spike encoding and training algorithms enable SNN to be applied in practical tasks. Biologically-inspired SNN communicates using sparse asynchronous events. Therefore, spike-timing is critical to SNN performance. In this aspect, most works focus on training synaptic weights and few have considered delays in event transmission, namely axonal delay. In this work, we consider a learnable axonal delay capped at a maximum value, which can be adapted according to the axonal delay distribution in each network layer. We show that our proposed method achieves the best classification results reported on the SHD dataset (92.45%) and NTIDIGITS dataset (95.09%). Our work illustrates the potential of training axonal delays for tasks with complex temporal structures.注释
Accepted by ICASSP 2023邮件日期
2023年02月20日- Surrogate Gradient Spiking Neural Networks as Encoders for Large Vocabulary Continuous Speech Recognition 时间：2023年02月16日 第一作者：Alex 链接.
邮件日期
2023年02月17日- Frameworks for SNNs: a Review of Data Science-oriented Software and an Expansion of SpykeTorch 时间：2023年02月15日 第一作者：Davide Liberato Manna 链接.
摘要：为神经形态（NM）领域的机器学习（ML）应用开发有效的学习系统需要大量的实验和模拟。软件框架通过提供一组研究人员可以利用的现成工具来帮助和简化这一过程。最近对神经网络技术的兴趣已经看到了一些新的框架的发展，这些框架可以实现这一点，并为属于神经科学领域的现有图书馆提供了全景。这项工作回顾了专门面向数据科学应用的Spiking神经网络（SNN）开发的9个框架。我们强调脉冲神经元模型和学习规则的可用性，以更容易地指导对最适合开展不同类型研究的框架的决策。此外，我们还提供了SpykeThorch框架的扩展，该框架允许用户访问更广泛的神经元模型选择，以嵌入SNN并公开代码。
英文摘要
Developing effective learning systems for Machine Learning (ML) applications in the Neuromorphic (NM) field requires extensive experimentation and simulation. Software frameworks aid and ease this process by providing a set of ready-to-use tools that researchers can leverage. The recent interest in NM technology has seen the development of several new frameworks that do this, and that add up to the panorama of already existing libraries that belong to neuroscience fields. This work reviews 9 frameworks for the development of Spiking Neural Networks (SNNs) that are specifically oriented towards data science applications. We emphasize the availability of spiking neuron models and learning rules to more easily direct decisions on the most suitable frameworks to carry out different types of research. Furthermore, we present an extension to the SpykeTorch framework that gives users access to a much broader choice of neuron models to embed in SNNs and make the code publicly available.邮件日期
2023年02月16日- Hybrid Spiking Neural Network Fine-tuning for Hippocampus Segmentation 时间：2023年02月14日 第一作者：Ye Yue 链接.
摘要：在过去十年中，人工神经网络（ANN）取得了巨大的进步，部分原因是注释数据的可用性增加。然而，ANN通常需要大量的功率和内存消耗才能达到其全部潜力。由于其稀疏性，Spiking神经网络（SNN）最近成为ANN的低功耗替代方案。然而，SNN不像ANN那样容易训练。在这项工作中，我们提出了一种混合SNN训练方案，并将其应用于从磁共振图像中分割人类海马。我们的方法将ANN-SNN转换作为初始化步骤，并依赖于基于脉冲的反向传播来微调网络。与转换和直接训练解决方案相比，我们的方法在分割精度和训练效率方面都具有优势。实验证明了我们的模型在实现设计目标方面的有效性。
英文摘要
Over the past decade, artificial neural networks (ANNs) have made tremendous advances, in part due to the increased availability of annotated data. However, ANNs typically require significant power and memory consumptions to reach their full potential. Spiking neural networks (SNNs) have recently emerged as a low-power alternative to ANNs due to their sparsity nature. SNN, however, are not as easy to train as ANNs. In this work, we propose a hybrid SNN training scheme and apply it to segment human hippocampi from magnetic resonance images. Our approach takes ANN-SNN conversion as an initialization step and relies on spike-based backpropagation to fine-tune the network. Compared with the conversion and direct training solutions, our method has advantages in both segmentation accuracy and training efficiency. Experiments demonstrate the effectiveness of our model in achieving the design goals.注释
Accepted to ISBI 2023 conference邮件日期
2023年02月16日- NeuroHSMD: Neuromorphic Hybrid Spiking Motion Detector 时间：2023年02月14日 第一作者：Pedro Machado 链接.
邮件日期
2023年02月16日- Semiconductor Fab Scheduling with Self-Supervised and Reinforcement Learning 时间：2023年02月14日 第一作者：Pierre Tassel 链接.
摘要：半导体制造是一个众所周知的复杂且成本高昂的多步骤过程，涉及在昂贵且数量有限的设备上进行的一系列操作。最近的芯片短缺及其影响凸显了半导体在全球供应链中的重要性，以及我们对这些日常生活的依赖程度。由于建设新工厂所需的投资成本、环境影响和时间规模，当需求激增时，很难提高产量。这项工作介绍了一种使用深度强化和自我监督学习来更有效地学习调度半导体制造设施的方法。我们提出了第一种自适应调度方法来处理复杂、连续、随机、动态的现代半导体制造模型。我们的方法优于半导体制造厂通常使用的传统分层调度策略，大大减少了每个订单的延迟和完成任务的时间
英文摘要
Semiconductor manufacturing is a notoriously complex and costly multi-step process involving a long sequence of operations on expensive and quantity-limited equipment. Recent chip shortages and their impacts have highlighted the importance of semiconductors in the global supply chains and how reliant on those our daily lives are. Due to the investment cost, environmental impact, and time scale needed to build new factories, it is difficult to ramp up production when demand spikes. This work introduces a method to successfully learn to schedule a semiconductor manufacturing facility more efficiently using deep reinforcement and self-supervised learning. We propose the first adaptive scheduling approach to handle complex, continuous, stochastic, dynamic, modern semiconductor manufacturing models. Our method outperforms the traditional hierarchical dispatching strategies typically used in semiconductor manufacturing plants, substantially reducing each order's tardiness and time until completion. As a result, our method yields a better allocation of resources in the semiconductor manufacturing process.邮件日期
2023年02月15日- A Macrocolumn Architecture Implemented with Spiking Neurons 时间：2023年02月14日 第一作者：James E. Smith 链接.
注释
This is a major revision. Neuron outputs are encoded as the body potential. Winner-take-all inhibition then compares body potentials to determine a winner. At the end of each cycle, a non-zero WTA output is converted to a binary spike. This method remains consistent with temporal neuron operation internal to a cycle, with only a single bit of temporal precision being maintained between cycles MSC-class: 68T07 ACM-class: I.2; C.3邮件日期
2023年02月15日- Workload-Balanced Pruning for Sparse Spiking Neural Networks 时间：2023年02月13日 第一作者：Ruokai Yin 链接.
摘要：用于Spiking神经网络（SNN）的修剪已成为在资源受限的边缘设备上部署深度SNN的基本方法。尽管现有的修剪方法可以为深度SNN提供极高的权重稀疏性，但高权重稀疏性带来了工作负载不平衡问题。具体而言，当不同数量的非零权重被分配给并行运行的硬件单元时，会发生工作负载失衡，这导致硬件利用率低，从而导致更长的延迟和更高的能源成本。在初步实验中，我们发现稀疏SNN（$\sim$98%的权重稀疏性）的利用率可能低至$\sim$59%。为了缓解工作负载不平衡问题，我们提出了u-Ticket，在基于彩票假设（LTH）的修剪过程中，我们监控和调整SNN的权重连接，从而确保最终彩票在部署到硬件上时获得最佳利用率。实验表明，我们的u-Ticket可以保证
英文摘要
Pruning for Spiking Neural Networks (SNNs) has emerged as a fundamental methodology for deploying deep SNNs on resource-constrained edge devices. Though the existing pruning methods can provide extremely high weight sparsity for deep SNNs, the high weight sparsity brings a workload imbalance problem. Specifically, the workload imbalance happens when a different number of non-zero weights are assigned to hardware units running in parallel, which results in low hardware utilization and thus imposes longer latency and higher energy costs. In preliminary experiments, we show that sparse SNNs ($\sim$98% weight sparsity) can suffer as low as $\sim$59% utilization. To alleviate the workload imbalance problem, we propose u-Ticket, where we monitor and adjust the weight connections of the SNN during Lottery Ticket Hypothesis (LTH) based pruning, thus guaranteeing the final ticket gets optimal utilization when deployed onto the hardware. Experiments indicate that our u-Ticket can guarantee up to 100% hardware utilization, thus reducing up to 76.9% latency and 63.8% energy cost compared to the non-utilization-aware LTH method.邮件日期
2023年02月15日- Event-based Backpropagation for Analog Neuromorphic Hardware 时间：2023年02月13日 第一作者：Christian Pehle 链接.
摘要：神经形态计算旨在将研究生物神经系统的经验融入计算机架构的设计中。虽然现有方法已经成功地实现了这些计算原理的各个方面，例如基于稀疏脉冲的计算，但基于事件的可扩展学习仍然是大规模系统中难以实现的目标。然而，只有这样，神经形态系统相对于其他硬件架构的潜在能效优势才能在学习过程中实现。我们以BrainScaleS-2模拟神经形态硬件为例，介绍了EventProp算法的实现进展。以前的基于梯度的学习方法使用“替代梯度”和可观测值的密集采样，或者受到基础动力学和损失函数假设的限制。相比之下，我们的方法只需要从系统中观察脉冲时间，同时能够结合其他系统可观测值，例如膜电压测量值
英文摘要
Neuromorphic computing aims to incorporate lessons from studying biological nervous systems in the design of computer architectures. While existing approaches have successfully implemented aspects of those computational principles, such as sparse spike-based computation, event-based scalable learning has remained an elusive goal in large-scale systems. However, only then the potential energy-efficiency advantages of neuromorphic systems relative to other hardware architectures can be realized during learning. We present our progress implementing the EventProp algorithm using the example of the BrainScaleS-2 analog neuromorphic hardware. Previous gradient-based approaches to learning used "surrogate gradients" and dense sampling of observables or were limited by assumptions on the underlying dynamics and loss functions. In contrast, our approach only needs spike time observations from the system while being able to incorporate other system observables, such as membrane voltage measurements, in a principled way. This leads to a one-order-of-magnitude improvement in the information efficiency of the gradient estimate, which would directly translate to corresponding energy efficiency improvements in an optimized hardware implementation. We present the theoretical framework for estimating gradients and results verifying the correctness of the estimation, as well as results on a low-dimensional classification task using the BrainScaleS-2 system. Building on this work has the potential to enable scalable gradient estimation in large-scale neuromorphic hardware as a continuous measurement of the system state would be prohibitive and energy-inefficient in such instances. It also suggests the feasibility of a full on-device implementation of the algorithm that would enable scalable, energy-efficient, event-based learning in large-scale analog neuromorphic hardware.邮件日期
2023年02月15日- Optical Flow estimation with Event-based Cameras and Spiking Neural Networks 时间：2023年02月13日 第一作者：Javier Cuadrado 链接.
摘要：基于事件的摄像机正在引起计算机视觉界的兴趣。当自上次事件以来给定像素处的亮度变化超过某个阈值时，这些传感器以异步像素、发射事件或“脉冲”进行操作。由于其固有的特性，如低功耗、低延迟和高动态范围，它们似乎特别适合具有挑战性时间限制和安全要求的应用。基于事件的传感器非常适合于Spiking神经网络（SNN），因为异步传感器与神经形态硬件的耦合可以产生具有最小功率需求的实时系统。在这项工作中，我们试图开发一个这样的系统，使用DSEC数据集的事件传感器数据和脉冲神经网络来估计驾驶场景的光流。我们提出了一种U-Net-like SNN，该SNN在监督训练后能够进行密集光流估计。为此，我们鼓励
英文摘要
Event-based cameras are raising interest within the computer vision community. These sensors operate with asynchronous pixels, emitting events, or "spikes", when the luminance change at a given pixel since the last event surpasses a certain threshold. Thanks to their inherent qualities, such as their low power consumption, low latency and high dynamic range, they seem particularly tailored to applications with challenging temporal constraints and safety requirements. Event-based sensors are an excellent fit for Spiking Neural Networks (SNNs), since the coupling of an asynchronous sensor with neuromorphic hardware can yield real-time systems with minimal power requirements. In this work, we seek to develop one such system, using both event sensor data from the DSEC dataset and spiking neural networks to estimate optical flow for driving scenarios. We propose a U-Net-like SNN which, after supervised training, is able to make dense optical flow estimations. To do so, we encourage both minimal norm for the error vector and minimal angle between ground-truth and predicted flow, training our model with back-propagation using a surrogate gradient. In addition, the use of 3d convolutions allows us to capture the dynamic nature of the data by increasing the temporal receptive fields. Upsampling after each decoding stage ensures that each decoder's output contributes to the final estimation. Thanks to separable convolutions, we have been able to develop a light model (when compared to competitors) that can nonetheless yield reasonably accurate optical flow estimates.注释
9 pages, 3 figures and 3 tables, plus Supplementary Materials邮件日期
2023年02月14日- Sneaky Spikes: Uncovering Stealthy Backdoor Attacks in Spiking Neural Networks with Neuromorphic Data 时间：2023年02月13日 第一作者：Gorka Abad 链接.
摘要：深度神经网络（DNN）在包括图像和语音识别在内的各种任务中取得了优异的结果。然而，优化DNN的性能需要通过训练仔细调整多个超参数和网络参数。高性能DNN利用大量参数，对应于训练期间的高能量消耗。为了解决这些局限性，研究人员开发了脉冲神经网络（SNN），该网络更节能，能够以生物学上合理的方式处理数据，使其非常适合涉及感官数据处理的任务，即神经形态数据。与DNN一样，SNN也容易受到各种威胁，例如对抗性示例和后门攻击。然而，针对SNN的攻击和对策几乎完全没有被探索。本文使用神经形态数据集和不同触发器研究了后门攻击在SNN中的应用。更准确地说，神经形态数据c中的后门触发
英文摘要
Deep neural networks (DNNs) have achieved excellent results in various tasks, including image and speech recognition. However, optimizing the performance of DNNs requires careful tuning of multiple hyperparameters and network parameters via training. High-performance DNNs utilize a large number of parameters, corresponding to high energy consumption during training. To address these limitations, researchers have developed spiking neural networks (SNNs), which are more energy-efficient and can process data in a biologically plausible manner, making them well-suited for tasks involving sensory data processing, i.e., neuromorphic data. Like DNNs, SNNs are vulnerable to various threats, such as adversarial examples and backdoor attacks. Yet, the attacks and countermeasures for SNNs have been almost fully unexplored. This paper investigates the application of backdoor attacks in SNNs using neuromorphic datasets and different triggers. More precisely, backdoor triggers in neuromorphic data can change their position and color, allowing a larger range of possibilities than common triggers in, e.g., the image domain. We propose different attacks achieving up to 100\% attack success rate without noticeable clean accuracy degradation. We also evaluate the stealthiness of the attacks via the structural similarity metric, showing our most powerful attacks being also stealthy. Finally, we adapt the state-of-the-art defenses from the image domain, demonstrating they are not necessarily effective for neuromorphic data resulting in inaccurate performance.邮件日期
2023年02月14日- GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks 时间：2023年02月13日 第一作者：Xingting Yao 链接.
注释
Accepted at NeurIPS 2022邮件日期
2023年02月14日- DPSNN: A Differentially Private Spiking Neural Network 时间：2023年02月09日 第一作者：Jihang Wang 链接.
注释
We find a mistake in our experiment. This leads us to the wrong conclusion邮件日期
2023年02月10日- Dynamic Training of Liquid State Machines 时间：2023年02月06日 第一作者：Pavithra Koralalage 链接.
摘要：Spiking Neural Networks（SNNs）是人工神经网络（ANN）领域中一种很有前途的解决方案，由于其能够模拟人脑并以惊人的速度和准确性处理复杂信息，因此吸引了研究人员的注意。这项研究旨在通过确定SNN中分配的最有效的权重范围来优化液态机器（LSM）的训练过程，这是SNN的一种循环架构，以实现期望输出和实际输出之间的最小差异。实验结果表明，通过使用脉冲度量和一系列权重，可以有效优化脉冲神经元的期望输出和实际输出，从而提高SNN的性能。使用三种不同的权重初始化方法对结果进行测试和确认，使用Barabasi Albert随机图方法获得最佳结果。
英文摘要
Spiking Neural Networks (SNNs) emerged as a promising solution in the field of Artificial Neural Networks (ANNs), attracting the attention of researchers due to their ability to mimic the human brain and process complex information with remarkable speed and accuracy. This research aimed to optimise the training process of Liquid State Machines (LSMs), a recurrent architecture of SNNs, by identifying the most effective weight range to be assigned in SNN to achieve the least difference between desired and actual output. The experimental results showed that by using spike metrics and a range of weights, the desired output and the actual output of spiking neurons could be effectively optimised, leading to improved performance of SNNs. The results were tested and confirmed using three different weight initialisation approaches, with the best results obtained using the Barabasi-Albert random graph method.邮件日期
2023年02月08日- Developmental Plasticity-inspired Adaptive Pruning for Deep Spiking and Artificial Neural Networks 时间：2023年02月06日 第一作者：Bing Han 链接.
邮件日期
2023年02月07日- Reducing ANN-SNN Conversion Error through Residual Membrane Potential 时间：2023年02月04日 第一作者：Zecheng Hao 链接.
摘要：脉冲神经网络（SNN）因其在神经形态芯片上具有低功耗和高速计算的独特财产而受到广泛的学术关注。在SNN的各种训练方法中，ANN-SNN转换在大规模数据集上表现出与ANN相同的性能水平。然而，不均匀误差（指脉冲到达激活层的不同时间序列引起的偏差）尚未得到有效解决，并且在短时间步长条件下严重影响了SNN的性能。本文对不均匀误差进行了详细分析，并将其分为四类。我们指出，ANN输出为零而SNN输出大于零的情况占最大百分比。在此基础上，我们从理论上证明了这种情况的充分必要条件，并提出了一种基于剩余膜电位的优化策略，以减少不均匀误差。T
英文摘要
Spiking Neural Networks (SNNs) have received extensive academic attention due to the unique properties of low power consumption and high-speed computing on neuromorphic chips. Among various training methods of SNNs, ANN-SNN conversion has shown the equivalent level of performance as ANNs on large-scale datasets. However, unevenness error, which refers to the deviation caused by different temporal sequences of spike arrival on activation layers, has not been effectively resolved and seriously suffers the performance of SNNs under the condition of short time-steps. In this paper, we make a detailed analysis of unevenness error and divide it into four categories. We point out that the case of the ANN output being zero while the SNN output being larger than zero accounts for the largest percentage. Based on this, we theoretically prove the sufficient and necessary conditions of this case and propose an optimization strategy based on residual membrane potential to reduce unevenness error. The experimental results show that the proposed method achieves state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet datasets. For example, we reach top-1 accuracy of 64.32\% on ImageNet with 10-steps. To the best of our knowledge, this is the first time ANN-SNN conversion can simultaneously achieve high accuracy and ultra-low-latency on the complex dataset. Code is available at https://github.com/hzc1208/ANN2SNN\_SRP.注释
Accepted as a AAAI 2023 Oral Paper邮件日期
2023年02月07日- Spiking Synaptic Penalty: Appropriate Penalty Term for Energy-Efficient Spiking Neural Networks 时间：2023年02月03日 第一作者：Kazuma Suetake 链接.
摘要：脉冲神经网络（SNN）由于其脉冲特性，是一种节能的神经网络。然而，随着SNN的脉冲发射率的增加，能量消耗也会增加，因此SNN的优势会减弱。在这里，我们通过在训练阶段的目标函数中引入扣球活动的新惩罚术语来解决这个问题。我们的方法被设计为在不修改网络架构的情况下直接优化能耗度量。因此，与其他方法相比，所提出的方法可以在保持精度的同时更大地减少能耗。我们对图像分类任务进行了实验，结果表明了所提出的方法的有效性，这缓解了能量-精度权衡的困境。
英文摘要
Spiking neural networks (SNNs) are energy-efficient neural networks because of their spiking nature. However, as the spike firing rate of SNNs increases, the energy consumption does as well, and thus, the advantage of SNNs diminishes. Here, we tackle this problem by introducing a novel penalty term for the spiking activity into the objective function in the training phase. Our method is designed so as to optimize the energy consumption metric directly without modifying the network architecture. Therefore, the proposed method can reduce the energy consumption more than other methods while maintaining the accuracy. We conducted experiments for image classification tasks, and the results indicate the effectiveness of the proposed method, which mitigates the dilemma of the energy--accuracy trade-off.注释
19 pages, 5 figures邮件日期
2023年02月06日- Adaptive Sparse Structure Development with Pruning and Regeneration for Spiking Neural Networks 时间：2023年02月03日 第一作者：Bing Han 链接.
邮件日期
2023年02月06日- S$^3$NN: Time Step Reduction of Spiking Surrogate Gradients for Training Energy Efficient Single-Step Spiking Neural Networks 时间：2023年02月03日 第一作者：Kazuma Suetake 链接.
注释
23 pages, 6 figures Journal-ref: Neural Networks,159 (2023) 208-219邮件日期
2023年02月06日- CMOS-based area-and-power-efficient neuron and synapse circuits for time-domain analog spiking neural networks 时间：2023年02月02日 第一作者：Xiangyu Chen 链接.
邮件日期
2023年02月06日- Complex Dynamic Neurons Improved Spiking Transformer Network for Efficient Automatic Speech Recognition 时间：2023年02月02日 第一作者：Minglun Han 链接.
摘要：使用漏集成和激发（LIF）神经元的脉冲神经网络（SNN）已广泛用于自动语音识别（ASR）任务。然而，与生物大脑相比，LIF神经元仍然相对简单。有必要进一步研究更多具有不同神经元动力学尺度的神经元。在这里，我们引入了四种类型的神经元动力学来对脉冲变压器生成的序列模式进行后处理，以获得复杂的动态神经元改进脉冲变压器神经网络（DyTr-SNN）。我们发现DyTr-SNN可以很好地处理非玩具自动语音识别任务，代表了较低的音素错误率、较低的计算成本和较高的鲁棒性。这些结果表明，SNN和神经动力学在神经元和网络尺度上的进一步合作可能会对未来产生很大影响，特别是在ASR任务上。
英文摘要
The spiking neural network (SNN) using leaky-integrated-and-fire (LIF) neurons has been commonly used in automatic speech recognition (ASR) tasks. However, the LIF neuron is still relatively simple compared to that in the biological brain. Further research on more types of neurons with different scales of neuronal dynamics is necessary. Here we introduce four types of neuronal dynamics to post-process the sequential patterns generated from the spiking transformer to get the complex dynamic neuron improved spiking transformer neural network (DyTr-SNN). We found that the DyTr-SNN could handle the non-toy automatic speech recognition task well, representing a lower phoneme error rate, lower computational cost, and higher robustness. These results indicate that the further cooperation of SNNs and neural dynamics at the neuron and network scales might have much in store for the future, especially on the ASR tasks.注释
8 pages. Spiking Neural Networks, ASR, Speech and Language Processing. The first three authors contributed equally邮件日期
2023年02月03日- Bayesian Inference on Binary Spiking Networks Leveraging Nanoscale Device Stochasticity 时间：2023年02月02日 第一作者：Prabodh Katti 链接.
摘要：贝叶斯神经网络（BNN）可以克服困扰传统频繁使用深度神经网络的过度自信问题，因此被认为是可靠人工智能系统的关键推动者。然而，BNN的传统硬件实现是资源密集型的，需要实现用于突触采样的随机数生成器。由于其在编程和读取操作过程中固有的随机性，纳米级忆阻器件可以直接用于采样，而不需要额外的硬件资源。在本文中，我们介绍了一种新的基于相变存储器（PCM）的硬件实现，用于具有二进制突触的BNN。所提出的架构由单独的权重和噪声平面组成，其中PCM单元被配置和操作以分别表示权重的标称值和生成采样所需的噪声。使用实验观察到的PCM噪声特性作为示例性乳腺癌症数据
英文摘要
Bayesian Neural Networks (BNNs) can overcome the problem of overconfidence that plagues traditional frequentist deep neural networks, and are hence considered to be a key enabler for reliable AI systems. However, conventional hardware realizations of BNNs are resource intensive, requiring the implementation of random number generators for synaptic sampling. Owing to their inherent stochasticity during programming and read operations, nanoscale memristive devices can be directly leveraged for sampling, without the need for additional hardware resources. In this paper, we introduce a novel Phase Change Memory (PCM)-based hardware implementation for BNNs with binary synapses. The proposed architecture consists of separate weight and noise planes, in which PCM cells are configured and operated to represent the nominal values of weights and to generate the required noise for sampling, respectively. Using experimentally observed PCM noise characteristics, for the exemplary Breast Cancer Dataset classification problem, we obtain hardware accuracy and expected calibration error matching that of an 8-bit fixed-point (FxP8) implementation, with projected savings of over 9$\times$ in terms of core area transistor count.注释
Submitted and Accepted in ISCAS 2023邮件日期
2023年02月03日- Energy Efficient Training of SNN using Local Zeroth Order Method 时间：2023年02月02日 第一作者：Bhaskar Mukhoty 链接.
摘要：Spiking神经网络因其在现实世界任务中的低能量需求而变得越来越流行，其精度与传统ANN相当。SNN训练算法在最小化模型参数上的模型损失方面面临着由于Heaviside函数而导致的梯度信息损失和不可微性。为了避免这个问题，替代方法在后向传球中使用Heaviside的可微近似，而前向传球使用Heaviside作为扣球函数。我们建议在神经元层面使用零阶技术来解决这种二分法，并在自动微分工具中使用它。因此，我们在所提出的局部零阶技术和现有替代方法之间建立了理论联系，反之亦然。所提出的方法自然适用于GPU上SNN的节能训练。神经形态数据集的实验结果表明，这种实现需要少于1
英文摘要
Spiking neural networks are becoming increasingly popular for their low energy requirement in real-world tasks with accuracy comparable to the traditional ANNs. SNN training algorithms face the loss of gradient information and non-differentiability due to the Heaviside function in minimizing the model loss over model parameters. To circumvent the problem surrogate method uses a differentiable approximation of the Heaviside in the backward pass, while the forward pass uses the Heaviside as the spiking function. We propose to use the zeroth order technique at the neuron level to resolve this dichotomy and use it within the automatic differentiation tool. As a result, we establish a theoretical connection between the proposed local zeroth-order technique and the existing surrogate methods and vice-versa. The proposed method naturally lends itself to energy-efficient training of SNNs on GPUs. Experimental results with neuromorphic datasets show that such implementation requires less than 1 percent neurons to be active in the backward pass, resulting in a 100x speed-up in the backward computation time. Our method offers better generalization compared to the state-of-the-art energy-efficient technique while maintaining similar efficiency.邮件日期
2023年02月03日- OpenSpike: An OpenRAM SNN Accelerator 时间：2023年02月02日 第一作者：Farhad Modaresi 链接.
摘要：本文介绍了一种脉冲神经网络（SNN）加速器，该加速器使用完全开源的EDA工具、过程设计工具包（PDK）和使用OpenRAM合成的内存宏。该芯片采用130纳米SkyWater工艺制作，集成了超过100万个突触权重，并提供了可重新编程的架构。它的时钟速度为40 MHz，电源电压为1.8 V，使用PicoRV32内核进行控制，占地面积为33.3 mm2。加速器的吞吐量为每秒48262张图像，墙上时钟时间为20.72 us，为56.8 GOPS/W。脉冲神经元使用滞后来提供自适应阈值（即施密特触发器），这可以减少状态不稳定性。这将在一系列基准中产生高性能SNN，与最先进的全精度SNN保持竞争力。该设计是开源的，可在线获得：https://github.com/sfmth/OpenSpike
英文摘要
This paper presents a spiking neural network (SNN) accelerator made using fully open-source EDA tools, process design kit (PDK), and memory macros synthesized using OpenRAM. The chip is taped out in the 130 nm SkyWater process and integrates over 1 million synaptic weights, and offers a reprogrammable architecture. It operates at a clock speed of 40 MHz, a supply of 1.8 V, uses a PicoRV32 core for control, and occupies an area of 33.3 mm^2. The throughput of the accelerator is 48,262 images per second with a wallclock time of 20.72 us, at 56.8 GOPS/W. The spiking neurons use hysteresis to provide an adaptive threshold (i.e., a Schmitt trigger) which can reduce state instability. This results in high performing SNNs across a range of benchmarks that remain competitive with state-of-the-art, full precision SNNs. The design is open sourced and available online: https://github.com/sfmth/OpenSpike注释
The design is open sourced and available online: https://github.com/sfmth/OpenSpike邮件日期
2023年02月03日- SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks 时间：2023年02月01日 第一作者：Mingqing Xiao 链接.
摘要：具有基于事件的计算的Spiking神经网络（SNN）是神经形态硬件上节能应用的大脑启发模型。然而，大多数有监督的SNN训练方法，例如从人工神经网络转换或使用替代梯度的直接训练，需要复杂的计算，而不是在训练期间对脉冲神经元进行基于脉冲的操作。在本文中，我们研究了基于峰值的平衡状态隐式微分（SPIDE），该方法扩展了最近提出的训练方法，即平衡状态隐微分（IDE），用于纯基于峰值的计算的监督学习，这证明了SNN的能量高效训练的潜力。具体来说，我们引入了三元脉冲神经元耦合，并证明了基于该设计的脉冲可以解决隐式微分，因此整个训练过程，包括前向和后向传球，都是作为事件驱动的脉冲计算进行的
英文摘要
Spiking neural networks (SNNs) with event-based computation are promising brain-inspired models for energy-efficient applications on neuromorphic hardware. However, most supervised SNN training methods, such as conversion from artificial neural networks or direct training with surrogate gradients, require complex computation rather than spike-based operations of spiking neurons during training. In this paper, we study spike-based implicit differentiation on the equilibrium state (SPIDE) that extends the recently proposed training method, implicit differentiation on the equilibrium state (IDE), for supervised learning with purely spike-based computation, which demonstrates the potential for energy-efficient training of SNNs. Specifically, we introduce ternary spiking neuron couples and prove that implicit differentiation can be solved by spikes based on this design, so the whole training procedure, including both forward and backward passes, is made as event-driven spike computation, and weights are updated locally with two-stage average firing rates. Then we propose to modify the reset membrane potential to reduce the approximation error of spikes. With these key components, we can train SNNs with flexible structures in a small number of time steps and with firing sparsity during training, and the theoretical estimation of energy costs demonstrates the potential for high efficiency. Meanwhile, experiments show that even with these constraints, our trained models can still achieve competitive results on MNIST, CIFAR-10, CIFAR-100, and CIFAR10-DVS. Our code is available at https://github.com/pkuxmq/SPIDE-FSNN.注释
Accepted by Neural Networks DOI: 10.1016/j.neunet.2023.01.026邮件日期
2023年02月02日- Spyker: High-performance Library for Spiking Deep Neural Networks 时间：2023年01月31日 第一作者：Shahriar Rezghi Shirsavar 链接.
摘要：Spiking神经网络（SNN）最近因其有前途的能力而受到关注。与前几代神经网络相比，SNN模拟的大脑具有更高的生物学合理性。用更少的样本学习和消耗更少的功率是这些网络的关键特征之一。然而，由于仿真工具的缓慢和所提出的网络结构的不实用性，SNN的理论优势在实践中尚未显现。在这项工作中，我们使用C++/CUDA从头开始实现了一个名为Spyker的高性能库，其性能优于其前身。在这项工作中，使用Spyker以不同的学习规则（依赖于脉冲时间的可塑性和强化学习）实现了几个SNN，这些规则实现了显著更好的运行时，以证明该库在大规模网络模拟中的实用性。据我们所知，还没有开发出这样的工具来模拟大规模脉冲神经网络
英文摘要
Spiking neural networks (SNNs) have been recently brought to light due to their promising capabilities. SNNs simulate the brain with higher biological plausibility compared to previous generations of neural networks. Learning with fewer samples and consuming less power are among the key features of these networks. However, the theoretical advantages of SNNs have not been seen in practice due to the slowness of simulation tools and the impracticality of the proposed network structures. In this work, we implement a high-performance library named Spyker using C++/CUDA from scratch that outperforms its predecessor. Several SNNs are implemented in this work with different learning rules (spike-timing-dependent plasticity and reinforcement learning) using Spyker that achieve significantly better runtimes, to prove the practicality of the library in the simulation of large-scale networks. To our knowledge, no such tools have been developed to simulate large-scale spiking neural networks with high performance using a modular structure. Furthermore, a comparison of the represented stimuli extracted from Spyker to recorded electrophysiology data is performed to demonstrate the applicability of SNNs in describing the underlying neural mechanisms of the brain functions. The aim of this library is to take a significant step toward uncovering the true potential of the brain computations using SNNs.注释
11 pages, 6 figures, 6 listings ACM-class: I.2.6邮件日期
2023年02月01日- Exploiting High Performance Spiking Neural Networks with Efficient Spiking Patterns 时间：2023年01月29日 第一作者：Guobin Shen 链接.
摘要：刺突神经网络（SNN）使用离散的刺突序列来传输信息，这显著模拟了大脑的信息传输。尽管这种二值化表示形式显著提高了SNN的能量效率和鲁棒性，但它也在SNN和基于真实值的人工神经网络之间留下了很大的差距。大脑中有许多不同的脉冲模式，这些脉冲模式的动态协同作用极大地丰富了表征能力。受生物神经元中脉冲模式的启发，本文引入了动态突发模式，并设计了泄漏集成和火灾或突发（LIFB）神经元，该神经元可以从网络信息容量的角度在短时性能和动态时间性能之间进行权衡。LIFB神经元表现出三种模式，休息、规则脉冲和突发脉冲。神经元的突发密度可以自适应调整，这显著地增加了
英文摘要
Spiking Neural Networks (SNNs) use discrete spike sequences to transmit information, which significantly mimics the information transmission of the brain. Although this binarized form of representation dramatically enhances the energy efficiency and robustness of SNNs, it also leaves a large gap between the performance of SNNs and Artificial Neural Networks based on real values. There are many different spike patterns in the brain, and the dynamic synergy of these spike patterns greatly enriches the representation capability. Inspired by spike patterns in biological neurons, this paper introduces the dynamic Burst pattern and designs the Leaky Integrate and Fire or Burst (LIFB) neuron that can make a trade-off between short-time performance and dynamic temporal performance from the perspective of network information capacity. LIFB neuron exhibits three modes, resting, Regular spike, and Burst spike. The burst density of the neuron can be adaptively adjusted, which significantly enriches the characterization capability. We also propose a decoupling method that can losslessly decouple LIFB neurons into equivalent LIF neurons, which demonstrates that LIFB neurons can be efficiently implemented on neuromorphic hardware. We conducted experiments on the static datasets CIFAR10, CIFAR100, and ImageNet, which showed that we greatly improved the performance of the SNNs while significantly reducing the network latency. We also conducted experiments on neuromorphic datasets DVS-CIFAR10 and NCALTECH101 and showed that we achieved state-of-the-art with a small network structure.邮件日期
2023年01月31日- Training Full Spike Neural Networks via Auxiliary Accumulation Pathway 时间：2023年01月27日 第一作者：Guangyao Chen 链接.
摘要：由于二进制脉冲信号使得将传统的高功率乘法累加（MAC）转换为低功率累加（AC）成为可能，脑启发的脉冲神经网络（SNN）正获得越来越多的关注。然而，具有有限时间步长的全脉冲神经网络（FSNN）的二进制脉冲传播容易导致显著的信息丢失。为了提高性能，从零开始训练的几个最先进的SNN模型不可避免地带来了许多非脉冲操作。非脉冲操作会导致额外的计算消耗，并且可能不会部署在某些仅允许脉冲操作的神经形态硬件上。为了训练具有高性能的大规模FSNN，本文提出了一种新的双流训练（DST）方法，该方法将可分离的辅助累积路径（AAP）添加到全脉冲残差网络中。AAP的积累可以弥补全刺推进前后过程中的信息损失
英文摘要
Due to the binary spike signals making converting the traditional high-power multiply-accumulation (MAC) into a low-power accumulation (AC) available, the brain-inspired Spiking Neural Networks (SNNs) are gaining more and more attention. However, the binary spike propagation of the Full-Spike Neural Networks (FSNN) with limited time steps is prone to significant information loss. To improve performance, several state-of-the-art SNN models trained from scratch inevitably bring many non-spike operations. The non-spike operations cause additional computational consumption and may not be deployed on some neuromorphic hardware where only spike operation is allowed. To train a large-scale FSNN with high performance, this paper proposes a novel Dual-Stream Training (DST) method which adds a detachable Auxiliary Accumulation Pathway (AAP) to the full spiking residual networks. The accumulation in AAP could compensate for the information loss during the forward and backward of full spike propagation, and facilitate the training of the FSNN. In the test phase, the AAP could be removed and only the FSNN remained. This not only keeps the lower energy consumption but also makes our model easy to deploy. Moreover, for some cases where the non-spike operations are available, the APP could also be retained in test inference and improve feature discrimination by introducing a little non-spike consumption. Extensive experiments on ImageNet, DVS Gesture, and CIFAR10-DVS datasets demonstrate the effectiveness of DST.邮件日期
2023年01月31日- Accelerate & Actualize: Can 2D Materials Bridge the Gap Between Neuromorphic Hardware and the Human Brain? 时间：2023年01月24日 第一作者：Xiwen Liu 链接.
摘要：二维（2D）材料为冯·诺依曼计算体系结构范式以外的设备和系统提供了一个令人兴奋的机会，因为它们的电子结构、物理财产和原子薄型范德瓦尔斯结构具有多样性，能够轻松与传统电子材料和基于硅的硬件集成。所有主要类别的非易失性存储器（NVM）设备都已使用2D材料进行了演示，包括它们作为突触设备在神经形态计算硬件中的应用。它们的原子薄型结构、优异的物理财产，即机械强度、导电性和导热性，以及无法通过网关的电子财产，为NVM设备和系统提供了性能优势和新颖的功能。然而，与现有材料和技术相比，器件性能和可变性仍然是实际应用的主要问题。最终，2D材料作为一种小说的进展
英文摘要
Two-dimensional (2D) materials present an exciting opportunity for devices and systems beyond the von Neumann computing architecture paradigm due to their diversity of electronic structure, physical properties, and atomically-thin, van der Waals structures that enable ease of integration with conventional electronic materials and silicon-based hardware. All major classes of non-volatile memory (NVM) devices have been demonstrated using 2D materials, including their operation as synaptic devices for applications in neuromorphic computing hardware. Their atomically-thin structure, superior physical properties, i.e., mechanical strength, electrical and thermal conductivity, as well as gate-tunable electronic properties provide performance advantages and novel functionality in NVM devices and systems. However, device performance and variability as compared to incumbent materials and technology remain major concerns for real applications. Ultimately, the progress of 2D materials as a novel class of electronic materials and specifically their application in the area of neuromorphic electronics will depend on their scalable synthesis in thin-film form with desired crystal quality, defect density, and phase purity.注释
Neuromorphic Computing, 2D Materials, Heterostructures, Emerging Memory Devices, Resistive, Phase-Change, Ferroelectric, Ferromagnetic, Crossbar Array, Machine Learning, Deep Learning, Spiking Neural Networks邮件日期
2023年01月26日- A Comparison of Temporal Encoders for Neuromorphic Keyword Spotting with Few Neurons 时间：2023年01月24日 第一作者：Mattias Nilsson 链接.
摘要：随着人工智能驱动的虚拟助理的扩展，需要低功耗的关键字识别系统，为后续计算昂贵的语音识别提供“唤醒”机制。一种有前途的方法是使用神经形态传感器和在神经形态处理器中实现的脉冲神经网络（SNN）进行稀疏事件驱动感知。然而，这需要用于时间编码的资源高效SNN机制，其需要考虑这些系统以流方式处理信息，而物理时间是其操作的固有属性。在这项工作中，在TIDIGITS数据集中对从语音数字计算出的共振峰进行关键词定位任务中，比较研究了近期文献中描述的SNN中用于时间编码和特征提取的两个候选神经计算元素——脉冲时差编码器（TDE）和非突触兴奋抑制（E-I）元素。虽然两种编码器都提高了性能
英文摘要
With the expansion of AI-powered virtual assistants, there is a need for low-power keyword spotting systems providing a "wake-up" mechanism for subsequent computationally expensive speech recognition. One promising approach is the use of neuromorphic sensors and spiking neural networks (SNNs) implemented in neuromorphic processors for sparse event-driven sensing. However, this requires resource-efficient SNN mechanisms for temporal encoding, which need to consider that these systems process information in a streaming manner, with physical time being an intrinsic property of their operation. In this work, two candidate neurocomputational elements for temporal encoding and feature extraction in SNNs described in recent literature - the spiking time-difference encoder (TDE) and disynaptic excitatory-inhibitory (E-I) elements - are comparatively investigated in a keyword-spotting task on formants computed from spoken digits in the TIDIGITS dataset. While both encoders improve performance over direct classification of the formant features in the training data, enabling a complete binary classification with a logistic regression model, they show no clear improvements on the test set. Resource-efficient keyword spotting applications may benefit from the use of these encoders, but further work on methods for learning the time constants and weights is required to investigate their full potential.注释
This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible邮件日期
2023年01月25日- Optimising Event-Driven Spiking Neural Network with Regularisation and Cutoff 时间：2023年01月23日 第一作者：Dengyu Wu 链接.
摘要：Spiking神经网络（SNNs）是人工神经网络（ANN）的一种变体，具有能效优势，在基准数据集（如CIFAR10/100和ImageNet）上，其精度已接近ANN。然而，与基于帧的输入（例如，图像）相比，由于SNN的异步工作机制，来自动态视觉传感器（DVS）的基于事件的输入可以更好地使用SNN。在本文中，我们通过一项建议来加强SNN和基于事件的输入之间的结合，即考虑任何时间的最优推理SNN（AOI SNN），该SNN可以在推理过程中的任何时间终止以获得最优推理结果。提出了两种新的优化技术来实现AOI SNN：正则化和截止。规则化使得SNN的训练和构建具有优化的性能，而截止技术优化了SNN对事件驱动输入的推断。我们在多个benc上进行了大量的实验
英文摘要
Spiking neural networks (SNNs), a variant of artificial neural networks (ANNs) with the benefit of energy efficiency, have achieved the accuracy close to its ANN counterparts, on benchmark datasets such as CIFAR10/100 and ImageNet. However, comparing with frame-based input (e.g., images), event-based inputs from e.g., Dynamic Vision Sensor (DVS) can make a better use of SNNs thanks to the SNNs' asynchronous working mechanism. In this paper, we strengthen the marriage between SNNs and event-based inputs with a proposal to consider anytime optimal inference SNNs, or AOI-SNNs, which can terminate anytime during the inference to achieve optimal inference result. Two novel optimisation techniques are presented to achieve AOI-SNNs: a regularisation and a cutoff. The regularisation enables the training and construction of SNNs with optimised performance, and the cutoff technique optimises the inference of SNNs on event-driven inputs. We conduct an extensive set of experiments on multiple benchmark event-based datasets, including CIFAR10-DVS, N-Caltech101 and DVS128 Gesture. The experimental results demonstrate that our techniques are superior to the state-of-the-art with respect to the accuracy and latency.邮件日期
2023年01月24日- FireFly: A High-Throughput and Reconfigurable Hardware Accelerator for Spiking Neural Networks 时间：2023年01月23日 第一作者：Jindong Li 链接.
邮件日期
2023年01月24日- ETLP: Event-based Three-factor Local Plasticity for online learning with neuromorphic hardware 时间：2023年01月19日 第一作者：Fern 链接.
摘要：具有基于事件的传感器、异步硬件和脉冲神经元的神经形态感知在嵌入式系统中显示出实时高效推理的前景。脑启发计算的下一个希望是通过在线学习实现对边缘变化的适应。然而，基于共定位计算和存储器的神经形态硬件的并行和分布式架构对片上学习规则施加了局部约束。我们在这项工作中提出了基于事件的三因素局部可塑性（ETLP）规则，该规则使用（1）突触前脉冲轨迹，（2）突触后膜电压和（3）投影标签形式的第三个因素，无错误计算，也用作更新触发器。我们将具有前馈和递归脉冲神经网络的ETLP应用于基于视觉和听觉事件的模式识别，并将其与时间反向传播（BPTT）和eProp进行比较。我们在准确度方面表现出了竞争力
英文摘要
Neuromorphic perception with event-based sensors, asynchronous hardware and spiking neurons is showing promising results for real-time and energy-efficient inference in embedded systems. The next promise of brain-inspired computing is to enable adaptation to changes at the edge with online learning. However, the parallel and distributed architectures of neuromorphic hardware based on co-localized compute and memory imposes locality constraints to the on-chip learning rules. We propose in this work the Event-Based Three-factor Local Plasticity (ETLP) rule that uses (1) the pre-synaptic spike trace, (2) the post-synaptic membrane voltage and (3) a third factor in the form of projected labels with no error calculation, that also serve as update triggers. We apply ETLP with feedforward and recurrent spiking neural networks on visual and auditory event-based pattern recognition, and compare it to Back-Propagation Through Time (BPTT) and eProp. We show a competitive performance in accuracy with a clear advantage in the computational complexity for ETLP. We also show that when using local plasticity, threshold adaptation in spiking neurons and a recurrent topology are necessary to learn spatio-temporal patterns with a rich temporal structure. Finally, we provide a proof of concept hardware implementation of ETLP on FPGA to highlight the simplicity of its computational primitives and how they can be mapped into neuromorphic hardware for online learning with low-energy consumption and real-time interaction.邮件日期
2023年01月23日- Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Network 时间：2022年12月29日 第一作者：Duzhen Zhang 链接.
摘要：从相互作用中学习是生物制剂了解环境和自身的主要方式。现代深度强化学习（DRL）探索了一种从交互中学习的计算方法，并在解决各种任务方面取得了显著进展。然而，强大的DRL在能源效率方面与生物制剂仍有很大差距。虽然根本机制尚未完全了解，但我们认为神经元之间的脉冲通信和生物学上合理的突触可塑性的整合发挥了重要作用。遵循这一生物学直觉，我们通过遗传算法优化脉冲策略网络（SPN），作为DRL的节能替代方案。我们的SPN模仿昆虫的感觉运动神经元通路，并通过基于事件的脉冲进行通信。受生物学研究的启发，大脑通过形成新的突触连接形成记忆，并根据新的经验重新连接这些连接，我们调整了
英文摘要
Learning from the interaction is the primary way biological agents know about the environment and themselves. Modern deep reinforcement learning (DRL) explores a computational approach to learning from interaction and has significantly progressed in solving various tasks. However, the powerful DRL is still far from biological agents in energy efficiency. Although the underlying mechanisms are not fully understood, we believe that the integration of spiking communication between neurons and biologically-plausible synaptic plasticity plays a prominent role. Following this biological intuition, we optimize a spiking policy network (SPN) by a genetic algorithm as an energy-efficient alternative to DRL. Our SPN mimics the sensorimotor neuron pathway of insects and communicates through event-based spikes. Inspired by biological research that the brain forms memories by forming new synaptic connections and rewires these connections based on new experiences, we tune the synaptic connections instead of weights in SPN to solve given tasks. Experimental results on several robotic control tasks show that our method can achieve the performance level of mainstream DRL methods and exhibit significantly higher energy efficiency.邮件日期
2023年01月26日- Multi-compartment Neuron and Population Encoding improved Spiking Neural Network for Deep Distributional Reinforcement Learning 时间：2023年01月18日 第一作者：Yinqian Sun 链接.
摘要：受大脑中具有二进制脉冲的信息处理的启发，脉冲神经网络（SNN）表现出显著的低能耗，更适合结合多尺度生物特征。刺突神经元作为SNN的基本信息处理单元，在大多数SNN中通常被简化，它们只考虑LIF点神经元，而不考虑生物神经元的多室结构特性。这限制了SNN的计算和学习能力。在本文中，我们结合生物启发多室神经元（MCN）模型和群体编码方法，提出了一种基于脑启发SNN的深度分布强化学习算法。所提出的多室神经元构建了顶端树突、基底树突和体细胞计算室的结构和功能，以实现接近生物神经元的计算能力。此外，我们提出了一个隐式分数emb
英文摘要
Inspired by the information processing with binary spikes in the brain, the spiking neural networks (SNNs) exhibit significant low energy consumption and are more suitable for incorporating multi-scale biological characteristics. Spiking Neurons, as the basic information processing unit of SNNs, are often simplified in most SNNs which only consider LIF point neuron and do not take into account the multi-compartmental structural properties of biological neurons. This limits the computational and learning capabilities of SNNs. In this paper, we proposed a brain-inspired SNN-based deep distributional reinforcement learning algorithm with combination of bio-inspired multi-compartment neuron (MCN) model and population coding method. The proposed multi-compartment neuron built the structure and function of apical dendritic, basal dendritic, and somatic computing compartments to achieve the computational power close to that of biological neurons. Besides, we present an implicit fractional embedding method based on spiking neuron population encoding. We tested our model on Atari games, and the experiment results show that the performance of our model surpasses the vanilla ANN-based FQF model and ANN-SNN conversion method based Spiking-FQF models. The ablation experiments show that the proposed multi-compartment neural model and quantile fraction implicit population spike representation play an important role in realizing SNN-based deep distributional reinforcement learning.邮件日期
2023年01月19日- Spiking Neural Network Decision Feedback Equalization 时间：2023年01月18日 第一作者：Eike-Manuel Bansbach 链接.
注释
accepted for publication at SCC 2023邮件日期
2023年01月19日- Improving Reliability of Spiking Neural Networks through Fault Aware Threshold Voltage Optimization 时间：2023年01月12日 第一作者：Ayesha Siddique 链接.
摘要：蜘蛛神经网络借助于神经形态硬件，在计算机视觉方面取得了突破。然而，神经形态硬件缺乏并行性，因此限制了边缘设备上SNN的吞吐量和硬件加速。为了解决这个问题，最近提出了许多收缩阵列SNN加速器（systolicSNN），但其可靠性仍然是一个主要问题。在本文中，我们首先广泛分析了永久性故障对收缩神经网络的影响。然后，我们提出了一种新的故障缓解方法，即再训练中的故障感知阈值电压优化（FalVolt）。FalVolt在再培训中优化了每一层的阈值电压，以在存在故障时实现接近基线的分类精度。为了证明我们提出的缓解措施的有效性，我们在256x256系统SNN上对静态（即MNIST）和神经形态数据集（即N-MNIST和DVS手势）进行了分类，该系统具有卡住故障。我们经验丰富
英文摘要
Spiking neural networks have made breakthroughs in computer vision by lending themselves to neuromorphic hardware. However, the neuromorphic hardware lacks parallelism and hence, limits the throughput and hardware acceleration of SNNs on edge devices. To address this problem, many systolic-array SNN accelerators (systolicSNNs) have been proposed recently, but their reliability is still a major concern. In this paper, we first extensively analyze the impact of permanent faults on the SystolicSNNs. Then, we present a novel fault mitigation method, i.e., fault-aware threshold voltage optimization in retraining (FalVolt). FalVolt optimizes the threshold voltage for each layer in retraining to achieve the classification accuracy close to the baseline in the presence of faults. To demonstrate the effectiveness of our proposed mitigation, we classify both static (i.e., MNIST) and neuromorphic datasets (i.e., N-MNIST and DVS Gesture) on a 256x256 systolicSNN with stuck-at faults. We empirically show that the classification accuracy of a systolicSNN drops significantly even at extremely low fault rates (as low as 0.012\%). Our proposed FalVolt mitigation method improves the performance of systolicSNNs by enabling them to operate at fault rates of up to 60\%, with a negligible drop in classification accuracy (as low as 0.1\%). Our results show that FalVolt is 2x faster compared to other state-of-the-art techniques common in artificial neural networks (ANNs), such as fault-aware pruning and retraining without threshold voltage optimization.注释
Accepted full paper in DATE 2023邮件日期
2023年01月16日- Security-Aware Approximate Spiking Neural Networks 时间：2023年01月12日 第一作者：Syed Tihaam Ahmad 链接.
摘要：深度神经网络（DNN）和刺突神经网络（SNN）都以其对对抗性攻击的敏感性而闻名。因此，最近的研究人员广泛研究了DNN和SNN在对抗性攻击下的鲁棒性和防御。与精确SNN（AccSNN）相比，已知近似SNN（AxSNN）在超低功率应用中的能效高达4倍。不幸的是，AxSNN在对抗性攻击下的鲁棒性尚未得到研究。在本文中，我们首先广泛分析了具有不同结构参数和近似水平的AxSNN在两种基于梯度和两种神经形态攻击下的鲁棒性。然后，我们提出了两种新的防御方法，即精确缩放和近似量化感知滤波（AQF），以保护AxSNN。我们使用静态和神经形态数据集评估了这两种防御方法的有效性。我们的结果表明，AxSNN更容易受到对抗性攻击
英文摘要
Deep Neural Networks (DNNs) and Spiking Neural Networks (SNNs) are both known for their susceptibility to adversarial attacks. Therefore, researchers in the recent past have extensively studied the robustness and defense of DNNs and SNNs under adversarial attacks. Compared to accurate SNNs (AccSNN), approximate SNNs (AxSNNs) are known to be up to 4X more energy-efficient for ultra-low power applications. Unfortunately, the robustness of AxSNNs under adversarial attacks is yet unexplored. In this paper, we first extensively analyze the robustness of AxSNNs with different structural parameters and approximation levels under two gradient-based and two neuromorphic attacks. Then, we propose two novel defense methods, i.e., precision scaling and approximate quantization-aware filtering (AQF), for securing AxSNNs. We evaluated the effectiveness of these two defense methods using both static and neuromorphic datasets. Our results demonstrate that AxSNNs are more prone to adversarial attacks than AccSNNs, but precision scaling and AQF significantly improve the robustness of AxSNNs. For instance, a PGD attack on AxSNN results in a 72\% accuracy loss compared to AccSNN without any attack, whereas the same attack on the precision-scaled AxSNN leads to only a 17\% accuracy loss in the static MNIST dataset (4X robustness improvement). Similarly, a Sparse Attack on AxSNN leads to a 77\% accuracy loss when compared to AccSNN without any attack, whereas the same attack on an AxSNN with AQF leads to only a 2\% accuracy loss in the neuromorphic DVS128 Gesture dataset (38X robustness improvement).注释
Accepted full paper in DATE 2023邮件日期
2023年01月16日- Semantic match: Debugging feature attribution methods in XAI for healthcare 时间：2023年01月05日 第一作者：Giovanni Cin`a 链接.
摘要：最近，经认证的医疗保健人工智能（AI）工具激增，重新引发了围绕采用这项技术的争论。这类争论的一个线索涉及可解释的人工智能及其使人工智能设备更加透明和可信的承诺。一些活跃在医学人工智能领域的声音对可解释人工智能技术的可靠性表示担忧，特别是特征归因方法，质疑其在指南和标准中的使用和包含。尽管存在合理的担忧，但我们认为，对事后局部解释方法可行性的现有批评通过概括图像数据特有的问题，将婴儿与洗澡水混为一谈。我们首先将问题描述为解释和人类理解之间缺乏语义匹配。为了了解何时可以可靠地使用特征重要性，我们引入了低级别和高级别特征的特征重要性之间的区别。我们认为，对于低级别的数据类型
英文摘要
The recent spike in certified Artificial Intelligence (AI) tools for healthcare has renewed the debate around adoption of this technology. One thread of such debate concerns Explainable AI and its promise to render AI devices more transparent and trustworthy. A few voices active in the medical AI space have expressed concerns on the reliability of Explainable AI techniques and especially feature attribution methods, questioning their use and inclusion in guidelines and standards. Despite valid concerns, we argue that existing criticism on the viability of post-hoc local explainability methods throws away the baby with the bathwater by generalizing a problem that is specific to image data. We begin by characterizing the problem as a lack of semantic match between explanations and human understanding. To understand when feature importance can be used reliably, we introduce a distinction between feature importance of low- and high-level features. We argue that for data types where low-level features come endowed with a clear semantics, such as tabular data like Electronic Health Records (EHRs), semantic match can be obtained, and thus feature attribution methods can still be employed in a meaningful and useful way.邮件日期
2023年01月06日- FireFly: A High-Throughput and Reconfigurable Hardware Accelerator for Spiking Neural Networks 时间：2023年01月05日 第一作者：Jindong Li 链接.
摘要：Spiking神经网络（SNN）由于其强大的生物学解释性和高能量效率而被广泛应用。随着反向传播算法和替代梯度的引入，脉冲神经网络的结构变得更加复杂，与人工神经网络的性能差距逐渐缩小。然而，用于现场可编程门阵列（FPGA）的大多数SNN硬件实现不能满足算术或存储器效率要求，这严重限制了SNN的发展。他们不会深入研究二进制脉冲和突触权重之间的算术运算，也不会通过在小任务上使用过于昂贵的设备来假设无限制的片上RAM资源。为了提高运算效率，我们分析了脉冲神经元的神经动力学，将SNN算术运算推广到多路累加运算，并提出了利用DSP48E2硬件实现这种运算的高性能方法
英文摘要
Spiking neural networks (SNNs) have been widely used due to their strong biological interpretability and high energy efficiency. With the introduction of the backpropagation algorithm and surrogate gradient, the structure of spiking neural networks has become more complex, and the performance gap with artificial neural networks has gradually decreased. However, most SNN hardware implementations for field-programmable gate arrays (FPGAs) cannot meet arithmetic or memory efficiency requirements, which significantly restricts the development of SNNs. They do not delve into the arithmetic operations between the binary spikes and synaptic weights or assume unlimited on-chip RAM resources by using overly expensive devices on small tasks. To improve arithmetic efficiency, we analyze the neural dynamics of spiking neurons, generalize the SNN arithmetic operation to the multiplex-accumulate operation, and propose a high-performance implementation of such operation by utilizing the DSP48E2 hard block in Xilinx Ultrascale FPGAs. To improve memory efficiency, we design a memory system to enable efficient synaptic weights and membrane voltage memory access with reasonable on-chip RAM consumption. Combining the above two improvements, we propose an FPGA accelerator that can process spikes generated by the firing neuron on-the-fly (FireFly). FireFly is implemented on several FPGA edge devices with limited resources but still guarantees a peak performance of 5.53TSOP/s at 300MHz. As a lightweight accelerator, FireFly achieves the highest computational density efficiency compared with existing research using large FPGA devices.邮件日期
2023年01月06日- Online Training Through Time for Spiking Neural Networks 时间：2022年12月31日 第一作者：Mingqing Xiao 链接.
注释
Accepted by NeurIPS 2022邮件日期
2023年01月03日- Power Quality Event Recognition and Classification Using an Online Sequential Extreme Learning Machine Network based on Wavelets 时间：2022年12月27日 第一作者：Rahul Kumar Dubey 链接.
摘要：降低的系统可靠性和较高的维护成本可能是电力质量差的结果，这可能会干扰正常的设备性能，加速老化，甚至导致彻底的故障。本研究实现并测试了一个基于小波的在线顺序极端学习机（OS-ELM）分类器原型，用于检测瞬态条件下的电能质量问题。为了创建分类器，将OSELM网络模型和离散小波变换（DWT）方法相结合。首先，使用离散小波变换（DWT）多分辨率分析（MRA）提取不同分辨率下的失真信号特征。OSELM然后根据瞬态持续时间和能量特征对检索到的数据进行排序，以确定扰动的类型。所建议的方法需要更少的存储空间和处理时间，因为它可以在不改变信号原始质量的情况下最小化大量失真信号的特性
英文摘要
Reduced system dependability and higher maintenance costs may be the consequence of poor electric power quality, which can disturb normal equipment performance, speed up aging, and even cause outright failures. This study implements and tests a prototype of an Online Sequential Extreme Learning Machine (OS-ELM) classifier based on wavelets for detecting power quality problems under transient conditions. In order to create the classifier, the OSELM-network model and the discrete wavelet transform (DWT) method are combined. First, discrete wavelet transform (DWT) multi-resolution analysis (MRA) was used to extract characteristics of the distorted signal at various resolutions. The OSELM then sorts the retrieved data by transient duration and energy features to determine the kind of disturbance. The suggested approach requires less memory space and processing time since it can minimize a large quantity of the distorted signal's characteristics without changing the signal's original quality. Several types of transient events were used to demonstrate the classifier's ability to detect and categorize various types of power disturbances, including sags, swells, momentary interruptions, oscillatory transients, harmonics, notches, spikes, flickers, sag swell, sag mi, sag harm, swell trans, sag spike, and swell spike.邮件日期
2022年12月29日- Closed-form control with spike coding networks 时间：2022年12月25日 第一作者：Filip S. Slijkhuis 链接.
摘要：使用脉冲神经网络（SNN）的有效和鲁棒控制仍然是一个开放的问题。虽然生物制剂的行为是通过稀疏和不规则的脉冲模式产生的，这种模式提供了鲁棒和有效的控制，但用于控制的大多数人工脉冲神经网络中的活动模式都是密集和规则的，这可能导致效率较低的代码。此外，对于大多数现有的控制解决方案，甚至对于完全识别的系统，网络培训或优化都是必要的，这使其在片上低功耗解决方案中的实现变得复杂。棘波编码网络（SCNs）的神经科学理论为在递归棘波神经网络中实现动态系统提供了一种完全解析的解决方案——同时保持不规则、稀疏和鲁棒的棘波活动——但目前还不清楚如何将其直接应用于控制问题。在这里，我们通过结合闭式最优估计和控制来扩展SCN理论。由此产生的网络
英文摘要
Efficient and robust control using spiking neural networks (SNNs) is still an open problem. Whilst behaviour of biological agents is produced through sparse and irregular spiking patterns, which provide both robust and efficient control, the activity patterns in most artificial spiking neural networks used for control are dense and regular -- resulting in potentially less efficient codes. Additionally, for most existing control solutions network training or optimization is necessary, even for fully identified systems, complicating their implementation in on-chip low-power solutions. The neuroscience theory of Spike Coding Networks (SCNs) offers a fully analytical solution for implementing dynamical systems in recurrent spiking neural networks -- while maintaining irregular, sparse, and robust spiking activity -- but it's not clear how to directly apply it to control problems. Here, we extend SCN theory by incorporating closed-form optimal estimation and control. The resulting networks work as a spiking equivalent of a linear-quadratic-Gaussian controller. We demonstrate robust spiking control of simulated spring-mass-damper and cart-pole systems, in the face of several perturbations, including input- and system-noise, system disturbances, and neural silencing. As our approach does not need learning or optimization, it offers opportunities for deploying fast and efficient task-specific on-chip spiking controllers with biologically realistic activity.注释
Under review in an IEEE journal邮件日期
2022年12月27日- Sitting Posture Recognition Using a Spiking Neural Network 时间：2022年12月25日 第一作者：Jianquan Wang 链接.
摘要：为了提高市民的生活质量，我们设计了一个个性化的智能椅子系统来识别坐姿。该系统可以从设计的传感器接收表面压力数据，并提供用于引导用户朝向正确坐姿的反馈。我们使用液体状态机和逻辑回归分类器构建了一个脉冲神经网络，用于对15种坐姿进行分类。为了让这个系统能够将我们的压力数据读入脉冲神经元，我们设计了一种算法，将类似地图的数据编码为余弦秩稀疏数据。由19名参与者的15个坐姿组成的实验结果表明，我们的SNN的预测精度为88.52%。
英文摘要
To increase the quality of citizens' lives, we designed a personalized smart chair system to recognize sitting behaviors. The system can receive surface pressure data from the designed sensor and provide feedback for guiding the user towards proper sitting postures. We used a liquid state machine and a logistic regression classifier to construct a spiking neural network for classifying 15 sitting postures. To allow this system to read our pressure data into the spiking neurons, we designed an algorithm to encode map-like data into cosine-rank sparsity data. The experimental results consisting of 15 sitting postures from 19 participants show that the prediction precision of our SNN is 88.52%.邮件日期
2022年12月27日- Mantis: Enabling Energy-Efficient Autonomous Mobile Agents with Spiking Neural Networks 时间：2022年12月24日 第一作者：Rachmad Vidya Wicaksana Putra 链接.
摘要：无人驾驶飞行器（UAV）和移动机器人等自主移动代理在提高人类生产力方面显示出巨大潜力。由于这些移动代理通常由电池供电，因此它们需要低功率/能量消耗以具有较长的寿命。这些代理还需要适应不断变化的/动态的环境，特别是当部署在遥远或危险的位置时，因此需要高效的在线学习能力。这些要求可以通过使用Spiking神经网络（SNN）来满足，因为SNN由于稀疏计算而提供低功耗/能耗。然而，仍然需要一种方法来在自主移动代理上使用适当的SNN模型。为此，我们提出了一种Mantis方法，在自主移动代理上系统地使用SNN，以在动态环境中实现节能处理和自适应能力。我们螳螂的主要**包括
英文摘要
Autonomous mobile agents such as unmanned aerial vehicles (UAVs) and mobile robots have shown huge potential for improving human productivity. These mobile agents require low power/energy consumption to have a long lifespan since they are usually powered by batteries. These agents also need to adapt to changing/dynamic environments, especially when deployed in far or dangerous locations, thus requiring efficient online learning capabilities. These requirements can be fulfilled by employing Spiking Neural Networks (SNNs) since SNNs offer low power/energy consumption due to sparse computations and efficient online learning due to bio-inspired learning mechanisms. However, a methodology is still required to employ appropriate SNN models on autonomous mobile agents. Towards this, we propose a Mantis methodology to systematically employ SNNs on autonomous mobile agents to enable energy-efficient processing and adaptive capabilities in dynamic environments. The key ideas of our Mantis include the optimization of SNN operations, the employment of a bio-plausible online learning mechanism, and the SNN model selection. The experimental results demonstrate that our methodology maintains high accuracy with a significantly smaller memory footprint and energy consumption (i.e., 3.32x memory reduction and 2.9x energy saving for an SNN model with 8-bit weights) compared to the baseline network with 32-bit weights. In this manner, our Mantis enables the employment of SNNs for resource- and energy-constrained mobile agents.注释
To appear at the 2023 International Conference on Automation, Robotics and Applications (ICARA), February 2023, Abu Dhabi, UAE. arXiv admin note: text overlap with arXiv:2206.08656邮件日期
2022年12月27日- hxtorch.snn: Machine-learning-inspired Spiking Neural Network Modeling on BrainScaleS-2 时间：2022年12月23日 第一作者：Philipp Spilger 链接.
摘要：神经形态系统需要用户友好的软件来支持实验的设计和优化。在这项工作中，我们通过介绍我们为BrainScaleS-2神经形态系统开发的基于机器学习的建模框架来解决这一需求。这项工作比以前的工作有所改进，以前的工作要么侧重于BrainScaleS-2的矩阵乘法模式，要么缺乏完全自动化。我们的框架名为hxtorch.snn，支持PyTorch内的脉冲神经网络的硬件在环训练，包括支持全自动硬件实验工作流中的自动区分。此外，hxtorch.snn促进了硬件仿真和软件仿真之间的无缝转换。我们使用阴阳数据集展示了hxtorch.snn在分类任务中的能力，该数据集采用基于梯度的方法，具有替代梯度和BrainScaleS-2硬件系统中密集采样的膜观察结果。
英文摘要
Neuromorphic systems require user-friendly software to support the design and optimization of experiments. In this work, we address this need by presenting our development of a machine learning-based modeling framework for the BrainScaleS-2 neuromorphic system. This work represents an improvement over previous efforts, which either focused on the matrix-multiplication mode of BrainScaleS-2 or lacked full automation. Our framework, called hxtorch.snn, enables the hardware-in-the-loop training of spiking neural networks within PyTorch, including support for auto differentiation in a fully-automated hardware experiment workflow. In addition, hxtorch.snn facilitates seamless transitions between emulating on hardware and simulating in software. We demonstrate the capabilities of hxtorch.snn on a classification task using the Yin-Yang dataset employing a gradient-based approach with surrogate gradients and densely sampled membrane observations from the BrainScaleS-2 hardware system.邮件日期
2022年12月26日- An Exact Mapping From ReLU Networks to Spiking Neural Networks 时间：2022年12月23日 第一作者：Ana Stanojevic 链接.
摘要：深度脉冲神经网络（SNN）提供了低功耗人工智能的前景。然而，从头开始训练深度SNN或将深度人工神经网络转换为SNN而不损失性能一直是一个挑战。在这里，我们提出了一个从具有整流线性单元（ReLU）的网络到每个神经元只发射一个脉冲的SNN的精确映射。对于我们的构造性证明，我们假设具有或不具有卷积层、批归一化和最大池化层的任意多层ReLU网络在某些训练集上被训练为高性能。此外，我们假设我们可以访问训练期间使用的输入数据的代表性示例以及训练后的ReLU网络的精确参数（权重和偏差）。从深度ReLU网络到SNN的映射导致CIFAR10、CIFAR100和类似ImageNet的数据集Places365和PASS的准确度下降了零。更一般地说，我们的工作表明，任意深度ReLU的网络都可以被
英文摘要
Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an exact mapping from a network with Rectified Linear Units (ReLUs) to an SNN that fires exactly one spike per neuron. For our constructive proof, we assume that an arbitrary multi-layer ReLU network with or without convolutional layers, batch normalization and max pooling layers was trained to high performance on some training set. Furthermore, we assume that we have access to a representative example of input data used during training and to the exact parameters (weights and biases) of the trained ReLU network. The mapping from deep ReLU networks to SNNs causes zero percent drop in accuracy on CIFAR10, CIFAR100 and the ImageNet-like data sets Places365 and PASS. More generally our work shows that an arbitrary deep ReLU network can be replaced by an energy-efficient single-spike neural network without any loss of performance.邮件日期
2022年12月26日- In-Sensor & Neuromorphic Computing are all you need for Energy Efficient Computer Vision 时间：2022年12月21日 第一作者：Gourav Datta 链接.
摘要：由于高激活稀疏性和使用累积（AC）而不是昂贵的乘法和累积（MAC），神经形态脉冲神经网络（SNN）已成为几种计算机视觉（CV）应用中传统DNN的低功耗替代方案。然而，大多数现有的SNN需要多个时间步骤来获得可接受的推理精度，这阻碍了实时部署，增加了峰值活动，从而增加了能耗。最近的工作提出了直接编码，其直接馈送SNN的第一层中的模拟像素值，以显著减少时间步长的数量。尽管具有直接编码的第一层MAC的开销对于深度SNN是可忽略的，并且CV处理使用SNN是有效的，但是图像传感器和下游处理之间的数据传输消耗了大量带宽，并且可能支配总能量。为了减轻这种担忧，我们提出了一种传感器内计算硬件软件
英文摘要
Due to the high activation sparsity and use of accumulates (AC) instead of expensive multiply-and-accumulates (MAC), neuromorphic spiking neural networks (SNNs) have emerged as a promising low-power alternative to traditional DNNs for several computer vision (CV) applications. However, most existing SNNs require multiple time steps for acceptable inference accuracy, hindering real-time deployment and increasing spiking activity and, consequently, energy consumption. Recent works proposed direct encoding that directly feeds the analog pixel values in the first layer of the SNN in order to significantly reduce the number of time steps. Although the overhead for the first layer MACs with direct encoding is negligible for deep SNNs and the CV processing is efficient using SNNs, the data transfer between the image sensors and the downstream processing costs significant bandwidth and may dominate the total energy. To mitigate this concern, we propose an in-sensor computing hardware-software co-design framework for SNNs targeting image recognition tasks. Our approach reduces the bandwidth between sensing and processing by 12-96x and the resulting total energy by 2.32x compared to traditional CV processing, with a 3.8% reduction in accuracy on ImageNet.邮件日期
2022年12月22日- Emergent communication enhances foraging behaviour in evolved swarms controlled by Spiking Neural Networks 时间：2022年12月16日 第一作者：Cristian Jimenez Romero 链接.
摘要：蚂蚁等群居昆虫通过信息素进行交流，信息素使它们能够协调活动，以群体的形式解决复杂的任务，例如觅食。这种行为是通过进化过程形成的。在计算模型中，使用概率或行动规则来实现群体中的自我协调，以形成每个代理的决策和集体行为。然而，人工调整的决策规则可能会限制群体的行为。在这项工作中，我们在没有定义任何规则的情况下，研究了进化群体中自我协调和沟通的出现。我们进化出一群代表蚁群的代理。我们使用遗传算法来优化脉冲神经网络（SNN），该网络充当人工大脑来控制每个代理的行为。蜂群的目标是在最短的时间内找到最佳的觅食方式。在进化阶段，蚂蚁能够通过在食物附近沉积信息素来学习协作
英文摘要
Social insects such as ants communicate via pheromones which allows them to coordinate their activity and solve complex tasks as a swarm, e.g. foraging for food. This behaviour was shaped through evolutionary processes. In computational models, self-coordination in swarms has been implemented using probabilistic or action rules to shape the decision of each agent and the collective behaviour. However, manual tuned decision rules may limit the behaviour of the swarm. In this work we investigate the emergence of self-coordination and communication in evolved swarms without defining any rule. We evolve a swarm of agents representing an ant colony. We use a genetic algorithm to optimize a spiking neural network (SNN) which serves as an artificial brain to control the behaviour of each agent. The goal of the colony is to find optimal ways to forage for food in the shortest amount of time. In the evolutionary phase, the ants are able to learn to collaborate by depositing pheromone near food piles and near the nest to guide its cohorts. The pheromone usage is not encoded into the network; instead, this behaviour is established through the optimization procedure. We observe that pheromone-based communication enables the ants to perform better in comparison to colonies where communication did not emerge. We assess the foraging performance by comparing the SNN based model to a rule based system. Our results show that the SNN based model can complete the foraging task more efficiently in a shorter time. Our approach illustrates that even in the absence of pre-defined rules, self coordination via pheromone emerges as a result of the network optimization. This work serves as a proof of concept for the possibility of creating complex applications utilizing SNNs as underlying architectures for multi-agent interactions where communication and self-coordination is desired.注释
18, pages, 8 figures邮件日期
2022年12月19日- Securing the Spike: On the Transferability and Security of Spiking Neural Networks to Adversarial Examples 时间：2022年12月12日 第一作者：Nuo Xu 链接.
邮件日期
2022年12月14日- Energy-based General Sequential Episodic Memory Networks at the Adiabatic Limit 时间：2022年12月11日 第一作者：Arjun Karuvally 链接.
摘要：通用联想记忆模型（GAMM）具有一个恒定的依赖于状态的能量表面，它将输出动态引向固定点，从一组可以异步预加载的存储器中检索单个存储器。我们引入了一类新的通用顺序情节记忆模型（GSEMM），该模型在绝热极限下表现出随时间变化的能量表面，导致一系列亚稳态，即顺序情节记忆。动态能量表面是通过新引入的不对称突触实现的，在网络的隐藏层中具有信号传播延迟。我们研究了GSEMM类的两个记忆模型的理论和经验性质，它们的激活函数不同。LISEM在特征层具有非线性，而DSEM在隐藏层具有非线性。原则上，DSEM的存储容量随网络中神经元的数量呈指数增长。我们介绍了突触基础的学习规则
英文摘要
The General Associative Memory Model (GAMM) has a constant state-dependant energy surface that leads the output dynamics to fixed points, retrieving single memories from a collection of memories that can be asynchronously preloaded. We introduce a new class of General Sequential Episodic Memory Models (GSEMM) that, in the adiabatic limit, exhibit temporally changing energy surface, leading to a series of meta-stable states that are sequential episodic memories. The dynamic energy surface is enabled by newly introduced asymmetric synapses with signal propagation delays in the network's hidden layer. We study the theoretical and empirical properties of two memory models from the GSEMM class, differing in their activation functions. LISEM has non-linearities in the feature layer, whereas DSEM has non-linearity in the hidden layer. In principle, DSEM has a storage capacity that grows exponentially with the number of neurons in the network. We introduce a learning rule for the synapses based on the energy minimization principle and show it can learn single memories and their sequential relationships online. This rule is similar to the Hebbian learning algorithm and Spike-Timing Dependent Plasticity (STDP), which describe conditions under which synapses between neurons change strength. Thus, GSEMM combines the static and dynamic properties of episodic memory under a single theoretical framework and bridges neuroscience, machine learning, and artificial intelligence.邮件日期
2022年12月13日- A Topological Deep Learning Framework for Neural Spike Decoding 时间：2022年12月01日 第一作者：Edward C. Mitchell 链接.
摘要：大脑的空间定向系统使用不同的神经元群来辅助基于环境的导航。大脑对空间信息进行编码的方式之一是通过网格细胞，即覆盖在一起的多层神经元，提供基于环境的导航。这些神经元在一个集合中激发，其中几个神经元同时激发以激活单个网格。我们希望捕获这个激发结构，并使用它来解码网格单元数据。理解、表示和解码这些神经结构需要包含比传统基于图的模型更高阶连接性的模型。为此，在这项工作中，我们开发了一个用于神经脉冲序列解码的拓扑深度学习框架。我们的框架将无监督的简单复杂发现与深度学习的能力相结合，通过我们在此开发的一种新架构，称为简单卷积递归神经网络（SCRNN）。简单复形，不仅使用顶点和边的拓扑空间
英文摘要
The brain's spatial orientation system uses different neuron ensembles to aid in environment-based navigation. One of the ways brains encode spatial information is through grid cells, layers of decked neurons that overlay to provide environment-based navigation. These neurons fire in ensembles where several neurons fire at once to activate a single grid. We want to capture this firing structure and use it to decode grid cell data. Understanding, representing, and decoding these neural structures require models that encompass higher order connectivity than traditional graph-based models may provide. To that end, in this work, we develop a topological deep learning framework for neural spike train decoding. Our framework combines unsupervised simplicial complex discovery with the power of deep learning via a new architecture we develop herein called a simplicial convolutional recurrent neural network (SCRNN). Simplicial complexes, topological spaces that use not only vertices and edges but also higher-dimensional objects, naturally generalize graphs and capture more than just pairwise relationships. Additionally, this approach does not require prior knowledge of the neural activity beyond spike counts, which removes the need for similarity measurements. The effectiveness and versatility of the SCRNN is demonstrated on head direction data to test its performance and then applied to grid cell datasets with the task to automatically predict trajectories.邮件日期
2022年12月12日- Fast Exploration of the Impact of Precision Reduction on Spiking Neural Networks 时间：2022年11月22日 第一作者：Sepide Saeedi 链接.
摘要：近似计算（AxC）技术以计算精度换取性能、能量和面积减少增益。当应用程序本质上能够容忍某些精度损失时，这种权衡尤其方便，如Spiking神经网络（SNN）的情况。当目标硬件达到计算边缘时，SNN是一个实用的选择，但这需要一些面积最小化策略。在这项工作中，我们使用区间算术（IA）模型来开发一种探索方法，该方法利用这种模型传播近似误差的能力，以检测何时近似超过应用程序的可容忍极限。实验结果证实了显著减少探测时间的能力，为进一步减小网络参数的大小提供了机会，并获得了更细粒度的结果。
英文摘要
Approximate Computing (AxC) techniques trade off the computation accuracy for performance, energy, and area reduction gains. The trade-off is particularly convenient when the applications are intrinsically tolerant to some accuracy loss, as in the Spiking Neural Networks (SNNs) case. SNNs are a practical choice when the target hardware reaches the edge of computing, but this requires some area minimization strategies. In this work, we employ an Interval Arithmetic (IA) model to develop an exploration methodology that takes advantage of the capability of such a model to propagate the approximation error to detect when the approximation exceeds tolerable limits by the application. Experimental results confirm the capability of reducing the exploration time significantly, providing the chance to reduce the network parameters' size further and with more fine-grained results.邮件日期
2022年12月23日- Models Developed for Spiking Neural Networks 时间：2022年12月08日 第一作者：Shahriar Rezghi Shirsavar 链接.
摘要：深度神经网络（DNN）的出现再次引起了人们对人工神经网络（ANN）的极大关注。它们已经成为最先进的模型，并赢得了不同的机器学习挑战。尽管这些网络受到大脑的启发，但它们缺乏生物学上的合理性，与大脑相比，它们具有结构上的差异。Spiking神经网络（SNN）已经存在了很长一段时间，并且已经对其进行了研究，以了解大脑的动态。然而，它们在现实世界和复杂的机器学习任务中的应用有限。最近，他们在解决此类任务方面表现出了巨大的潜力。由于它们的能源效率和时间动态，它们的未来发展前景广阔。在这项工作中，我们回顾了SNN在图像分类任务中的结构和性能。这些比较表明，这些网络对于更复杂的问题表现出强大的能力。此外，简单的学习
英文摘要
Emergence of deep neural networks (DNNs) has raised enormous attention towards artificial neural networks (ANNs) once again. They have become the state-of-the-art models and have won different machine learning challenges. Although these networks are inspired by the brain, they lack biological plausibility, and they have structural differences compared to the brain. Spiking neural networks (SNNs) have been around for a long time, and they have been investigated to understand the dynamics of the brain. However, their application in real-world and complicated machine learning tasks were limited. Recently, they have shown great potential in solving such tasks. Due to their energy efficiency and temporal dynamics there are many promises in their future development. In this work, we reviewed the structures and performances of SNNs on image classification tasks. The comparisons illustrate that these networks show great capabilities for more complicated problems. Furthermore, the simple learning rules developed for SNNs, such as STDP and R-STDP, can be a potential alternative to replace the backpropagation algorithm used in DNNs.注释
9 pages, 4 figures, 2 tables ACM-class: I.2.6邮件日期
2022年12月09日- Learning to See Through with Events 时间：2022年12月05日 第一作者：Lei Yu 链接.
摘要：尽管合成孔径成像（SAI）可以通过模糊掉离焦前景遮挡，同时从多视图图像中恢复焦内遮挡场景来实现透视效果，但其性能通常会因密集遮挡和极端照明条件而恶化。为了解决这个问题，本文提出了一种基于事件的SAI（E-SAI）方法，该方法依赖于事件摄像机获取的具有极低延迟和高动态范围的异步事件。具体地说，收集的事件首先由重新聚焦网络模块重新聚焦，以对齐对焦事件，同时分散离焦事件。随后，提出了一种由脉冲神经网络（SNN）和卷积神经网络（CNN）组成的混合网络，以编码来自重新聚焦事件的时空信息，并重建被遮挡目标的视觉图像。大量实验表明，我们提出的E-SAI方法在处理非常密集的
英文摘要
Although synthetic aperture imaging (SAI) can achieve the seeing-through effect by blurring out off-focus foreground occlusions while recovering in-focus occluded scenes from multi-view images, its performance is often deteriorated by dense occlusions and extreme lighting conditions. To address the problem, this paper presents an Event-based SAI (E-SAI) method by relying on the asynchronous events with extremely low latency and high dynamic range acquired by an event camera. Specifically, the collected events are first refocused by a Refocus-Net module to align in-focus events while scattering out off-focus ones. Following that, a hybrid network composed of spiking neural networks (SNNs) and convolutional neural networks (CNNs) is proposed to encode the spatio-temporal information from the refocused events and reconstruct a visual image of the occluded targets. Extensive experiments demonstrate that our proposed E-SAI method can achieve remarkable performance in dealing with very dense occlusions and extreme lighting conditions and produce high-quality images from pure events. Codes and datasets are available at https://dvs-whu.cn/projects/esai/.注释
Accepted by IEEE TPAMI. arXiv admin note: text overlap with arXiv:2103.02376邮件日期
2022年12月06日- Adversarial Attacks on Spiking Convolutional Neural Networks for Event-based Vision 时间：2022年12月05日 第一作者：Julian B"uchel 链接.
注释
9 pages plus Supplementary Material. Accepted in Frontiers in Neuroscience -- Neuromorphic Engineering邮件日期
2022年12月06日- THOR -- A Neuromorphic Processor with 7.29G TSOP$^2$/mm$^2$Js Energy-Throughput Efficiency 时间：2022年12月03日 第一作者：Mayank Senapati 链接.
摘要：使用生物启发的Spiking神经网络（SNN）进行神经形态计算是一种很有前途的解决方案，可以满足边缘计算设备所需的能量吞吐量（ET）效率。已经提出了在模拟/混合信号域中仿真SNN的神经形态硬件架构，以实现比所有数字架构更高数量级的能量效率，然而代价是有限的可扩展性、对噪声的敏感性、复杂的验证和较差的灵活性。另一方面，最先进的数字神经形态架构关注于实现高能量效率（焦耳/突触操作（SOP））或吞吐量效率（SOP/秒/面积），导致ET效率低下。在这项工作中，我们介绍了THOR，一种全数字神经形态处理器，具有新颖的存储器层次结构和神经元更新架构，解决了能耗和吞吐量瓶颈。我们在28nm FDSOI CMOS技术中实现了THOR，我们的后布局结果要求
英文摘要
Neuromorphic computing using biologically inspired Spiking Neural Networks (SNNs) is a promising solution to meet Energy-Throughput (ET) efficiency needed for edge computing devices. Neuromorphic hardware architectures that emulate SNNs in analog/mixed-signal domains have been proposed to achieve order-of-magnitude higher energy efficiency than all-digital architectures, however at the expense of limited scalability, susceptibility to noise, complex verification, and poor flexibility. On the other hand, state-of-the-art digital neuromorphic architectures focus either on achieving high energy efficiency (Joules/synaptic operation (SOP)) or throughput efficiency (SOPs/second/area), resulting in poor ET efficiency. In this work, we present THOR, an all-digital neuromorphic processor with a novel memory hierarchy and neuron update architecture that addresses both energy consumption and throughput bottlenecks. We implemented THOR in 28nm FDSOI CMOS technology and our post-layout results demonstrate an ET efficiency of 7.29G邮件日期
2022年12月06日- Loss shaping enhances exact gradient learning with EventProp in Spiking Neural Networks 时间：2022年12月02日 第一作者：Thomas Nowotny 链接.
摘要：在最近的一篇论文中，Wunderlich和Pehle介绍了EventProp算法，该算法能够通过精确梯度上的梯度下降来训练脉冲神经网络。在本文中，我们提出了EventProp的扩展，以支持更广泛的损失函数，并在利用稀疏性的GPU增强神经元网络框架中实现。GPU加速允许我们在更具挑战性的学习基准上广泛测试EventProp。我们发现EventProp在某些任务上表现良好，但在其他任务上存在学习缓慢或完全失败的问题。在这里，我们详细分析了这些问题，并发现它们与损失函数的精确梯度的使用有关，其本质上不提供由于脉冲创建或脉冲删除而导致的损失变化的信息。根据任务的细节和损失函数，使用EventProp降低精确的梯度会导致删除重要的峰值，从而导致损失的意外增加
英文摘要
In a recent paper Wunderlich and Pehle introduced the EventProp algorithm that enables training spiking neural networks by gradient descent on exact gradients. In this paper we present extensions of EventProp to support a wider class of loss functions and an implementation in the GPU enhanced neuronal networks framework which exploits sparsity. The GPU acceleration allows us to test EventProp extensively on more challenging learning benchmarks. We find that EventProp performs well on some tasks but for others there are issues where learning is slow or fails entirely. Here, we analyse these issues in detail and discover that they relate to the use of the exact gradient of the loss function, which by its nature does not provide information about loss changes due to spike creation or spike deletion. Depending on the details of the task and loss function, descending the exact gradient with EventProp can lead to the deletion of important spikes and so to an inadvertent increase of the loss and decrease of classification accuracy and hence a failure to learn. In other situations the lack of knowledge about the benefits of creating additional spikes can lead to a lack of gradient flow into earlier layers, slowing down learning. We eventually present a first glimpse of a solution to these problems in the form of `loss shaping', where we introduce a suitable weighting function into an integral loss to increase gradient flow from the output layer towards earlier layers.注释
18 pages, 7 figures MSC-class: 68T05 (Primary) 68T10, 68T07 (Secondary) ACM-class: I.2; I.2.6; I.5.1邮件日期
2022年12月05日- Surrogate Gradient Spiking Neural Networks as Encoders for Large Vocabulary Continuous Speech Recognition 时间：2022年12月01日 第一作者：Alex 链接.
摘要：与传统的人工神经元相比，受生物启发的脉冲神经元可以传递稀疏的二进制信息，这也可以实现节能。最近的研究表明，使用替代梯度方法可以像标准递归神经网络一样训练脉冲神经网络。他们在语音命令识别任务上显示出了有希望的结果。使用相同的技术，我们证明了它们可以扩展到大词汇连续语音识别，在这种情况下，它们能够替换编码器中的LSTM，而性能损失很小。这表明它们可能适用于更复杂的顺序任务。此外，与它们经常出现的非峰值对应物相比，它们在不需要使用门的情况下显示出对爆炸性梯度问题的鲁棒性。
英文摘要
Compared to conventional artificial neurons that produce dense and real-valued responses, biologically-inspired spiking neurons transmit sparse and binary information, which can also lead to energy-efficient implementations. Recent research has shown that spiking neural networks can be trained like standard recurrent neural networks using the surrogate gradient method. They have shown promising results on speech command recognition tasks. Using the same technique, we show that they are scalable to large vocabulary continuous speech recognition, where they are capable of replacing LSTMs in the encoder with only minor loss of performance. This suggests that they may be applicable to more involved sequence-to-sequence tasks. Moreover, in contrast to their recurrent non-spiking counterparts, they show robustness to exploding gradient problems without the need to use gates.注释
Submitted to ICASSP 2023邮件日期
2022年12月05日- Exploring Temporal Information Dynamics in Spiking Neural Networks 时间：2022年11月30日 第一作者：Youngeun Kim 链接.
注释
Accepted to AAAI2023邮件日期
2022年12月01日- Unsupervised Spike Depth Estimation via Cross-modality Cross-domain Knowledge Transfer 时间：2022年11月30日 第一作者：Jiaming Liu 链接.
邮件日期
2022年12月01日- Sequence learning in a spiking neuronal network with memristive synapses 时间：2022年11月29日 第一作者：Younes Bouhadjar 链接.
摘要：灵感来自大脑的计算提出了一套算法原理，有望推动人工智能的发展。它们赋予系统自学习能力、高效能源利用和高存储容量。大脑计算的核心概念是序列学习和预测。这种形式的计算对于我们几乎所有的日常任务都是必不可少的，比如运动生成、感知和语言。了解大脑是如何进行这种计算的，这不仅对推进神经科学很重要，而且也为新的技术脑启发应用铺平道路。先前开发的序列预测和回忆的脉冲神经网络实现通过局部的、受生物启发的可塑性规则以无监督的方式学习复杂的高阶序列。神经形态硬件是一种新兴的硬件类型，有望有效地运行这类算法。它模仿了大脑处理信息的方式
英文摘要
Brain-inspired computing proposes a set of algorithmic principles that hold promise for advancing artificial intelligence. They endow systems with self learning capabilities, efficient energy usage, and high storage capacity. A core concept that lies at the heart of brain computation is sequence learning and prediction. This form of computation is essential for almost all our daily tasks such as movement generation, perception, and language. Understanding how the brain performs such a computation is not only important to advance neuroscience but also to pave the way to new technological brain-inspired applications. A previously developed spiking neural network implementation of sequence prediction and recall learns complex, high-order sequences in an unsupervised manner by local, biologically inspired plasticity rules. An emerging type of hardware that holds promise for efficiently running this type of algorithm is neuromorphic hardware. It emulates the way the brain processes information and maps neurons and synapses directly into a physical substrate. Memristive devices have been identified as potential synaptic elements in neuromorphic hardware. In particular, redox-induced resistive random access memories (ReRAM) devices stand out at many aspects. They permit scalability, are energy efficient and fast, and can implement biological plasticity rules. In this work, we study the feasibility of using ReRAM devices as a replacement of the biological synapses in the sequence learning model. We implement and simulate the model including the ReRAM plasticity using the neural simulator NEST. We investigate the effect of different device properties on the performance characteristics of the sequence learning model, and demonstrate resilience with respect to different on-off ratios, conductance resolutions, device variability, and synaptic failure.注释
23 pages, 13 Figures邮件日期
2022年12月01日- Wearable-based Human Activity Recognition with Spatio-Temporal Spiking Neural Networks 时间：2022年11月14日 第一作者：Yuhang Li 链接.
摘要：我们研究了人类活动识别（HAR）任务，该任务基于可穿戴传感器的时间序列数据预测用户的日常活动。最近，研究人员使用端到端人工神经网络（ANN）在HAR中提取特征并执行分类。然而，ANN对可穿戴设备造成了巨大的计算负担，并且缺乏时间特征提取。在这项工作中，我们利用Spiking Neural Networks（SNNs）——一种受生物神经元启发的架构——来完成HAR任务。SNN允许特征的时空提取，并享受具有二进制峰值的低功耗计算。我们使用SNN在三个HAR数据集上进行了广泛的实验，证明SNN在精度方面与ANN不相上下，同时降低了高达94%的能耗。该代码在https://github.com/Intelligent-Computing-Lab-Yale/SNN_HAR
英文摘要
We study the Human Activity Recognition (HAR) task, which predicts user daily activity based on time series data from wearable sensors. Recently, researchers use end-to-end Artificial Neural Networks (ANNs) to extract the features and perform classification in HAR. However, ANNs pose a huge computation burden on wearable devices and lack temporal feature extraction. In this work, we leverage Spiking Neural Networks (SNNs)--an architecture inspired by biological neurons--to HAR tasks. SNNs allow spatio-temporal extraction of features and enjoy low-power computation with binary spikes. We conduct extensive experiments on three HAR datasets with SNNs, demonstrating that SNNs are on par with ANNs in terms of accuracy while reducing up to 94% energy consumption. The code is publicly available in https://github.com/Intelligent-Computing-Lab-Yale/SNN_HAR注释
Workshop on Learning from Time Series for Health邮件日期
2022年12月06日- Review of medical data analysis based on spiking neural networks 时间：2022年11月13日 第一作者：X. Li (1) 链接.
摘要：医学数据主要包括各种生物医学信号和医学图像，医生可以通过医学数据对患者的身体状况做出判断。然而，医疗数据的解释需要大量的人力成本，并且可能被误判，因此许多学者使用神经网络和深度学习对医疗数据进行分类和研究，从而提高医生的工作效率和准确性，实现疾病的早期发现和早期诊断，因此具有广泛的应用前景。然而，传统的神经网络具有高能耗和高延迟（计算速度慢）等缺点。本文介绍了近年来利用脑电图（EEG）、心电图（ECG）、肌电图（EMG）、磁共振成像（MRI）等医学数据，基于第三代神经网络脉冲神经网络的信号分类和疾病诊断研究，总结了其优缺点
英文摘要
Medical data mainly includes various biomedical signals and medical images, and doctors can make judgments on the physical condition of patients through medical data. However, the interpretation of medical data requires a lot of labor costs and may be misjudged, so many scholars use neural networks and deep learning to classify and study medical data, thereby improving doctors' work efficiency and accuracy, achieving early detection of diseases and early diagnosis, so it has a wide range of application prospects. However, traditional neural networks have disadvantages such as high energy consumption and high latency (slow calculation speed). This paper introduces the research on signal classification and disease diagnosis based on the third-generation neural network - pulse neural network in recent years, using medical data, such as electroencephalogram (EEG), electrocardiogram (ECG), electromyography (EMG), magnetic resonance imaging (MRI), etc., summarizes the advantages and disadvantages of pulse neural networks compared with traditional networks, and looks forward to the future development direction.注释
in Chinese language邮件日期
2022年12月06日- Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions 时间：2022年11月29日 第一作者：Kakei Yamamoto 链接.
摘要：我们提出了一种用于训练脉冲神经网络（SNN）的新的反向传播算法，该算法在没有单个脉冲限制的情况下，在单个神经元的相对多个脉冲定时中编码信息。所提出的算法继承了传统的基于定时的方法的优点，因为它计算关于脉冲定时的精确梯度，这促进了理想的时间编码。与每个神经元最多发射一次的传统方法不同，所提出的算法允许每个神经元发射多次。这种扩展自然提高了SNN的计算能力。我们的SNN模型优于可比SNN模型，并实现了与非卷积人工神经网络一样高的精度。我们网络的脉冲计数特性根据突触后电流的时间常数和膜电位而改变。此外，我们发现存在具有最大测试精度的最佳时间常数。这是传统的
英文摘要
We propose a novel backpropagation algorithm for training spiking neural networks (SNNs) that encodes information in the relative multiple spike timing of individual neurons without single-spike restrictions. The proposed algorithm inherits the advantages of conventional timing-based methods in that it computes accurate gradients with respect to spike timing, which promotes ideal temporal coding. Unlike conventional methods where each neuron fires at most once, the proposed algorithm allows each neuron to fire multiple times. This extension naturally improves the computational capacity of SNNs. Our SNN model outperformed comparable SNN models and achieved as high accuracy as non-convolutional artificial neural networks. The spike count property of our networks was altered depending on the time constant of the postsynaptic current and the membrane potential. Moreover, we found that there existed the optimal time constant with the maximum test accuracy. That was not seen in conventional SNNs with single-spike restrictions on time-to-fast-spike (TTFS) coding. This result demonstrates the computational properties of SNNs that biologically encode information into the multi-spike timing of individual neurons. Our code would be publicly available.注释
10 pages, 5 figures ACM-class: I.5.1邮件日期
2022年11月30日- Matrix representations of spiking neural P systems: Revisited 时间：2022年11月28日 第一作者：Henry N. Adorna 链接.
摘要：2010年，提出了无延迟SN P系统的矩阵表示，而在有延迟的SN P系统中，2017年提出了矩阵表示。这些表示使用计算机软件和硬件技术对SN P系统进行了一系列模拟。在这项工作中，我们重新审视了这些表示，并对SNP系统的计算行为提供了一些观察。在有延迟和无延迟的SNP系统中都考虑了配置可达性的概念。在具有延迟的SN P系统的情况下，提出了一种更好的下一配置计算方法。
英文摘要
In the 2010, matrix representation of SN P system without delay was presented while in the case of SN P systems with delay, matrix representation was suggested in the 2017. These representations brought about series of simulation of SN P systems using computer software and hardware technology. In this work, we revisit these representation and provide some observations on the behavior of the computations of SN P systems. The concept of reachability of configuration is considered in both SN P systems with and without delays. A better computation of next configuration is proposed in the case of SN P system with delay.注释
In: Gheorghe Paun (Ed) Proceedings of the 20th International Conference on Membrane Computing (CMC20), Editura Bibliostar, Ramnicu Valcea (2019) pp 227-247邮件日期
2022年11月29日- Exploring Temporal Information Dynamics in Spiking Neural Networks 时间：2022年11月26日 第一作者：Youngeun Kim 链接.
摘要：大多数现有的脉冲神经网络（SNN）工作表明，SNN可以利用脉冲的时间信息动态。然而，对时间信息动态的明确分析仍然缺失。在本文中，我们提出了几个重要问题，以提供对SNN的基本理解：SNN内部的时间信息动态是什么？我们如何测量时间信息动态？时间信息动态如何影响整体学习绩效？为了回答这些问题，我们估计权重的Fisher信息，以实证的方式测量训练期间时间信息的分布。令人惊讶的是，随着训练的进行，费舍尔的信息开始集中在早期阶段。在训练之后，我们观察到信息在前几个时间段变得高度集中，这一现象我们称之为时间信息集中。我们观察到，时间信息集中现象是一种常见的现象
英文摘要
Most existing Spiking Neural Network (SNN) works state that SNNs may utilize temporal information dynamics of spikes. However, an explicit analysis of temporal information dynamics is still missing. In this paper, we ask several important questions for providing a fundamental understanding of SNNs: What are temporal information dynamics inside SNNs? How can we measure the temporal information dynamics? How do the temporal information dynamics affect the overall learning performance? To answer these questions, we estimate the Fisher Information of the weights to measure the distribution of temporal information during training in an empirical manner. Surprisingly, as training goes on, Fisher information starts to concentrate in the early timesteps. After training, we observe that information becomes highly concentrated in earlier few timesteps, a phenomenon we refer to as temporal information concentration. We observe that the temporal information concentration phenomenon is a common learning feature of SNNs by conducting extensive experiments on various configurations such as architecture, dataset, optimization strategy, time constant, and timesteps. Furthermore, to reveal how temporal information concentration affects the performance of SNNs, we design a loss function to change the trend of temporal information. We find that temporal information concentration is crucial to building a robust SNN but has little effect on classification accuracy. Finally, we propose an efficient iterative pruning method based on our observation on temporal information concentration. Code is available at https://github.com/Intelligent-Computing-Lab-Yale/Exploring-Temporal-Information-Dynamics-in-Spiking-Neural-Networks.注释
Accepted to AAAI2023邮件日期
2022年11月29日- Game Theoretic Mixed Experts for Combinational Adversarial Machine Learning 时间：2022年11月26日 第一作者：Ethan Rathbun 链接.
摘要：对抗性机器学习的最新进展表明，被认为是强大的防御实际上容易受到针对其弱点而专门定制的对抗性攻击。这些防御措施包括随机变换障碍（BaRT）、友好对抗训练（FAT）、垃圾就是宝藏（TiT）以及由视觉变换器（ViT）、大转移模型和Spiking神经网络（SNN）组成的集成模型。一个自然的问题出现了：如何最好地利用对抗性防御的组合来阻止此类攻击？在本文中，我们提供了一个综合对抗性攻击和防御的博弈论框架，以回答这个问题。除了我们的框架之外，我们还进行了第一次对抗性防御可转移性研究，以进一步激发对利用多种防御体系结构的组合防御的需求。我们的框架被称为博弈论混合专家（Game），旨在找到防御的混合纳什策略
英文摘要
Recent advances in adversarial machine learning have shown that defenses considered to be robust are actually susceptible to adversarial attacks which are specifically tailored to target their weaknesses. These defenses include Barrage of Random Transforms (BaRT), Friendly Adversarial Training (FAT), Trash is Treasure (TiT) and ensemble models made up of Vision Transformers (ViTs), Big Transfer models and Spiking Neural Networks (SNNs). A natural question arises: how can one best leverage a combination of adversarial defenses to thwart such attacks? In this paper, we provide a game-theoretic framework for ensemble adversarial attacks and defenses which answers this question. In addition to our framework we produce the first adversarial defense transferability study to further motivate a need for combinational defenses utilizing a diverse set of defense architectures. Our framework is called Game theoretic Mixed Experts (GaME) and is designed to find the Mixed-Nash strategy for a defender when facing an attacker employing compositional adversarial attacks. We show that this framework creates an ensemble of defenses with greater robustness than multiple state-of-the-art, single-model defenses in addition to combinational defenses with uniform probability distributions. Overall, our framework and analyses advance the field of adversarial machine learning by yielding new insights into compositional attack and defense formulations.注释
21 pages, 6 figures ACM-class: I.2; I.4邮件日期
2022年11月29日- Scaling Up Dynamic Graph Representation Learning via Spiking Neural Networks 时间：2022年11月26日 第一作者：Jintang Li 链接.
注释
Accepted by AAAI 2023.Contains appendix with additional details about algorithms and experiments. Code available at https://github.com/EdisonLeeeee/SpikeNet邮件日期
2022年11月29日- PC-SNN: Supervised Learning with Local Hebbian Synaptic Plasticity based on Predictive Coding in Spiking Neural Networks 时间：2022年11月24日 第一作者：Mengting Lan 链接.
摘要：被认为是第三代神经网络，事件驱动的Spiking神经网络（SNN）与生物似本地学习规则相结合，使其有望为SNN构建低功耗的神经形态硬件。然而，由于脉冲神经网络的非线性和离散性，SNN的训练仍然很困难，仍在讨论中。源于梯度下降，backop在多层SNN中取得了惊人的成功。然而，它被认为缺乏生物合理性，同时消耗了相对较高的计算资源。在本文中，我们提出了一种受预测编码理论启发的新的学习算法，并表明它可以完全自主地和成功地执行监督学习，作为反向操作，仅利用局部Hebbian可塑性。此外，与最先进的多层SNN相比，该方法实现了良好的性能：加州理工学院人脸/摩托车数据集的测试精度为99.25%，84.25%
英文摘要
Deemed as the third generation of neural networks, the event-driven Spiking Neural Networks(SNNs) combined with bio-plausible local learning rules make it promising to build low-power, neuromorphic hardware for SNNs. However, because of the non-linearity and discrete property of spiking neural networks, the training of SNN remains difficult and is still under discussion. Originating from gradient descent, backprop has achieved stunning success in multi-layer SNNs. Nevertheless, it is assumed to lack biological plausibility, while consuming relatively high computational resources. In this paper, we propose a novel learning algorithm inspired by predictive coding theory and show that it can perform supervised learning fully autonomously and successfully as the backprop, utilizing only local Hebbian plasticity. Furthermore, this method achieves a favorable performance compared to the state-of-the-art multi-layer SNNs: test accuracy of 99.25% for the Caltech Face/Motorbike dataset, 84.25% for the ETH-80 dataset, 98.1% for the MNIST dataset and 98.5% for the neuromorphic dataset: N-MNIST. Furthermore, our work provides a new perspective on how supervised learning algorithms are directly implemented in spiking neural circuitry, which may give some new insights into neuromorphological calculation in neuroscience.注释
15 pages, 11figs ACM-class: I.2.3; I.2.10邮件日期
2022年11月29日- Developmental Plasticity-inspired Adaptive Pruning for Deep Spiking and Artificial Neural Networks 时间：2022年11月23日 第一作者：Bing Han 链接.
摘要：发育可塑性在不断学习过程中对大脑结构的塑造起着至关重要的作用，以应对动态变化的环境。然而，现有的深度人工神经网络（ANN）和脉冲神经网络（SNN）的网络压缩方法很少从大脑的发育可塑性机制中获得灵感，因此限制了它们高效、快速和准确地学习的能力。本文提出了一种基于发育可塑性的适应性修剪（DPAP）方法，其灵感来源于树突棘、突触和神经元的适应性发育修剪，遵循“要么使用，要么失去，逐渐衰退”原则。所提出的DPAP模型考虑了多种生物学现实机制（如树突脊柱动态可塑性、活动依赖性神经脉冲轨迹、局部突触可塑性），并添加了自适应修剪策略，从而可以在学习过程中动态优化网络结构
英文摘要
Developmental plasticity plays a vital role in shaping the brain's structure during ongoing learning in response to the dynamically changing environments. However, the existing network compression methods for deep artificial neural networks (ANNs) and spiking neural networks (SNNs) draw little inspiration from the brain's developmental plasticity mechanisms, thus limiting their ability to learn efficiently, rapidly, and accurately. This paper proposed a developmental plasticity-inspired adaptive pruning (DPAP) method, with inspiration from the adaptive developmental pruning of dendritic spines, synapses, and neurons according to the "use it or lose it, gradually decay" principle. The proposed DPAP model considers multiple biologically realistic mechanisms (such as dendritic spine dynamic plasticity, activity-dependent neural spiking trace, local synaptic plasticity), with the addition of an adaptive pruning strategy, so that the network structure can be dynamically optimized during learning without any pre-training and retraining. We demonstrated that the proposed DPAP method applied to deep ANNs and SNNs could learn efficient network architectures that retain only relevant important connections and neurons. Extensive comparative experiments show consistent and remarkable performance and speed boost with the extremely compressed networks on a diverse set of benchmark tasks, especially neuromorphic datasets for SNNs. This work explores how developmental plasticity enables the complex deep networks to gradually evolve into brain-like efficient and compact structures, eventually achieving state-of-the-art (SOTA) performance for biologically realistic SNNs.邮件日期
2022年11月24日- MSS-DepthNet: Depth Prediction with Multi-Step Spiking Neural Network 时间：2022年11月22日 第一作者：Xiaoshan Wu 链接.
摘要：事件摄像机由于其高时间分辨率和低功耗特性，被认为在计算机视觉和机器人应用中具有巨大潜力。然而，事件摄像机输出的事件流具有现有计算机视觉算法无法处理的异步、稀疏特性。Spiking神经网络是一种新的基于事件的计算范式，被认为非常适合处理事件摄像机任务。然而，深度SNN的直接训练存在退化问题。这项工作通过提出一种脉冲神经网络架构来解决这些问题，该架构设计了新的残差块，并结合了多维注意力模块，重点关注深度预测问题。此外，还提出了一种新的SNN事件流表示方法。该模型在MVSEC数据集上优于相同大小的先前ANN网络，并显示出极大的计算效率。
英文摘要
Event cameras are considered to have great potential for computer vision and robotics applications because of their high temporal resolution and low power consumption characteristics. However, the event stream output from event cameras has asynchronous, sparse characteristics that existing computer vision algorithms cannot handle. Spiking neural network is a novel event-based computational paradigm that is considered to be well suited for processing event camera tasks. However, direct training of deep SNNs suffers from degradation problems. This work addresses these problems by proposing a spiking neural network architecture with a novel residual block designed and multi-dimension attention modules combined, focusing on the problem of depth prediction. In addition, a novel event stream representation method is explicitly proposed for SNNs. This model outperforms previous ANN networks of the same size on the MVSEC dataset and shows great computational efficiency.邮件日期
2022年11月23日- Adaptive Sparse Structure Development with Pruning and Regeneration for Spiking Neural Networks 时间：2022年11月22日 第一作者：Bing Han 链接.
摘要：Spiking神经网络（SNN）在生物学上更合理，计算效率更高。因此，SNN具有利用大脑发育的稀疏结构可塑性来缓解深度神经网络因其复杂和固定结构而导致的能量问题的天然优势。然而，以往的SNN压缩工作缺乏来自大脑发育可塑性机制的深入启发。本文提出了一种新的SNN（SD-SNN）自适应结构发展方法，引入了基于树突脊柱可塑性的突触约束、神经元修剪和突触再生。我们发现，突触约束和神经元修剪可以检测和消除SNN中的大量冗余，结合突触再生可以有效地防止和修复过度修剪。此外，受神经营养假说的启发，神经元修剪率和突触再生率在边学习边修剪过程中被自适应地调整，
英文摘要
Spiking Neural Networks (SNNs) are more biologically plausible and computationally efficient. Therefore, SNNs have the natural advantage of drawing the sparse structural plasticity of brain development to alleviate the energy problems of deep neural networks caused by their complex and fixed structures. However, previous SNNs compression works are lack of in-depth inspiration from the brain development plasticity mechanism. This paper proposed a novel method for the adaptive structural development of SNN (SD-SNN), introducing dendritic spine plasticity-based synaptic constraint, neuronal pruning and synaptic regeneration. We found that synaptic constraint and neuronal pruning can detect and remove a large amount of redundancy in SNNs, coupled with synaptic regeneration can effectively prevent and repair over-pruning. Moreover, inspired by the neurotrophic hypothesis, neuronal pruning rate and synaptic regeneration rate were adaptively adjusted during the learning-while-pruning process, which eventually led to the structural stability of SNNs. Experimental results on spatial (MNIST, CIFAR-10) and temporal neuromorphic (N-MNIST, DVS-Gesture) datasets demonstrate that our method can flexibly learn appropriate compression rate for various tasks and effectively achieve superior performance while massively reducing the network energy consumption. Specifically, for the spatial MNIST dataset, our SD-SNN achieves 99.51\% accuracy at the pruning rate 49.83\%, which has a 0.05\% accuracy improvement compared to the baseline without compression. For the neuromorphic DVS-Gesture dataset, 98.20\% accuracy with 1.09\% improvement is achieved by our method when the compression rate reaches 55.50\%.邮件日期
2022年11月23日- Spikformer: When Spiking Neural Network Meets Transformer 时间：2022年11月22日 第一作者：Zhaokun Zhou 链接.
邮件日期
2022年11月23日- A Low Latency Adaptive Coding Spiking Framework for Deep Reinforcement Learning 时间：2022年11月21日 第一作者：Lang Qin 链接.
摘要：在过去几年中，借助于深度神经网络，深度强化学习（DRL）在许多复杂任务上取得了巨大成功。Spiking神经网络（SNN）已被用于在专用神经形态硬件上实现具有超高能量效率的深度神经网络，近年来，SNN与强化学习的结合受到了越来越多的关注，而大多数方法仍然在巨大的能量消耗和高延迟下工作。这项工作提出了基于SNN的DRL的自适应编码Spiking框架（ACSF），同时实现了低延迟和高能效。受生物学中经典条件反射的启发，我们分别用棘波编码器、SNN和棘波解码器模拟受体、**中间神经元和效应器。我们使用我们提出的ACSF来估计强化学习中的价值函数，并进行广泛的实验来验证我们提出的框架的有效性。
英文摘要
With the help of Deep Neural Networks, Deep Reinforcement Learning (DRL) has achieved great success on many complex tasks during the past few years. Spiking Neural Networks (SNNs) have been used for the implementation of Deep Neural Networks with superb energy efficiency on dedicated neuromorphic hardware, and recent years have witnessed increasing attention on combining SNNs with Reinforcement Learning, whereas most approaches still work with huge energy consumption and high latency. This work proposes the Adaptive Coding Spiking Framework (ACSF) for SNN-based DRL and achieves low latency and great energy efficiency at the same time. Inspired by classical conditioning in biology, we simulate receptors, central interneurons, and effectors with spike encoders, SNNs, and spike decoders, respectively. We use our proposed ACSF to estimate the value function in reinforcement learning and conduct extensive experiments to verify the effectiveness of our proposed framework.邮件日期
2022年11月23日- HALSIE -- Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalities 时间：2022年11月19日 第一作者：Shristi Das Biswas 链接.
摘要：在自主导航等具有挑战性的实时应用中，标准的基于帧的算法无法检索精确的分割地图，这是由于传统相机中普遍存在的有限动态范围和运动模糊。事件摄像机通过异步检测每像素强度的变化来解决这些限制，以生成具有高时间分辨率、高动态范围和无运动模糊的事件流。然而，事件摄像机输出不能直接用于生成可靠的分割图，因为它们仅在运动像素处捕获信息。为了补充缺失的上下文信息，我们假设将空间密集的帧与时间密集的事件融合可以生成具有细粒度预测的语义图。为此，我们提出了HALSIE，这是一种通过同时利用图像和事件模态来学习分割的混合方法。为了实现跨模态的高效学习，我们提出的混合框架包括两个输入分支，即Sp
英文摘要
Standard frame-based algorithms fail to retrieve accurate segmentation maps in challenging real-time applications like autonomous navigation, owing to the limited dynamic range and motion blur prevalent in traditional cameras. Event cameras address these limitations by asynchronously detecting changes in per-pixel intensity to generate event streams with high temporal resolution, high dynamic range, and no motion blur. However, event camera outputs cannot be directly used to generate reliable segmentation maps as they only capture information at the pixels in motion. To augment the missing contextual information, we postulate that fusing spatially dense frames with temporally dense events can generate semantic maps with fine-grained predictions. To this end, we propose HALSIE, a hybrid approach to learning segmentation by simultaneously leveraging image and event modalities. To enable efficient learning across modalities, our proposed hybrid framework comprises two input branches, a Spiking Neural Network (SNN) branch and a standard Artificial Neural Network (ANN) branch to process event and frame data respectively, while exploiting their corresponding neural dynamics. Our hybrid network outperforms the state-of-the-art semantic segmentation benchmarks on DDD17 and MVSEC datasets and shows comparable performance on the DSEC-Semantic dataset with upto 33.23$\times$ reduction in network parameters. Further, our method shows upto 18.92$\times$ improvement in inference cost compared to existing SOTA approaches, making it suitable for resource-constrained edge applications.邮件日期
2022年11月22日- Spikeformer: A Novel Architecture for Training High-Performance Low-Latency Spiking Neural Network 时间：2022年11月19日 第一作者：Yudong Li 链接.
摘要：在过去几年中，Spiking神经网络（SNN）在性能和效率方面都取得了巨大进步，但其独特的工作模式使得训练高性能低延迟SNN变得困难。因此，SNN的发展仍然落后于传统的人工神经网络（ANN）。为了弥补这一差距，人们提出了许多非凡的工作。然而，这些工作主要基于相同类型的网络结构（即CNN），并且它们的性能比它们的ANN同行差，这限制了SNN的应用。为此，我们提出了一种新的基于Transformer的SNN，称为“Spikeformer”，它在静态数据集和神经形态数据集上都优于其ANN对应物，可能是CNN训练高性能SNN的替代架构。首先，为了解决香草模型中“数据饥饿”和训练周期不稳定的问题，我们设计了卷积令牌化器（CT）模块，这提高了原始模型的准确性
英文摘要
Spiking neural networks (SNNs) have made great progress on both performance and efficiency over the last few years,but their unique working pattern makes it hard to train a high-performance low-latency SNN.Thus the development of SNNs still lags behind traditional artificial neural networks (ANNs).To compensate this gap,many extraordinary works have been proposed.Nevertheless,these works are mainly based on the same kind of network structure (i.e.CNN) and their performance is worse than their ANN counterparts,which limits the applications of SNNs.To this end,we propose a novel Transformer-based SNN,termed "Spikeformer",which outperforms its ANN counterpart on both static dataset and neuromorphic dataset and may be an alternative architecture to CNN for training high-performance SNNs.First,to deal with the problem of "data hungry" and the unstable training period exhibited in the vanilla model,we design the Convolutional Tokenizer (CT) module,which improves the accuracy of the original model on DVS-Gesture by more than 16%.Besides,in order to better incorporate the attention mechanism inside Transformer and the spatio-temporal information inherent to SNN,we adopt spatio-temporal attention (STA) instead of spatial-wise or temporal-wise attention.With our proposed method,we achieve competitive or state-of-the-art (SOTA) SNN performance on DVS-CIFAR10,DVS-Gesture,and ImageNet datasets with the least simulation time steps (i.e.low latency).Remarkably,our Spikeformer outperforms other SNNs on ImageNet by a large margin (i.e.more than 5%) and even outperforms its ANN counterpart by 3.1% and 2.2% on DVS-Gesture and ImageNet respectively,indicating that Spikeformer is a promising architecture for training large-scale SNNs and may be more suitable for SNNs compared to CNN.We believe that this work shall keep the development of SNNs in step with ANNs as much as possible.Code will be available.邮件日期
2022年11月22日- Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants 时间：2022年11月19日 第一作者：L. Mars Gao 链接.
摘要：在$\ell_1$约束下基于自动编码器的非线性动力学稀疏识别（SINDy）的最新进展允许从时空数据（包括模拟视频帧）中联合发现控制方程和潜在坐标系统。然而，基于$\ell_1$的稀疏推理很难对真实数据进行正确的识别，因为测量值有噪声，样本大小通常有限。为了解决低数据和高噪声状态下的数据驱动的物理发现，我们提出了贝叶斯SINDy自动编码器，它结合了分层贝叶斯稀疏先验：脉冲和板条高斯拉索。贝叶斯SINDy自动编码器能够在理论上保证不确定性估计的情况下联合发现控制方程和坐标系统。为了解决贝叶斯分层设置的具有挑战性的计算可处理性，我们采用了具有Stochatic梯度Langevin动力学（SGLD）的自适应经验贝叶斯方法
英文摘要
Recent progress in autoencoder-based sparse identification of nonlinear dynamics (SINDy) under注释
28 pages, 11 figures邮件日期
2022年11月22日- SMS: Spiking Marching Scheme for Efficient Long Time Integration of Differential Equations 时间：2022年11月17日 第一作者：Qian Zhang 链接.
摘要：我们提出了一种基于Spiking神经网络（SNN）的显式数值格式，用于时间相关的常微分方程和偏微分方程（ODEs，PDE）的长时间积分。该方法的核心元素是SNN，该SNN被训练为在前一时间步使用关于解决方案的脉冲编码信息来预测下一时间步的脉冲编码的信息。在训练网络之后，它作为一个显式的数值方案运行，在给定脉冲编码初始条件的情况下，可以用于在未来的时间步长计算解。解码器用于将演进的脉冲编码解转换回函数值。我们给出了对不同复杂度的ODE和PDE使用所提出方法的数值实验结果。
英文摘要
We propose a Spiking Neural Network (SNN)-based explicit numerical scheme for long time integration of time-dependent Ordinary and Partial Differential Equations (ODEs, PDEs). The core element of the method is a SNN, trained to use spike-encoded information about the solution at previous timesteps to predict spike-encoded information at the next timestep. After the network has been trained, it operates as an explicit numerical scheme that can be used to compute the solution at future timesteps, given a spike-encoded initial condition. A decoder is used to transform the evolved spiking-encoded solution back to function values. We present results from numerical experiments of using the proposed method for ODEs and PDEs of varying complexity.注释
14 pages, 7 figures Report-no: PNNL-SA-179601 MSC-class: 65M99邮件日期
2022年11月21日- Impact of spiking neurons leakages and network recurrences on event-based spatio-temporal pattern recognition 时间：2022年11月14日 第一作者：Mohamed Sadek Bouanane 链接.
摘要：Spiking神经网络与神经形态硬件和基于事件的传感器相结合，对边缘的低延迟和低功耗推理越来越感兴趣。然而，文献中提出了具有不同生物合理性水平、不同计算特征和复杂性的多个脉冲神经元模型。因此，需要定义正确的生物学抽象级别，以便在神经形态硬件中获得准确、高效和快速推理的最佳性能。在此背景下，我们探讨了突触和膜泄漏对刺突神经元的影响。我们使用基于事件的视觉和听觉模式识别的前馈和递归拓扑来对抗具有不同计算复杂性的三个神经模型。我们的结果表明，就准确性而言，当数据中同时存在时间信息和网络中的显式重复时，泄漏是重要的。此外，无需泄漏
英文摘要
Spiking neural networks coupled with neuromorphic hardware and event-based sensors are getting increased interest for low-latency and low-power inference at the edge. However, multiple spiking neuron models have been proposed in the literature with different levels of biological plausibility and different computational features and complexities. Consequently, there is a need to define the right level of abstraction from biology in order to get the best performance in accurate, efficient and fast inference in neuromorphic hardware. In this context, we explore the impact of synaptic and membrane leakages in spiking neurons. We confront three neural models with different computational complexities using feedforward and recurrent topologies for event-based visual and auditory pattern recognition. Our results show that, in terms of accuracy, leakages are important when there are both temporal information in the data and explicit recurrence in the network. In addition, leakages do not necessarily increase the sparsity of spikes flowing in the network. We also investigate the impact of heterogeneity in the time constant of leakages, and the results show a slight improvement in accuracy when using data with a rich temporal structure. These results advance our understanding of the computational role of the neural leakages and network recurrences, and provide valuable insights for the design of compact and energy-efficient neuromorphic hardware for embedded systems.邮件日期
2022年11月16日- Motif-topology improved Spiking Neural Network for the Cocktail Party Effect and McGurk Effect 时间：2022年11月12日 第一作者：Shuncheng Jia 链接.
摘要：在人工神经网络（ANN）和脉冲神经网络（SNN）中，网络结构和学习原理是形成复杂函数的关键。SNN被认为是新一代人工网络，它结合了比ANN更多的生物学特征，包括动态脉冲神经元、功能特定的结构和高效的学习范式。网络架构也被认为体现了网络的功能。在这里，我们提出了一种Motif拓扑改进的SNN（M-SNN），用于有效的多感官整合和认知现象模拟。我们模拟的认知现象模拟包括鸡尾酒会效应和麦格克效应，这是许多研究人员讨论的。我们的M-SNN由称为网络基元的元运算符构成。从空间或时间数据集中预先学习的来自人工网络的3节点网络图案拓扑的来源。在单感官分类任务中，结果显示了准确性
英文摘要
Network architectures and learning principles are playing key in forming complex functions in artificial neural networks (ANNs) and spiking neural networks (SNNs). SNNs are considered the new-generation artificial networks by incorporating more biological features than ANNs, including dynamic spiking neurons, functionally specified architectures, and efficient learning paradigms. Network architectures are also considered embodying the function of the network. Here, we propose a Motif-topology improved SNN (M-SNN) for the efficient multi-sensory integration and cognitive phenomenon simulations. The cognitive phenomenon simulation we simulated includes the cocktail party effect and McGurk effect, which are discussed by many researchers. Our M-SNN constituted by the meta operator called network motifs. The source of 3-node network motifs topology from artificial one pre-learned from the spatial or temporal dataset. In the single-sensory classification task, the results showed the accuracy of M-SNN using network motif topologies was higher than the pure feedforward network topology without using them. In the multi-sensory integration task, the performance of M-SNN using artificial network motif was better than the state-of-the-art SNN using BRP (biologically-plausible reward propagation). Furthermore, the M-SNN could better simulate the cocktail party effect and McGurk effect with lower computational cost. We think the artificial network motifs could be considered as some prior knowledge that would contribute to the multi-sensory integration of SNNs and provide more benefits for simulating the cognitive phenomenon.邮件日期
2022年11月16日- Local learning through propagation delays in spiking neural networks 时间：2022年10月27日 第一作者：J{\o}rgen Jensen Farner 链接.
摘要：我们为脉冲神经网络提出了一种新的局部学习规则，其中脉冲传播时间经历活动依赖性可塑性。我们的可塑性规则调整了突触前脉冲时间，以产生更强、更快的反应。输入通过延迟编码进行编码，输出通过匹配输出脉冲活动的类似模式进行解码。我们演示了该方法在三层前馈网络中的使用，该网络具有来自手写数字数据库的输入。网络在训练后不断提高其分类精度，使用这种方法的训练也允许网络推广到训练期间不可见的输入类。我们提出的方法利用了刺突神经元支持许多不同时间锁定的刺突序列的能力，每个刺突序列都可以由不同的输入激活激活。这里展示的概念证明证明了局部延迟学习在扩展spi的记忆容量和可推广性方面的巨大潜力
英文摘要
We propose a novel local learning rule for spiking neural networks in which spike propagation times undergo activity-dependent plasticity. Our plasticity rule aligns pre-synaptic spike times to produce a stronger and more rapid response. Inputs are encoded by latency coding and outputs decoded by matching similar patterns of output spiking activity. We demonstrate the use of this method in a three-layer feedfoward network with inputs from a database of handwritten digits. Networks consistently improve their classification accuracy after training, and training with this method also allowed networks to generalize to an input class unseen during training. Our proposed method takes advantage of the ability of spiking neurons to support many different time-locked sequences of spikes, each of which can be activated by different input activations. The proof-of-concept shown here demonstrates the great potential for local delay learning to expand the memory capacity and generalizability of spiking neural networks.注释
4 pages, 4 figures; longer version under preparation邮件日期
2022年11月16日- Low Latency Conversion of Artificial Neural Network Models to Rate-encoded Spiking Neural Networks 时间：2022年10月27日 第一作者：Zhanglu Yan 链接.
摘要：Spiking神经网络（SNN）非常适合资源受限的应用，因为它们不需要昂贵的乘数。在典型的速率编码SNN中，在全局固定时间窗内的一系列二进制脉冲用于激发神经元。该时间窗口中的最大峰值数也是网络执行单个推断的延迟，并决定了模型的总体能效。本文的目的是在将ANN转换为其等效SNN时，在保持精度的同时减少这一点。最先进的转换方案产生了SNN，其精度仅与大窗口尺寸的ANN相当。在本文中，我们从理解从预先存在的ANN模型转换为标准速率编码SNN模型时的信息损失开始。根据这些见解，我们提出了一套新颖的技术，这些技术共同减轻了转换过程中的信息损失，并实现了最先进的SNN精度以及非常低的延迟
英文摘要
Spiking neural networks (SNNs) are well suited for resource-constrained applications as they do not need expensive multipliers. In a typical rate-encoded SNN, a series of binary spikes within a globally fixed time window is used to fire the neurons. The maximum number of spikes in this time window is also the latency of the network in performing a single inference, as well as determines the overall energy efficiency of the model. The aim of this paper is to reduce this while maintaining accuracy when converting ANNs to their equivalent SNNs. The state-of-the-art conversion schemes yield SNNs with accuracies comparable with ANNs only for large window sizes. In this paper, we start with understanding the information loss when converting from pre-existing ANN models to standard rate-encoded SNN models. From these insights, we propose a suite of novel techniques that together mitigate the information lost in the conversion, and achieve state-of-art SNN accuracies along with very low latency. Our method achieved a Top-1 SNN accuracy of 98.73% (1 time step) on the MNIST dataset, 76.38% (8 time steps) on the CIFAR-100 dataset, and 93.71% (8 time steps) on the CIFAR-10 dataset. On ImageNet, an SNN accuracy of 75.35%/79.16% was achieved with 100/200 time steps.邮件日期
2022年11月16日- Forecasting Bitcoin volatility spikes from whale transactions and CryptoQuant data using Synthesizer Transformer models 时间：2022年10月06日 第一作者：Dorien Herremans 链接.
摘要：与传统金融市场相比，加密货币市场波动性很大。因此，预测其波动性对于风险管理至关重要。在本文中，我们调查了CryptoQuant数据（例如链上分析、交易所和矿工数据）和鲸鱼警报推文，并探讨了它们与比特币次日波动率的关系，重点关注极端波动率峰值。我们提出了一种用于预测波动性的深度学习合成器-变压器模型。我们的结果表明，在使用CryptoQuant数据和鲸鱼警报推文预测比特币的极端波动峰值时，该模型优于现有的最先进模型。我们使用Captum XAI库分析了我们的模型，以研究哪些特性最重要。我们还用不同的基准交易策略对我们的预测结果进行了后验，结果表明，我们能够在保持稳定利润的同时最大限度地减少亏损。我们的研究结果强调，所提出的方法是一个有用的工具
英文摘要
The cryptocurrency market is highly volatile compared to traditional financial markets. Hence, forecasting its volatility is crucial for risk management. In this paper, we investigate CryptoQuant data (e.g. on-chain analytics, exchange and miner data) and whale-alert tweets, and explore their relationship to Bitcoin's next-day volatility, with a focus on extreme volatility spikes. We propose a deep learning Synthesizer Transformer model for forecasting volatility. Our results show that the model outperforms existing state-of-the-art models when forecasting extreme volatility spikes for Bitcoin using CryptoQuant data as well as whale-alert tweets. We analysed our model with the Captum XAI library to investigate which features are most important. We also backtested our prediction results with different baseline trading strategies and the results show that we are able to minimize drawdown while keeping steady profits. Our findings underscore that the proposed method is a useful tool for forecasting extreme volatility movements in the Bitcoin market.注释
Co-first authors邮件日期
2022年11月16日- Brain inspired neuronal silencing mechanism to enable reliable sequence identification 时间：2022年03月24日 第一作者：Shiri Hodassman 链接.
摘要：实时序列识别是人工神经网络（ANN）的核心用例，从识别时间事件到识别验证码。现有的方法应用了递归神经网络，其存在训练困难；然而，在没有反馈回路的情况下执行该功能仍然是一个挑战。在这里，我们提出了一种无反馈回路的高精度前馈序列识别网络（ID网络）的实验神经元长期可塑性机制，其中输入对象具有给定的顺序和时间。这种机制使神经元在最近的脉冲活动后暂时沉默。因此，暂态对象作用于不同的动态创建的前馈子网络。ID网络被证明能够可靠地识别10个手写数字序列，并被推广到具有在图像序列上训练的连续激活节点的深度卷积ANN。与直觉相反的是，它们的分类性能，即使是有限的努
英文摘要
Real-time sequence identification is a core use-case of artificial neural networks (ANNs), ranging from recognizing temporal events to identifying verification codes. Existing methods apply recurrent neural networks, which suffer from training difficulties; however, performing this function without feedback loops remains a challenge. Here, we present an experimental neuronal long-term plasticity mechanism for high-precision feedforward sequence identification networks (ID-nets) without feedback loops, wherein input objects have a given order and timing. This mechanism temporarily silences neurons following their recent spiking activity. Therefore, transitory objects act on different dynamically created feedforward sub-networks. ID-nets are demonstrated to reliably identify 10 handwritten digit sequences, and are generalized to deep convolutional ANNs with continuous activation nodes trained on image sequences. Counterintuitively, their classification performance, even with a limited number of training examples, is high for sequences but low for individual objects. ID-nets are also implemented for writer-dependent recognition, and suggested as a cryptographic tool for encrypted authentication. The presented mechanism opens new horizons for advanced ANN algorithms.注释
38 pages, 11 figures Journal-ref: Sci Rep 12, 16003 (2022) DOI: 10.1038/s41598-022-20337-x邮件日期
2022年11月16日- GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks 时间：2022年11月13日 第一作者：Xingting Yao 链接.
注释
Accepted at NeurIPS 2022 Spotlight邮件日期
2022年11月15日- NeuroHSMD: Neuromorphic Hybrid Spiking Motion Detector 时间：2022年11月12日 第一作者：Pedro Machado 链接.
邮件日期
2022年11月15日- Dance of SNN and ANN: Solving binding problem by combining spike timing and reconstructive attention 时间：2022年11月11日 第一作者：Hao Zheng 链接.
摘要：绑定问题是阻碍人工神经网络（ANN）对人类感知世界的合成理解的根本挑战之一，因为当呈现具有多个对象的复杂数据时，生成因子的非纠缠和分布式表示可能会干扰并导致模糊。在本文中，我们提出了一种大脑启发的混合神经网络（HNN），该网络通过将脉冲计时动力学（通过脉冲神经网络，SNN）与重建注意力（通过ANN）相结合，将源自神经科学的时间绑定理论引入到ANN中。脉冲定时为分组提供了额外的维度，而重建反馈将脉冲协调为时间相干状态。通过ANN和SNN的迭代交互，该模型在SNN编码空间中以交替的同步触发时间连续绑定多个对象。在二值图像的合成数据集上评估了模型的有效性。通过visu
英文摘要
The binding problem is one of the fundamental challenges that prevent the artificial neural network (ANNs) from a compositional understanding of the world like human perception, because disentangled and distributed representations of generative factors can interfere and lead to ambiguity when complex data with multiple objects are presented. In this paper, we propose a brain-inspired hybrid neural network (HNN) that introduces temporal binding theory originated from neuroscience into ANNs by integrating spike timing dynamics (via spiking neural networks, SNNs) with reconstructive attention (by ANNs). Spike timing provides an additional dimension for grouping, while reconstructive feedback coordinates the spikes into temporal coherent states. Through iterative interaction of ANN and SNN, the model continuously binds multiple objects at alternative synchronous firing times in the SNN coding space. The effectiveness of the model is evaluated on synthetic datasets of binary images. By visualization and analysis, we demonstrate that the binding is explainable, soft, flexible, and hierarchical. Notably, the model is trained on single object datasets without explicit supervision on grouping, but successfully binds multiple objects on test datasets, showing its compositional generalization capability. Further results show its binding ability in dynamic situations.邮件日期
2022年11月14日- Accurate online training of dynamical spiking neural networks through Forward Propagation Through Time 时间：2022年11月11日 第一作者：Bojian Yin 链接.
注释
12 pages, 4 figures邮件日期
2022年11月14日- Spiking Neural Network Decision Feedback Equalization 时间：2022年11月11日 第一作者：Eike-Manuel Bansbach 链接.
注释
Submitted to SCC 2023邮件日期
2022年11月14日- Spiking Neural Network Decision Feedback Equalization 时间：2022年11月09日 第一作者：Eike-Manuel Bansbach 链接.
摘要：在过去的几年中，人工神经网络（ANN）已经成为解决通信工程中难以用传统方法解决的任务的事实标准。与此同时，人工智能社区将其研究转向了生物学启发的、类似大脑的脉冲神经网络（SNNs），这一网络有望实现极其节能的计算。在本文中，我们研究了SNN在超低复杂度接收机信道均衡中的应用。我们提出了一种基于SNN的均衡器，其反馈结构类似于决策反馈均衡器（DFE）。为了将真实世界数据转换为脉冲信号，我们引入了一种新的三值编码，并将其与传统的对数尺度编码进行了比较。我们表明，对于三个不同的示例性信道，我们的方法明显优于传统的线性均衡器。我们强调，主要是将信道输出转换为脉冲会带来较小的性能损失。建议的SNN具有dec
英文摘要
In the past years, artificial neural networks (ANNs) have become the de-facto standard to solve tasks in communications engineering that are difficult to solve with traditional methods. In parallel, the artificial intelligence community drives its research to biology-inspired, brain-like spiking neural networks (SNNs), which promise extremely energy-efficient computing. In this paper, we investigate the use of SNNs in the context of channel equalization for ultra-low complexity receivers. We propose an SNN-based equalizer with a feedback structure akin to the decision feedback equalizer (DFE). For conversion of real-world data into spike signals we introduce a novel ternary encoding and compare it with traditional log-scale encoding. We show that our approach clearly outperforms conventional linear equalizers for three different exemplary channels. We highlight that mainly the conversion of the channel output to spikes introduces a small performance penalty. The proposed SNN with a decision feedback structure enables the path to competitive energy-efficient transceivers.注释
Submitted to SCC 2023邮件日期
2022年11月10日- Spiking sampling network for image sparse representation and dynamic vision sensor data compression 时间：2022年11月08日 第一作者：Chunming Jiang 链接.
摘要：稀疏表示已经引起了极大的关注，因为它可以极大地节省存储资源，并在低维空间中找到数据的代表性特征。因此，它可以广泛应用于工程领域，包括特征提取、压缩感知、信号去噪、图像聚类和字典学习，仅举几个例子。在本文中，我们提出了一种脉冲采样网络。这个网络由脉冲神经元组成，它可以根据输入动态决定哪些像素点应该保留，哪些像素点需要掩蔽。我们的实验表明，与随机采样相比，该方法能够更好地稀疏表示原始图像，并有助于图像重建。因此，我们使用这种方法来压缩来自动态视觉传感器的海量数据，这大大降低了对事件数据的存储要求。
英文摘要
Sparse representation has attracted great attention because it can greatly save storage re- sources and find representative features of data in a low-dimensional space. As a result, it may be widely applied in engineering domains including feature extraction, compressed sensing, signal denoising, picture clustering, and dictionary learning, just to name a few. In this paper, we propose a spiking sampling network. This network is composed of spiking neurons, and it can dynamically decide which pixel points should be retained and which ones need to be masked according to the input. Our experiments demonstrate that this approach enables better sparse representation of the original image and facilitates image reconstruction compared to random sampling. We thus use this approach for compressing massive data from the dynamic vision sensor, which greatly reduces the storage requirements for event data.邮件日期
2022年11月09日- Heterogeneous Recurrent Spiking Neural Network for Spatio-Temporal Classification 时间：2022年09月22日 第一作者：Biswadeep Chakraborty 链接.
摘要：脉冲神经网络经常被吹捧为第三波人工智能的大脑启发学习模型。尽管最近用监督反向传播训练的SNN显示出与深度网络相当的分类精度，但基于无监督学习的SNN的性能仍然低得多。本文提出了一种具有无监督学习的异构递归脉冲神经网络（HRSNN），用于基于RGB（KTH，UCF11，UCF101）和基于事件的数据集（DVS128手势）的视频活动识别任务的时空分类。HRSNN的关键新颖之处在于，HRSNN中的重复层由具有不同放电/弛豫动力学的异质神经元组成，并且通过具有不同学习动力学的异质脉冲时间依赖性可塑性（STDP）对每个突触进行训练。我们表明，这种结构和学习方法异质性的新组合优于当前的同质脉冲神经网络。我们进一步表明，HRSNN
英文摘要
Spiking Neural Networks are often touted as brain-inspired learning models for the third wave of Artificial Intelligence. Although recent SNNs trained with supervised backpropagation show classification accuracy comparable to deep networks, the performance of unsupervised learning-based SNNs remains much lower. This paper presents a heterogeneous recurrent spiking neural network (HRSNN) with unsupervised learning for spatio-temporal classification of video activity recognition tasks on RGB (KTH, UCF11, UCF101) and event-based datasets (DVS128 Gesture). The key novelty of the HRSNN is that the recurrent layer in HRSNN consists of heterogeneous neurons with varying firing/relaxation dynamics, and they are trained via heterogeneous spike-time-dependent-plasticity (STDP) with varying learning dynamics for each synapse. We show that this novel combination of heterogeneity in architecture and learning method outperforms current homogeneous spiking neural networks. We further show that HRSNN can achieve similar performance to state-of-the-art backpropagation trained supervised SNN, but with less computation (fewer neurons and sparse connection) and less training data.注释
32 pages, 11 Figures, 4 Tables. arXiv admin note: text overlap with arXiv:1511.03198 by other authors邮件日期
2022年11月09日- Spike-based local synaptic plasticity: A survey of computational models and neuromorphic circuits 时间：2022年11月05日 第一作者：Lyes Khacef 链接.
邮件日期
2022年11月08日- Adversarial Defense via Neural Oscillation inspired Gradient Masking 时间：2022年11月04日 第一作者：Chunming Jiang 链接.
摘要：脉冲神经网络（SNN）由于其低功耗、低延迟和生物合理性而备受关注。随着它们被广泛部署在用于低功耗脑启发计算的神经形态设备中，安全问题变得越来越重要。然而，与深度神经网络（DNN）相比，SNN目前缺乏针对对抗性攻击的专门设计的防御方法。受神经膜电位振荡的启发，我们提出了一种新的神经模型，该模型结合了生物激励振荡机制，以增强SNN的安全性。我们的实验表明，在各种架构和数据集上，具有神经振荡神经元的SNN比具有LIF神经元的普通SNN具有更好的对抗性攻击能力。此外，我们提出了一种防御方法，通过替换振荡的形式来改变模型的梯度，这种方法隐藏了原始的训练梯度，并使攻击者迷惑于使用“假”神经元的梯度来生成
英文摘要
Spiking neural networks (SNNs) attract great attention due to their low power consumption, low latency, and biological plausibility. As they are widely deployed in neuromorphic devices for low-power brain-inspired computing, security issues become increasingly important. However, compared to deep neural networks (DNNs), SNNs currently lack specifically designed defense methods against adversarial attacks. Inspired by neural membrane potential oscillation, we propose a novel neural model that incorporates the bio-inspired oscillation mechanism to enhance the security of SNNs. Our experiments show that SNNs with neural oscillation neurons have better resistance to adversarial attacks than ordinary SNNs with LIF neurons on kinds of architectures and datasets. Furthermore, we propose a defense method that changes model's gradients by replacing the form of oscillation, which hides the original training gradients and confuses the attacker into using gradients of 'fake' neurons to generate invalid adversarial samples. Our experiments suggest that the proposed defense method can effectively resist both single-step and iterative attacks with comparable defense effectiveness and much less computational costs than adversarial training methods on DNNs. To the best of our knowledge, this is the first work that establishes adversarial defense through masking surrogate gradients on SNNs.邮件日期
2022年11月07日- A Ferroelectric Tunnel Junction-based Integrate-and-Fire Neuron 时间：2022年11月04日 第一作者：Paolo Gibertini 链接.
摘要：基于事件的神经形态系统通过使用人工神经元和突触以脉冲形式异步处理数据，提供了一种低功耗的解决方案。铁电隧道结（FTJ）是超低功耗存储器件，非常适合集成在这些系统中。在这里，我们提出了一种混合FTJ-CMOS集成和Fire神经元，它构成了用于边缘计算的新一代神经形态网络的基本构建块。我们演示了通过调谐FTJ装置的开关可实现的电可调谐神经动力学。
英文摘要
Event-based neuromorphic systems provide a low-power solution by using artificial neurons and synapses to process data asynchronously in the form of spikes. Ferroelectric Tunnel Junctions (FTJs) are ultra low-power memory devices and are well-suited to be integrated in these systems. Here, we present a hybrid FTJ-CMOS Integrate-and-Fire neuron which constitutes a fundamental building block for new-generation neuromorphic networks for edge computing. We demonstrate electrically tunable neural dynamics achievable by tuning the switching of the FTJ device.邮件日期
2022年11月07日- Hardware/Software co-design with ADC-Less In-memory Computing Hardware for Spiking Neural Networks 时间：2022年11月03日 第一作者：Marco Paul E. Apolinario 链接.
摘要：脉冲神经网络（SNN）是一种生物可信的模型，在资源受限的边缘设备上实现顺序任务的节能实现具有巨大潜力。然而，基于标准GPU的商业边缘平台没有优化以部署SNN，导致高能量和延迟。虽然模拟内存计算（IMC）平台可以作为节能推理引擎，但它们受到高精度ADC（HP-ADC）巨大的能量、延迟和面积要求的困扰，从而掩盖了内存计算的好处。我们提出了一种硬件/软件协同设计方法，以将SNN部署到无ADC IMC架构中，使用感测放大器作为1位ADC来取代传统的HP ADC并缓解上述问题。我们提出的框架通过执行硬件感知训练而导致最小的精度下降，并且能够从简单的图像分类任务扩展到更复杂的序列回归任务。com上的实验
英文摘要
Spiking Neural Networks (SNNs) are bio-plausible models that hold great potential for realizing energy-efficient implementations of sequential tasks on resource-constrained edge devices. However, commercial edge platforms based on standard GPUs are not optimized to deploy SNNs, resulting in high energy and latency. While analog In-Memory Computing (IMC) platforms can serve as energy-efficient inference engines, they are accursed by the immense energy, latency, and area requirements of high-precision ADCs (HP-ADC), overshadowing the benefits of in-memory computations. We propose a hardware/software co-design methodology to deploy SNNs into an ADC-Less IMC architecture using sense-amplifiers as 1-bit ADCs replacing conventional HP-ADCs and alleviating the above issues. Our proposed framework incurs minimal accuracy degradation by performing hardware-aware training and is able to scale beyond simple image classification tasks to more complex sequential regression tasks. Experiments on complex tasks of optical flow estimation and gesture recognition show that progressively increasing the hardware awareness during SNN training allows the model to adapt and learn the errors due to the non-idealities associated with ADC-Less IMC. Also, the proposed ADC-Less IMC offers significant energy and latency improvements,注释
12 pages, 13 figures邮件日期
2022年11月07日- StereoSpike: Depth Learning with a Spiking Neural Network 时间：2022年11月03日 第一作者：Ulysse Ran\c{c}on 链接.
邮件日期
2022年11月04日- GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks 时间：2022年11月03日 第一作者：Xingting Yao 链接.
注释
Accepted at NeurIPS 2022邮件日期
2022年11月04日- Spiking Variational Graph Auto-Encoders for Efficient Graph Representation Learning 时间：2022年10月24日 第一作者：Hanxuan Yang 链接.
摘要：图表示学习是一个基本的研究问题，有利于图结构数据的广泛应用。传统的基于人工神经网络的方法，如图神经网络（GNNs）和变分图自动编码器（VGAEs）在图上学习方面取得了很好的结果，但它们在训练和推理阶段的能耗极高。受脉冲神经网络（SNN）的生物保真度和能量效率的启发，最近的方法试图通过用脉冲神经元代替激活功能，使GNN适应SNN框架。然而，现有的基于SNN的GNN方法不能应用于由链路预测表示的更一般的多节点表示学习问题。此外，这些方法没有充分利用SNN的生物保真度，因为它们仍然需要昂贵的乘法累加（MAC）操作，这严重损害了能源效率。解决上述问题并改善能源
英文摘要
Graph representation learning is a fundamental research issue and benefits a wide range of applications on graph-structured data. Conventional artificial neural network-based methods such as graph neural networks (GNNs) and variational graph auto-encoders (VGAEs) have achieved promising results in learning on graphs, but they suffer from extremely high energy consumption during training and inference stages. Inspired by the bio-fidelity and energy-efficiency of spiking neural networks (SNNs), recent methods attempt to adapt GNNs to the SNN framework by substituting spiking neurons for the activation functions. However, existing SNN-based GNN methods cannot be applied to the more general multi-node representation learning problem represented by link prediction. Moreover, these methods did not fully exploit the bio-fidelity of SNNs, as they still require costly multiply-accumulate (MAC) operations, which severely harm the energy efficiency. To address the above issues and improve energy efficiency, in this paper, we propose an SNN-based deep generative method, namely the Spiking Variational Graph Auto-Encoders (S-VGAE) for efficient graph representation learning. To deal with the multi-node problem, we propose a probabilistic decoder that generates binary latent variables as spiking node representations and reconstructs graphs via the weighted inner product. To avoid the MAC operations for energy efficiency, we further decouple the propagation and transformation layers of conventional GNN aggregators. We conduct link prediction experiments on multiple benchmark graph datasets, and the results demonstrate that our model consumes significantly lower energy with the performances superior or comparable to other ANN- and SNN-based methods for graph representation learning.邮件日期
2022年11月04日- Bayesian Continual Learning via Spiking Neural Networks 时间：2022年11月01日 第一作者：Nicolas Skatchkovsky 链接.
注释
Accepted for publication in Frontiers in Computational Neuroscience邮件日期
2022年11月02日- A Faster Approach to Spiking Deep Convolutional Neural Networks 时间：2022年10月31日 第一作者：Shahriar Rezghi Shirsavar (University of Tehran 链接.
摘要：脉冲神经网络（SNN）比当前的深度神经网络更接近大脑的动态。它们的低功耗和采样效率使这些网络变得有趣。最近，已经提出了几种深度卷积脉冲神经网络。这些网络旨在提高生物合理性，同时创建用于机器学习任务的强大工具。在这里，我们建议基于先前工作的网络结构，以提高网络运行时间和准确性。对网络的改进包括将训练迭代减少到一次，有效地使用主成分分析（PCA）降维、权重量化、分类的定时输出以及更好的超参数调整。此外，改变预处理步骤以允许处理彩色图像而不是仅处理黑白图像，以提高精度。所提出的结构将运行时细分，并引入了深度卷积SNN的有效方法。
英文摘要
Spiking neural networks (SNNs) have closer dynamics to the brain than current deep neural networks. Their low power consumption and sample efficiency make these networks interesting. Recently, several deep convolutional spiking neural networks have been proposed. These networks aim to increase biological plausibility while creating powerful tools to be applied to machine learning tasks. Here, we suggest a network structure based on previous work to improve network runtime and accuracy. Improvements to the network include reducing training iterations to only once, effectively using principal component analysis (PCA) dimension reduction, weight quantization, timed outputs for classification, and better hyperparameter tuning. Furthermore, the preprocessing step is changed to allow the processing of colored images instead of only black and white to improve accuracy. The proposed structure fractionalizes runtime and introduces an efficient approach to deep convolutional SNNs.注释
6 pages, 7 figures, to be published in the Asilomar 2022 conference ACM-class: I.2.6邮件日期
2022年11月01日- Spiking neural networks for nonlinear regression 时间：2022年10月26日 第一作者：Alex 链接.
邮件日期
2022年10月27日- GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks 时间：2022年10月25日 第一作者：Xingting Yao 链接.
摘要：几十年来，人们对脉冲神经网络（SNN）进行了研究，以结合其生物学合理性并利用其有希望的能源效率。在现有的SNN中，通常采用泄漏整合和激发（LIF）模型来描述脉冲神经元，并演化为具有不同生物学特征的多种变体。然而，大多数基于LIF的神经元在不同的神经元行为中仅支持单一的生物学特征，限制了它们的表达能力和神经元动态多样性。在本文中，我们提出了一种统一的脉冲神经元GLIF，以融合不同神经元行为中的不同生物特征，扩大脉冲神经元的表示空间。在GLIF中，用于确定融合生物特征比例的门控因子在训练期间是可学习的。结合所有可学习的膜相关参数，我们的方法可以使脉冲神经元不同并不断变化，从而增加脉冲神经元的异质性和适应性
英文摘要
Spiking Neural Networks (SNNs) have been studied over decades to incorporate their biological plausibility and leverage their promising energy efficiency. Throughout existing SNNs, the leaky integrate-and-fire (LIF) model is commonly adopted to formulate the spiking neuron and evolves into numerous variants with different biological features. However, most LIF-based neurons support only single biological feature in different neuronal behaviors, limiting their expressiveness and neuronal dynamic diversity. In this paper, we propose GLIF, a unified spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. In GLIF, gating factors, which are exploited to determine the proportion of the fused bio-features, are learnable during training. Combining all learnable membrane-related parameters, our method can make spiking neurons different and constantly changing, thus increasing the heterogeneity and adaptivity of spiking neurons. Extensive experiments on a variety of datasets demonstrate that our method obtains superior performance compared with other SNNs by simply changing their neuronal formulations to GLIF. In particular, we train a spiking ResNet-19 with GLIF and achieve注释
Accepted at NeurIPS 2022邮件日期
2022年10月26日- Does Medical Imaging learn different Convolution Filters? 时间：2022年10月25日 第一作者：Paul Gavrikov 链接.
摘要：最近的工作通过包含数百个异构图像模型的大规模研究，研究了学习卷积滤波器的分布。令人惊讶的是，平均而言，在包括学习任务、图像域或数据集在内的各种研究维度的比较中，分布仅显示出微小的偏差。然而，在所研究的图像域中，医学成像模型似乎通过“脉冲”分布显示出显著的异常值，因此，学习了不同于其他域的高度特异性滤波器簇。根据这一观察，我们更详细地研究了收集的医学成像模型。我们表明，异常值不是根本差异，而是由于某些架构中的特定处理。恰恰相反，对于标准化架构，我们发现基于医学数据训练的模型在过滤器分布上与基于其他领域数据训练的类似架构没有显著差异。我们的结论加强了预测
英文摘要
Recent work has investigated the distributions of learned convolution filters through a large-scale study containing hundreds of heterogeneous image models. Surprisingly, on average, the distributions only show minor drifts in comparisons of various studied dimensions including the learned task, image domain, or dataset. However, among the studied image domains, medical imaging models appeared to show significant outliers through "spikey" distributions, and, therefore, learn clusters of highly specific filters different from other domains. Following this observation, we study the collected medical imaging models in more detail. We show that instead of fundamental differences, the outliers are due to specific processing in some architectures. Quite the contrary, for standardized architectures, we find that models trained on medical data do not significantly differ in their filter distributions from similar architectures trained on data from other domains. Our conclusions reinforce previous hypotheses stating that pre-training of imaging models can be done with any kind of diverse image data.注释
Accepted at MedNeurIPS 2022邮件日期
2022年10月26日- SpikeSim: An end-to-end Compute-in-Memory Hardware Evaluation Tool for Benchmarking Spiking Neural Networks 时间：2022年10月24日 第一作者：Abhishek Moitra 链接.
摘要：SNN是面向能效机器智能的一个积极研究领域。与传统的神经网络相比，SNN使用时间脉冲数据和生物似然的神经元激活功能，如泄漏积分火/积分火（LIF/IF）进行数据处理。然而，在标准冯·诺依曼计算平台中，SNN会产生大量的点积运算，导致高内存和计算开销。如今，内存计算（IMC）架构已被提出，以缓解冯·诺依曼架构中普遍存在的“内存墙瓶颈”。尽管最近的工作提出了基于IMC的SNN硬件加速器，但以下几点被忽略了：1）由于在多个时间步长上重复模拟点积运算，交叉非理想性对SNN性能的不利影响，2）基本SNN特定组件（如LIF/IF和数据通信模块）的硬件开销。为此，我们建议使用SpikeSim，这是一种可以执行逼真性能的工具，ener
英文摘要
SNNs are an active research domain towards energy efficient machine intelligence. Compared to conventional ANNs, SNNs use temporal spike data and bio-plausible neuronal activation functions such as Leaky-Integrate Fire/Integrate Fire (LIF/IF) for data processing. However, SNNs incur significant dot-product operations causing high memory and computation overhead in standard von-Neumann computing platforms. Today, In-Memory Computing (IMC) architectures have been proposed to alleviate the "memory-wall bottleneck" prevalent in von-Neumann architectures. Although recent works have proposed IMC-based SNN hardware accelerators, the following have been overlooked- 1) the adverse effects of crossbar non-ideality on SNN performance due to repeated analog dot-product operations over multiple time-steps, 2) hardware overheads of essential SNN-specific components such as the LIF/IF and data communication modules. To this end, we propose SpikeSim, a tool that can perform realistic performance, energy, latency and area evaluation of IMC-mapped SNNs. SpikeSim consists of a practical monolithic IMC architecture called SpikeFlow for mapping SNNs. Additionally, the non-ideality computation engine (NICE) and energy-latency-area (ELA) engine performs hardware-realistic evaluation of SpikeFlow-mapped SNNs. Based on 65nm CMOS implementation and experiments on CIFAR10, CIFAR100 and TinyImagenet datasets, we find that the LIF/IF neuronal module has significant area contribution (>11% of the total hardware area). We propose SNN topological modifications leading to 1.24x and 10x reduction in the neuronal module's area and the overall energy-delay-product value, respectively. Furthermore, in this work, we perform a holistic comparison between IMC implemented ANN and SNNs and conclude that lower number of time-steps are the key to achieve higher throughput and energy-efficiency for SNNs compared to 4-bit ANNs.注释
14 pages, 22 figures邮件日期
2022年10月25日- Towards Energy-Efficient, Low-Latency and Accurate Spiking LSTMs 时间：2022年10月23日 第一作者：Gourav Datta 链接.
摘要：脉冲神经网络（SNN）已成为复杂视觉任务的一种极具吸引力的时空计算范式。然而，大多数现有的工作产生的模型需要许多时间步长，并且没有利用脉冲神经网络的固有时间动态，即使是连续任务。基于这一观察，我们提出了一种优化的脉冲长-短期记忆网络（LSTM）训练框架，该框架包括一种新的ANN到SNN转换框架，然后是SNN训练。特别是，我们在源LSTM架构中提出了新的激活函数，并明智地选择其中的一个子集进行转换，以集成和激发（IF）具有最佳偏置偏移的激活。此外，我们推导了从其非脉冲LSTM对应物转换的泄漏积分和激发（LIF）激活函数，这证明了需要联合优化权重、阈值和泄漏参数。我们还提出了一种流水线并行处理方案
英文摘要
Spiking Neural Networks (SNNs) have emerged as an attractive spatio-temporal computing paradigm for complex vision tasks. However, most existing works yield models that require many time steps and do not leverage the inherent temporal dynamics of spiking neural networks, even for sequential tasks. Motivated by this observation, we propose an \rev{optimized spiking long short-term memory networks (LSTM) training framework that involves a novel ANN-to-SNN conversion framework, followed by SNN training}. In particular, we propose novel activation functions in the source LSTM architecture and judiciously select a subset of them for conversion to integrate-and-fire (IF) activations with optimal bias shifts. Additionally, we derive the leaky-integrate-and-fire (LIF) activation functions converted from their non-spiking LSTM counterparts which justifies the need to jointly optimize the weights, threshold, and leak parameter. We also propose a pipelined parallel processing scheme which hides the SNN time steps, significantly improving system latency, especially for long sequences. The resulting SNNs have high activation sparsity and require only accumulate operations (AC), in contrast to expensive multiply-and-accumulates (MAC) needed for ANNs, except for the input layer when using direct encoding, yielding significant improvements in energy efficiency. We evaluate our framework on sequential learning tasks including temporal MNIST, Google Speech Commands (GSC), and UCI Smartphone datasets on different LSTM architectures. We obtain test accuracy of 94.75% with only 2 time steps with direct encoding on the GSC dataset with 4.1x lower energy than an iso-architecture standard LSTM.邮件日期
2022年10月25日- Biologically Plausible Variational Policy Gradient with Spiking Recurrent Winner-Take-All Networks 时间：2022年10月21日 第一作者：Zhile Yang 链接.
摘要：强化学习研究的一个方向是探索生物学上合理的模型和算法，以模拟生物智能并适应神经形态硬件。其中，奖励调节的脉冲时间依赖性可塑性（R-STDP）是一个新的分支，在能量效率方面具有良好的潜力。然而，当前的R-STDP方法依赖于局部学习规则的启发式设计，因此需要特定于任务的专家知识。在本文中，我们考虑了一个脉冲递归赢家通吃网络，并提出了一种新的R-STDP方法，即脉冲变分策略梯度（SVPG），其局部学习规则是从全局策略梯度中导出的，因此不需要启发式设计。在MNIST分类和体操倒立摆实验中，我们的SVPG实现了良好的训练性能，并且比传统方法对各种噪声具有更好的鲁棒性。
英文摘要
One stream of reinforcement learning research is exploring biologically plausible models and algorithms to simulate biological intelligence and fit neuromorphic hardware. Among them, reward-modulated spike-timing-dependent plasticity (R-STDP) is a recent branch with good potential in energy efficiency. However, current R-STDP methods rely on heuristic designs of local learning rules, thus requiring task-specific expert knowledge. In this paper, we consider a spiking recurrent winner-take-all network, and propose a new R-STDP method, spiking variational policy gradient (SVPG), whose local learning rules are derived from the global policy gradient and thus eliminate the need for heuristic designs. In experiments of MNIST classification and Gym InvertedPendulum, our SVPG achieves good training performance, and also presents better robustness to various kinds of noises than conventional methods.注释
Accepted to BMVC 2022邮件日期
2022年10月25日- Exact Gradient Computation for Spiking Neural Networks Through Forward Propagation 时间：2022年10月18日 第一作者：Jane H. Lee 链接.
摘要：脉冲神经网络（SNN）最近作为传统神经网络的替代品出现，因为它具有能量效率优势和更好地捕捉生物神经元机制的能力。然而，用于训练传统网络的经典反向传播算法因脉冲时间的硬阈值和不连续性而难以应用于SNN。因此，大多数先前的工作认为SNN相对于其权重的精确梯度不存在，并且专注于产生替代梯度的近似方法。在本文中，（1）通过将隐函数定理应用于离散脉冲时间的SNN，我们证明了尽管SNN在时间上是不可微的，但相对于其权重，SNN具有明确定义的梯度，并且（2）我们提出了一种新的训练算法，称为\emph{前向传播}（FP），它计算SNN的精确梯度。FP利用脉冲之间的因果关系结构，允许我们并行计算
英文摘要
Spiking neural networks (SNN) have recently emerged as alternatives to traditional neural networks, owing to energy efficiency benefits and capacity to better capture biological neuronal mechanisms. However, the classic backpropagation algorithm for training traditional networks has been notoriously difficult to apply to SNN due to the hard-thresholding and discontinuities at spike times. Therefore, a large majority of prior work believes exact gradients for SNN w.r.t. their weights do not exist and has focused on approximation methods to produce surrogate gradients. In this paper, (1) by applying the implicit function theorem to SNN at the discrete spike times, we prove that, albeit being non-differentiable in time, SNNs have well-defined gradients w.r.t. their weights, and (2) we propose a novel training algorithm, called \emph{forward propagation} (FP), that computes exact gradients for SNN. FP exploits the causality structure between the spikes and allows us to parallelize computation forward in time. It can be used with other algorithms that simulate the forward pass, and it also provides insights on why other related algorithms such as Hebbian learning and also recently-proposed surrogate gradient methods may perform well.邮件日期
2022年10月28日- Event-based Stereo Depth Estimation from Ego-motion using Ray Density Fusion 时间：2022年10月17日 第一作者：Suman Ghosh 链接.
摘要：事件摄像机是仿生传感器，通过响应场景中的亮度变化来模拟人类视网膜。它们以微秒分辨率生成基于异步脉冲的输出，与传统相机相比具有高动态范围、低运动模糊和能效等优点。大多数基于事件的立体方法试图利用摄像机的高时间分辨率和摄像机间事件的同时性来建立匹配和估计深度。相比之下，这项工作研究了如何通过融合反向投影光线密度，在没有明确数据关联的情况下，从立体事件摄像机中估计深度，并证明了其对以自我为中心的方式记录的头戴式摄像机数据的有效性。代码和视频可在https://github.com/tub-rip/dvs_mcemvs
英文摘要
Event cameras are bio-inspired sensors that mimic the human retina by responding to brightness changes in the scene. They generate asynchronous spike-based outputs at microsecond resolution, providing advantages over traditional cameras like high dynamic range, low motion blur and power efficiency. Most event-based stereo methods attempt to exploit the high temporal resolution of the camera and the simultaneity of events across cameras to establish matches and estimate depth. By contrast, this work investigates how to estimate depth from stereo event cameras without explicit data association by fusing back-projected ray densities, and demonstrates its effectiveness on head-mounted camera data, which is recorded in an egocentric fashion. Code and video are available at https://github.com/tub-rip/dvs_mcemvs注释
6 pages, 3 figures, project page: https://github.com/tub-rip/dvs_mcemvs Journal-ref: 2nd International Ego4D Workshop at ECCV 2022邮件日期
2022年10月18日- Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks 时间：2022年10月12日 第一作者：Lang Feng 链接.
摘要：脉冲神经网络（SNN）是一种具有异步离散和稀疏特性的仿生神经网络，其在低能耗方面的优势日益显现。最近的研究致力于利用时空信息通过反向传播直接训练SNN。然而，脉冲活动的二元和不可微分特性迫使直接训练的SNN遭受严重的梯度消失和网络退化，这极大地限制了直接训练SNN的性能，并阻止其深入。在本文中，我们提出了一种基于现有时空反向传播（STBP）方法和脉冲休眠抑制残差网络（脉冲DS-ResNet）的多级激发（MLF）方法。MLF能够实现更有效的梯度传播和神经元的增量表达能力。Spiking DS ResNet可以有效地执行离散脉冲的身份映射，并提供更合适的连接
英文摘要
Spiking neural networks (SNNs) are bio-inspired neural networks with asynchronous discrete and sparse characteristics, which have increasingly manifested their superiority in low energy consumption. Recent research is devoted to utilizing spatio-temporal information to directly train SNNs by backpropagation. However, the binary and non-differentiable properties of spike activities force directly trained SNNs to suffer from serious gradient vanishing and network degradation, which greatly limits the performance of directly trained SNNs and prevents them from going deeper. In this paper, we propose a multi-level firing (MLF) method based on the existing spatio-temporal back propagation (STBP) method, and spiking dormant-suppressed residual network (spiking DS-ResNet). MLF enables more efficient gradient propagation and the incremental expression ability of the neurons. Spiking DS-ResNet can efficiently perform identity mapping of discrete spikes, as well as provide a more suitable connection for gradient propagation in deep SNNs. With the proposed method, our model achieves superior performances on a non-neuromorphic dataset and two neuromorphic datasets with much fewer trainable parameters and demonstrates the great ability to combat the gradient vanishing and degradation problem in deep SNNs.邮件日期
2022年10月13日- Spiking Neural Operators for Scientific Machine Learning 时间：2022年10月12日 第一作者：Adar Kahana 链接.
注释
16 pages, 6 figures and 4 tables邮件日期
2022年10月13日- Robust and accelerated single-spike spiking neural network training with applicability to challenging temporal tasks 时间：2022年10月11日 第一作者：Luke Taylor 链接.
注释
18 pages, 6 figures, under review at ICLR 2023邮件日期
2022年10月13日- STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networks 时间：2022年10月11日 第一作者：Chengting Yu 链接.
摘要：脉冲神经网络（SNNs）作为神经形态计算中的算法模型之一，由于具有时间信息处理能力、低功耗和高生物可信性，受到了广泛的研究关注。高效提取时空特征的潜力使其适合于处理事件流。然而，SNN中现有的突触结构几乎是完全连接或空间2D卷积，两者都不能充分提取时间依赖性。在这项工作中，我们从生物突触中获得灵感，并提出了时空突触连接SNN（STSC-SNN）模型，以增强突触连接的时空感受野，从而建立跨层的时间依赖性。具体来说，我们结合了时间卷积和注意力机制来实现突触过滤和门控功能。我们表明，赋予突触模型时间依赖性可以提高
英文摘要
Spiking Neural Networks (SNNs), as one of the algorithmic models in neuromorphic computing, have gained a great deal of research attention owing to temporal information processing capability, low power consumption, and high biological plausibility. The potential to efficiently extract spatio-temporal features makes it suitable for processing the event streams. However, existing synaptic structures in SNNs are almost full-connections or spatial 2D convolution, neither of which can extract temporal dependencies adequately. In this work, we take inspiration from biological synapses and propose a spatio-temporal synaptic connection SNN (STSC-SNN) model, to enhance the spatio-temporal receptive fields of synaptic connections, thereby establishing temporal dependencies across layers. Concretely, we incorporate temporal convolution and attention mechanisms to implement synaptic filtering and gating functions. We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks. In addition, we investigate the impact of performance vias varied spatial-temporal receptive fields and reevaluate the temporal modules in SNNs. Our approach is tested on neuromorphic datasets, including DVS128 Gesture (gesture recognition), N-MNIST, CIFAR10-DVS (image classification), and SHD (speech digit recognition). The results show that the proposed model outperforms the state-of-the-art accuracy on nearly all datasets.邮件日期
2022年10月12日- Event-Driven Tactile Learning with Various Location Spiking Neurons 时间：2022年10月11日 第一作者：Peng Kang 链接.
注释
This paper is under review in IEEE TNNLS. Please note that this paper is a journal extension of our previous conference paper: arXiv:2209.01080. This paper includes more novel models, data, experiments, and demonstration邮件日期
2022年10月12日- Energy-Efficient Deployment of Machine Learning Workloads on Neuromorphic Hardware 时间：2022年10月10日 第一作者：Peyton Ch 链接.
摘要：随着科技行业正朝着在较小的边缘计算设备上实现自然语言处理、路径规划、图像分类等任务的方向发展，对算法和硬件加速器更高效实现的需求已成为一个重要的研究领域。近年来，已经发布了几款边缘深度学习硬件加速器，专门致力于减少深度神经网络（DNN）消耗的功率和面积。另一方面，在离散时间序列数据上运行的脉冲神经网络（SNN）已被证明在部署在基于神经形态事件的专用/异步硬件上时，甚至比上述边缘DNN加速器实现了显著的功率降低。尽管神经形态硬件在加速边缘深度学习任务方面显示出巨大的潜力，但目前算法和硬件的空间有限，仍处于相当早期的发展阶段。因此，许多混合方法
英文摘要
As the technology industry is moving towards implementing tasks such as natural language processing, path planning, image classification, and more on smaller edge computing devices, the demand for more efficient implementations of algorithms and hardware accelerators has become a significant area of research. In recent years, several edge deep learning hardware accelerators have been released that specifically focus on reducing the power and area consumed by deep neural networks (DNNs). On the other hand, spiking neural networks (SNNs) which operate on discrete time-series data, have been shown to achieve substantial power reductions over even the aforementioned edge DNN accelerators when deployed on specialized neuromorphic event-based/asynchronous hardware. While neuromorphic hardware has demonstrated great potential for accelerating deep learning tasks at the edge, the current space of algorithms and hardware is limited and still in rather early development. Thus, many hybrid approaches have been proposed which aim to convert pre-trained DNNs into SNNs. In this work, we provide a general guide to converting pre-trained DNNs into SNNs while also presenting techniques to improve the deployment of converted SNNs on neuromorphic hardware with respect to latency, power, and energy. Our experimental results show that when compared against the Intel Neural Compute Stick 2, Intel's neuromorphic processor, Loihi, consumes up to 27x less power and 5x less energy in the tested image classification tasks by using our SNN improvement techniques.邮件日期
2022年10月12日- Training Spiking Neural Networks with Local Tandem Learning 时间：2022年10月10日 第一作者：Qu Yang 链接.
摘要：脉冲神经网络（SNN）被证明在生物学上比其前辈更合理，更节能。然而，对于深度SNN，尤其是对于部署在模拟计算基板上的深度SNN来说，缺乏一种有效且通用的训练方法。在本文中，我们提出了一种广义学习规则，称为局部串联学习（LTL）。LTL规则通过模仿预先训练的ANN的中间特征表示来遵循师生学习方法。通过解耦网络层的学习并利用高度信息量的监督信号，我们在CIFAR-10数据集的五个训练周期内证明了网络的快速收敛，同时具有较低的计算复杂度。我们的实验结果还表明，这样训练的SNN可以在CIFAR-10、CIFAR-100和Tiny ImageNet数据集上达到与其教师ANN相当的精度。此外，所提出的LTL规则是硬件友好的。它可以很容易地在芯片上实现
英文摘要
Spiking neural networks (SNNs) are shown to be more biologically plausible and energy efficient over their predecessors. However, there is a lack of an efficient and generalized training method for deep SNNs, especially for deployment on analog computing substrates. In this paper, we put forward a generalized learning rule, termed Local Tandem Learning (LTL). The LTL rule follows the teacher-student learning approach by mimicking the intermediate feature representations of a pre-trained ANN. By decoupling the learning of network layers and leveraging highly informative supervisor signals, we demonstrate rapid network convergence within five training epochs on the CIFAR-10 dataset while having low computational complexity. Our experimental results have also shown that the SNNs thus trained can achieve comparable accuracies to their teacher ANNs on CIFAR-10, CIFAR-100, and Tiny ImageNet datasets. Moreover, the proposed LTL rule is hardware friendly. It can be easily implemented on-chip to perform fast parameter calibration and provide robustness against the notorious device non-ideality issues. It, therefore, opens up a myriad of opportunities for training and deployment of SNN on ultra-low-power mixed-signal neuromorphic computing chips.10注释
Accepted by NeurIPS 2022邮件日期
2022年10月11日- Online Training Through Time for Spiking Neural Networks 时间：2022年10月09日 第一作者：Mingqing Xiao 链接.
摘要：脉冲神经网络（SNN）是一种很有前途的以大脑为灵感的节能模型。训练方法的最新进展使深度SNN能够成功地执行低延迟的大规模任务。特别是，具有替代梯度（SG）的时间反向传播（BPTT）被广泛用于在非常小的时间步长内实现高性能。然而，它的代价是大量的内存消耗用于训练，缺乏优化的理论清晰度，以及与生物学习的在线属性和神经形态硬件的规则不一致。其他工作将SNN的脉冲表示与等效人工神经网络公式联系起来，并通过来自等效映射的梯度来训练SNN，以确保下降方向。但它们无法实现低延迟，也无法在线。在这项工作中，我们提出了SNN的在线时间训练（OTTT），它源自BPTT，通过跟踪突触前活动和杠杆来实现前向时间学习
英文摘要
Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Recent progress in training methods has enabled successful deep SNNs on large-scale tasks with low latency. Particularly, backpropagation through time (BPTT) with surrogate gradients (SG) is popularly used to achieve high performance in a very small number of time steps. However, it is at the cost of large memory consumption for training, lack of theoretical clarity for optimization, and inconsistency with the online property of biological learning and rules on neuromorphic hardware. Other works connect spike representations of SNNs with equivalent artificial neural network formulation and train SNNs by gradients from equivalent mappings to ensure descent directions. But they fail to achieve low latency and are also not online. In this work, we propose online training through time (OTTT) for SNNs, which is derived from BPTT to enable forward-in-time learning by tracking presynaptic activities and leveraging instantaneous loss and gradients. Meanwhile, we theoretically analyze and prove that gradients of OTTT can provide a similar descent direction for optimization as gradients based on spike representations under both feedforward and recurrent conditions. OTTT only requires constant training memory costs agnostic to time steps, avoiding the significant memory costs of BPTT for GPU training. Furthermore, the update rule of OTTT is in the form of three-factor Hebbian learning, which could pave a path for online on-chip learning. With OTTT, it is the first time that two mainstream supervised SNN training methods, BPTT with SG and spike representation-based training, are connected, and meanwhile in a biologically plausible form. Experiments on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS demonstrate the superior performance of our method on large-scale static and neuromorphic datasets in small time steps.注释
Accepted by NeurIPS 2022邮件日期
2022年10月11日- Event-Driven Tactile Learning with Various Location Spiking Neurons 时间：2022年10月09日 第一作者：Peng Kang 链接.
摘要：触觉感应对于各种日常任务至关重要。事件驱动触觉传感器和脉冲神经网络（SNN）的新进展推动了相关领域的研究。然而，由于现有脉冲神经元的有限表示能力和数据的高时空复杂性，SNN支持的事件驱动触觉学习仍处于初级阶段。在本文中，为了提高现有脉冲神经元的表示能力，我们提出了一种称为“位置脉冲神经元”的新神经元模型，该模型使我们能够以一种新的方式提取基于事件的数据的特征。具体而言，基于经典的时间脉冲响应模型（TSRM），我们开发了位置脉冲响应模式（LSRM）。此外，基于最常用的时间泄漏集成和火灾（TLIF）模型，我们开发了位置泄漏集成和消防（LLIF）模式。通过利用新的位置脉冲神经元，我们提出了几种模型来捕捉复杂的时空依赖关系
英文摘要
Tactile sensing is essential for a variety of daily tasks. New advances in event-driven tactile sensors and Spiking Neural Networks (SNNs) spur the research in related fields. However, SNN-enabled event-driven tactile learning is still in its infancy due to the limited representation abilities of existing spiking neurons and high spatio-temporal complexity in the data. In this paper, to improve the representation capability of existing spiking neurons, we propose a novel neuron model called "location spiking neuron", which enables us to extract features of event-based data in a novel way. Specifically, based on the classical Time Spike Response Model (TSRM), we develop the Location Spike Response Model (LSRM). In addition, based on the most commonly-used Time Leaky Integrate-and-Fire (TLIF) model, we develop the Location Leaky Integrate-and-Fire (LLIF) model. By exploiting the novel location spiking neurons, we propose several models to capture the complex spatio-temporal dependencies in the event-driven tactile data. Extensive experiments demonstrate the significant improvements of our models over other works on event-driven tactile learning and show the superior energy efficiency of our models and location spiking neurons, which may unlock their potential on neuromorphic hardware.注释
under review. arXiv admin note: substantial text overlap with arXiv:2209.01080邮件日期
2022年10月11日- Fusing Event-based Camera and Radar for SLAM Using Spiking Neural Networks with Continual STDP Learning 时间：2022年10月09日 第一作者：Ali Safa 链接.
摘要：这项工作提出了一种融合基于事件的摄像机和调频连续波（FMCW）雷达的无人机导航SLAM架构。每个传感器都由生物启发的脉冲神经网络（SNN）处理，具有连续的脉冲时间依赖性可塑性（STDP）学习，如在大脑中观察到的。与大多数基于学习的SLAM系统%（a）需要获取必须执行导航的环境的代表性数据集，b）需要离线训练阶段）相比，我们的方法不需要任何离线训练阶段，而是SNN通过STDP不断地从飞行中的输入数据中学习特征。同时，SNN输出被用作环路闭合检测和映射校正的特征描述符。我们进行了大量实验，以对照最先进的RGB方法对我们的系统进行基准测试，并证明了我们的DVS雷达SLAM方法在强光照变化下的鲁棒性。
英文摘要
This work proposes a first-of-its-kind SLAM architecture fusing an event-based camera and a Frequency Modulated Continuous Wave (FMCW) radar for drone navigation. Each sensor is processed by a bio-inspired Spiking Neural Network (SNN) with continual Spike-Timing-Dependent Plasticity (STDP) learning, as observed in the brain. In contrast to most learning-based SLAM systems%, which a) require the acquisition of a representative dataset of the environment in which navigation must be performed and b) require an off-line training phase, our method does not require any offline training phase, but rather the SNN continuously learns features from the input data on the fly via STDP. At the same time, the SNN outputs are used as feature descriptors for loop closure detection and map correction. We conduct numerous experiments to benchmark our system against state-of-the-art RGB methods and we demonstrate the robustness of our DVS-Radar SLAM approach under strong lighting variations.邮件日期
2022年10月11日- Toward an Over-parameterized Direct-Fit Model of Visual Perception 时间：2022年10月07日 第一作者：Xin Li 链接.
摘要：在这篇文章中，我们重新讨论了简单和复杂细胞的计算建模问题，以用于过度参数化和直接拟合的视觉感知模型。与传统观点不同，我们强调了简单细胞和复杂细胞之间平行和顺序结合机制的差异。我们提出了一个将它们抽象为空间划分和组合的新建议，作为我们新层次结构的基础。我们的构造可以解释为现有k-d树的基于产品拓扑的泛化，使其适用于高维空间中的蛮力直接拟合。所构建的模型已应用于神经科学和心理学的几个经典实验。我们提供了对构建的视觉模型的反稀疏编码解释，并展示了它如何基于$\ell_｛\infty｝$优化导致类似动态规划（DP）的近似最近邻搜索。我们还简要讨论了两种可能的基于
英文摘要
In this paper, we revisit the problem of computational modeling of simple and complex cells for an over-parameterized and direct-fit model of visual perception. Unlike conventional wisdom, we highlight the difference in parallel and sequential binding mechanisms between simple and complex cells. A new proposal for abstracting them into space partitioning and composition is developed as the foundation of our new hierarchical construction. Our construction can be interpreted as a product topology-based generalization of the existing k-d tree, making it suitable for brute-force direct-fit in a high-dimensional space. The constructed model has been applied to several classical experiments in neuroscience and psychology. We provide an anti-sparse coding interpretation of the constructed vision model and show how it leads to a dynamic programming (DP)-like approximate nearest-neighbor search based on邮件日期
2022年10月11日- Spiking neural network for nonlinear regression 时间：2022年10月06日 第一作者：Alex 链接.
摘要：脉冲神经网络，也常被称为第三代神经网络，与传统的第二代神经网络相比，具有大幅减少记忆和能量消耗的潜力。受人类大脑无可争议的效率启发，他们引入了时间和神经元稀疏性，这可以被下一代神经形态硬件利用。为了打开工程应用的道路，我们在连续介质力学的背景下引入了这项令人兴奋的技术。然而，脉冲神经网络的性质对回归问题提出了挑战，这些问题在工程科学建模中经常出现。为了克服这个问题，提出了一种使用脉冲神经网络的回归框架。特别地，介绍了一种利用脉冲神经元的膜电位将二进制脉冲序列解码为实数的网络拓扑。由于本文的目的是简明扼要地介绍这种新方法
英文摘要
Spiking neural networks, also often referred to as the third generation of neural networks, carry the potential for a massive reduction in memory and energy consumption over traditional, second-generation neural networks. Inspired by the undisputed efficiency of the human brain, they introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. To open the pathway toward engineering applications, we introduce this exciting technology in the context of continuum mechanics. However, the nature of spiking neural networks poses a challenge for regression problems, which frequently arise in the modeling of engineering sciences. To overcome this problem, a framework for regression using spiking neural networks is proposed. In particular, a network topology for decoding binary spike trains to real numbers is introduced, utilizing the membrane potential of spiking neurons. As the aim of this contribution is a concise introduction to this new methodology, several different spiking neural architectures, ranging from simple spiking feed-forward to complex spiking long short-term memory neural networks, are derived. Several numerical experiments directed towards regression of linear and nonlinear, history-dependent material models are carried out. A direct comparison with counterparts of traditional neural networks shows that the proposed framework is much more efficient while retaining precision and generalizability. All code has been made publicly available in the interest of reproducibility and to promote continued enhancement in this new domain.邮件日期
2022年10月10日- A Basic Compositional Model for Spiking Neural Networks 时间：2022年10月06日 第一作者：Nancy Lynch 链接.
邮件日期
2022年10月10日- SATA: Sparsity-Aware Training Accelerator for Spiking Neural Networks 时间：2022年10月06日 第一作者：Ruokai Yin 链接.
注释
13 pages. Accepted to IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2022)邮件日期
2022年10月07日- GLM-130B: An Open Bilingual Pre-trained Model 时间：2022年10月05日 第一作者：Aohan Zeng 链接.
摘要：我们介绍了GLM-130B，一个双语（英语和汉语）预训练语言模型，具有1300亿个参数。这是一次尝试，试图开源一个至少与GPT-3一样好的100B规模模型，并揭示如何成功地预训练这样规模的模型。在这一努力的过程中，我们面临着许多意想不到的技术和工程挑战，特别是在损耗峰值和不收敛方面。在本文中，我们介绍了GLM-130B的训练过程，包括其设计选择、效率和稳定性的训练策略以及工程努力。最终的GLM-130B模型在广泛的流行英语基准测试中表现优于GPT-3 175B，而OPT-175B和BLOOM-176B没有表现出性能优势。在相关基准测试中，它也始终显著优于最大的中文模型ERNIE TITAN 3.0 260B。最后，我们利用GLM-130B的独特缩放特性来达到INT4量化
英文摘要
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source a 100B-scale model at least as good as GPT-3 and unveil how models of such a scale can be successfully pre-trained. Over the course of this effort, we face numerous unexpected technical and engineering challenges, particularly on loss spikes and disconvergence. In this paper, we introduce the training process of GLM-130B including its design choices, training strategies for both efficiency and stability, and engineering efforts. The resultant GLM-130B model offers significant outperformance over GPT-3 175B on a wide range of popular English benchmarks while the performance advantage is not observed in OPT-175B and BLOOM-176B. It also consistently and significantly outperforms ERNIE TITAN 3.0 260B -- the largest Chinese language model -- across related benchmarks. Finally, we leverage a unique scaling property of GLM-130B to reach INT4 quantization, without quantization aware training and with almost no performance loss, making it the first among 100B-scale models. More importantly, the property allows its effective inference on 4$\times$RTX 3090 (24G) or 8$\times$RTX 2080 Ti (11G) GPUs, the most ever affordable GPUs required for using 100B-scale models. The GLM-130B model weights are publicly accessible and its code, training logs, related toolkit, and lessons learned are open-sourced at https://github.com/THUDM/GLM-130B .注释
47 pages邮件日期
2022年10月06日- In the realm of hybrid Brain: Human Brain and AI 时间：2022年10月04日 第一作者：Hoda Fares 链接.
摘要：随着神经科学和工程学的最新发展，现在可以记录大脑信号并对其进行解码。此外，越来越多的刺激方法已经出现，以调节和影响大脑活动。当前的脑机接口（BCI）技术主要是治疗结果，它已经证明了其作为严重运动障碍患者的辅助和康复技术的效率。最近，人工智能（AI）和机器学习（ML）技术被用于解码大脑信号。除此之外，人工智能与先进脑机接口以植入式神经技术的形式结合，为神经和精神疾病的诊断、预测和治疗提供了新的可能性。在此背景下，我们设想开发闭环、智能、低功耗和小型化的神经接口，该接口将使用脑启发的人工智能技术和神经形态硬件来处理来自大脑的数据。这将
英文摘要
With the recent developments in neuroscience and engineering, it is now possible to record brain signals and decode them. Also, a growing number of stimulation methods have emerged to modulate and influence brain activity. Current brain-computer interface (BCI) technology is mainly on therapeutic outcomes, it already demonstrated its efficiency as assistive and rehabilitative technology for patients with severe motor impairments. Recently, artificial intelligence (AI) and machine learning (ML) technologies have been used to decode brain signals. Beyond this progress, combining AI with advanced BCIs in the form of implantable neurotechnologies grants new possibilities for the diagnosis, prediction, and treatment of neurological and psychiatric disorders. In this context, we envision the development of closed loop, intelligent, low-power, and miniaturized neural interfaces that will use brain inspired AI techniques with neuromorphic hardware to process the data from the brain. This will be referred to as Brain Inspired Brain Computer Interfaces (BI-BCIs). Such neural interfaces would offer access to deeper brain regions and better understanding for brain's functions and working mechanism, which improves BCIs operative stability and system's efficiency. On one hand, brain inspired AI algorithms represented by spiking neural networks (SNNs) would be used to interpret the multimodal neural signals in the BCI system. On the other hand, due to the ability of SNNs to capture rich dynamics of biological neurons and to represent and integrate different information dimensions such as time, frequency, and phase, it would be used to model and encode complex information processing in the brain and to provide feedback to the users. This paper provides an overview of the different methods to interface with the brain, presents future applications and discusses the merger of AI and BCIs.注释
41 Pages, 10 Figures,邮件日期
2022年10月05日- Event-based Temporally Dense Optical Flow Estimation with Sequential Neural Networks 时间：2022年10月03日 第一作者：Wachirawit Ponghiran 链接.
摘要：先前关于基于事件的光流估计的工作已经研究了几种基于梯度的学习方法，以训练用于预测光流的神经网络。然而，它们不利用事件数据流的快速数据速率，并且依赖于由固定时间段内（通常在两个灰度帧之间）的事件集合构建的时空表示。结果，仅在远低于基于事件的相机产生的速率数据的频率下评估光流，导致时间上稀疏的光流估计。为了预测时间密集的光流，我们将该问题视为一个顺序学习任务，并提出了一种训练方法来训练顺序网络，以便对事件流进行连续预测。我们提出了两种类型的网络：一种注重性能，另一种注重计算效率。我们首先在DSEC数据集上训练长短期存储网络（LSTM），并演示了10倍的时间密集光流
英文摘要
Prior works on event-based optical flow estimation have investigated several gradient-based learning methods to train neural networks for predicting optical flow. However, they do not utilize the fast data rate of event data streams and rely on a spatio-temporal representation constructed from a collection of events over a fixed period of time (often between two grayscale frames). As a result, optical flow is only evaluated at a frequency much lower than the rate data is produced by an event-based camera, leading to a temporally sparse optical flow estimation. To predict temporally dense optical flow, we cast the problem as a sequential learning task and propose a training methodology to train sequential networks for continuous prediction on an event stream. We propose two types of networks: one focused on performance and another focused on compute efficiency. We first train long-short term memory networks (LSTMs) on the DSEC dataset and demonstrated 10x temporally dense optical flow estimation over existing flow estimation approaches. The additional benefit of having a memory to draw long temporal correlations back in time results in a 19.7% improvement in flow prediction accuracy of LSTMs over similar networks with no memory elements. We subsequently show that the inherent recurrence of spiking neural networks (SNNs) enables them to learn and estimate temporally dense optical flow with 31.8% lesser parameters than LSTM, but with a slightly increased error. This demonstrates potential for energy-efficient implementation of fast optical flow prediction using SNNs.注释
There are 16 pages, 5 figures and 2 tables in total邮件日期
2022年10月05日- Efficient Spiking Transformer Enabled By Partial Information 时间：2022年10月03日 第一作者：Ziqing Wang 链接.
摘要：近年来，脉冲神经网络（SNN）由于其稀疏和异步通信特性而受到了广泛关注，因此可以部署在神经形态硬件中并实现极高的能效。然而，SNN目前难以实现与人工神经网络（ANN）相当的性能，因为其有限的可扩展性不允许大规模网络。特别是对于Transformer，作为一种在各种机器学习任务中表现出色的神经网络模型，它通过传统方法在SNN中的实现需要大量神经元，尤其是在自我注意模块中。受神经系统机制的启发，我们提出了一种有效的脉冲变压器（EST）框架，该框架由部分信息支持，以解决上述问题。在该模型中，我们不仅用合理数量的神经元实现了自我注意模块，还引入了部分信息自组织模式
英文摘要
Spiking neural networks (SNNs) have received substantial attention in recent years due to their sparse and asynchronous communication nature, and thus can be deployed in neuromorphic hardware and achieve extremely high energy efficiency. However, SNNs currently can hardly realize a comparable performance to that of artificial neural networks (ANNs) because their limited scalability does not allow for large-scale networks. Especially for Transformer, as a model of ANNs that has accomplished remarkable performance in various machine learning tasks, its implementation in SNNs by conventional methods requires a large number of neurons, notably in the self-attention module. Inspired by the mechanisms in the nervous system, we propose an efficient spiking Transformer (EST) framework enabled by partial information to address the above problem. In this model, we not only implemented the self-attention module with a reasonable number of neurons, but also introduced partial-information self-attention (PSA), which utilizes only partial input signals, further reducing computational resources compared to conventional methods. The experimental results show that our EST can outperform the state-of-the-art SNN model in terms of accuracy and the number of time steps on both Cifar-10/100 and ImageNet datasets. In particular, the proposed EST model achieves 78.48% top-1 accuracy on the ImageNet dataset with only 16 time steps. In addition, our proposed PSA reduces flops by 49.8% with negligible performance loss compared to a self-attention module with full information.邮件日期
2022年10月05日- Learning over time using a neuromorphic adaptive control algorithm for robotic arms 时间：2022年10月03日 第一作者：Lazar Supic 链接.
摘要：在本文中，我们通过部署和彻底评估基于脉冲神经网络的自适应控制算法，探索了机器人手臂学习由手臂末端执行器可到达的位置（x，y，z）定义的底层操作空间的能力，包括干扰。尽管传统的机器人控制算法在适应新环境和动态环境方面存在局限性，但我们表明，机器人手臂可以随着时间的推移更快地学习操作空间和完成任务。我们还证明了基于SNN的自适应机器人控制算法能够在保持能量效率的同时实现快速响应。我们通过对自适应算法参数空间进行广泛搜索，并评估不同SNN网络大小、学习速率、动态机器人手臂轨迹和响应时间的算法性能，获得了这些结果。我们表明，机器人手臂在特定的实验场景中学习完成任务的速度要快15%，比如wi场景
英文摘要
In this paper, we explore the ability of a robot arm to learn the underlying operation space defined by the positions (x, y, z) that the arm's end-effector can reach, including disturbances, by deploying and thoroughly evaluating a Spiking Neural Network SNN-based adaptive control algorithm. While traditional control algorithms for robotics have limitations in both adapting to new and dynamic environments, we show that the robot arm can learn the operational space and complete tasks faster over time. We also demonstrate that the adaptive robot control algorithm based on SNNs enables a fast response while maintaining energy efficiency. We obtained these results by performing an extensive search of the adaptive algorithm parameter space, and evaluating algorithm performance for different SNN network sizes, learning rates, dynamic robot arm trajectories, and response times. We show that the robot arm learns to complete tasks 15% faster in specific experiment scenarios such as scenarios with six or nine random target points.邮件日期
2022年10月05日- DOTIE -- Detecting Objects through Temporal Isolation of Events using a Spiking Architecture 时间：2022年10月03日 第一作者：Manish Nagaraj 链接.
摘要：基于视觉的自主导航系统依靠快速准确的目标检测算法来避开障碍物。由于用于部署的硬件能量有限，为此类系统设计的算法和传感器需要计算效率高。生物启发的事件摄像机由于其速度、能效和对不同照明条件的鲁棒性，是此类系统的视觉传感器的良好候选。然而，传统的计算机视觉算法无法处理基于事件的输出，因为它们缺乏光度特征，如光强度和纹理。在这项工作中，我们提出了一种利用事件中固有的时间信息来有效检测运动对象的新技术。我们的技术包括一个轻量级的脉冲神经架构，它能够根据相应对象的速度来分离事件。然后，这些分离的事件在空间上进一步分组以确定对象边界。这个m
英文摘要
Vision-based autonomous navigation systems rely on fast and accurate object detection algorithms to avoid obstacles. Algorithms and sensors designed for such systems need to be computationally efficient, due to the limited energy of the hardware used for deployment. Biologically inspired event cameras are a good candidate as a vision sensor for such systems due to their speed, energy efficiency, and robustness to varying lighting conditions. However, traditional computer vision algorithms fail to work on event-based outputs, as they lack photometric features such as light intensity and texture. In this work, we propose a novel technique that utilizes the temporal information inherently present in the events to efficiently detect moving objects. Our technique consists of a lightweight spiking neural architecture that is able to separate events based on the speed of the corresponding objects. These separated events are then further grouped spatially to determine object boundaries. This method of object detection is both asynchronous and robust to camera noise. In addition, it shows good performance in scenarios with events generated by static objects in the background, where existing event-based algorithms fail. We show that by utilizing our architecture, autonomous navigation systems can have minimal latency and energy overheads for performing object detection.邮件日期
2022年10月04日- Supervised Parameter Estimation of Neuron Populations from Multiple Firing Events 时间：2022年10月02日 第一作者：Long Le 链接.
摘要：数学模型中生物神经元的放电动力学通常由模型参数决定，这些参数代表了神经元的潜在特性。参数估计问题寻求从单个神经元或神经元群体对外部刺激的反应以及它们之间的相互作用中恢复这些参数。文献中解决此问题的最常用方法是使用一些机械模型与基于仿真或基于解决方案的优化方案相结合。在本文中，我们研究了一种通过监督学习从由脉冲序列和参数标签对组成的训练集中学习神经元种群参数的自动方法。与之前的工作不同，这种自动学习不需要在推理时进行额外的模拟，也不需要在推导分析解或构建一些近似模型时获得专家知识。我们用不同的参数设置模拟了许多神经元群体
英文摘要
The firing dynamics of biological neurons in mathematical models is often determined by the model's parameters, representing the neurons' underlying properties. The parameter estimation problem seeks to recover those parameters of a single neuron or a neuron population from their responses to external stimuli and interactions between themselves. Most common methods for tackling this problem in the literature use some mechanistic models in conjunction with either a simulation-based or solution-based optimization scheme. In this paper, we study an automatic approach of learning the parameters of neuron populations from a training set consisting of pairs of spiking series and parameter labels via supervised learning. Unlike previous work, this automatic learning does not require additional simulations at inference time nor expert knowledge in deriving an analytical solution or in constructing some approximate models. We simulate many neuronal populations with different parameter settings using a stochastic neuron model. Using that data, we train a variety of supervised machine learning models, including convolutional and deep neural networks, random forest, and support vector regression. We then compare their performance against classical approaches including a genetic search, Bayesian sequential estimation, and a random walk approximate model. The supervised models almost always outperform the classical methods in parameter estimation and spike reconstruction errors, and computation expense. Convolutional neural network, in particular, is the best among all models across all metrics. The supervised models can also generalize to out-of-distribution data to a certain extent.注释
31 pages邮件日期
2022年10月05日- RISC-V Toolchain and Agile Development based Open-source Neuromorphic Processor 时间：2022年10月02日 第一作者：Jiulong Wang 链接.
摘要：近几十年来，旨在模仿大脑行为的神经形态计算在计算机科学的各个领域得到了发展。人工神经网络（ANN）是人工智能（AI）中的一个重要概念。它用于识别和分类。为了探索一种更好的方法来在硬件上模拟获得的大脑行为，这种方法既快速又节能，研究人员需要一种先进的方法，如神经形态计算。在这种情况下，脉冲神经网络（SNN）成为硬件实现的最佳选择。最近的工作集中于加速SNN计算。然而，大多数加速器解决方案基于CPU加速器架构，由于该结构中的复杂控制流，该架构能量效率低下。本文提出了文曲星22A，一种低功耗的神经形态处理器，它结合了通用CPU功能和SNN，通过RISC-V SNN扩展指令有效地计算它。文曲星22A的主要**是
英文摘要
In recent decades, neuromorphic computing aiming to imitate brains' behaviors has been developed in various fields of computer science. The Artificial Neural Network (ANN) is an important concept in Artificial Intelligence (AI). It is utilized in recognition and classification. To explore a better way to simulate obtained brain behaviors, which is fast and energy-efficient, on hardware, researchers need an advanced method such as neuromorphic computing. In this case, Spiking Neural Network (SNN) becomes an optimal choice in hardware implementation. Recent works are focusing on accelerating SNN computing. However, most accelerator solutions are based on CPU-accelerator architecture which is energy-inefficient due to the complex control flows in this structure. This paper proposes Wenquxing 22A, a low-power neuromorphic processor that combines general-purpose CPU functions and SNN to efficiently compute it with RISC-V SNN extension instructions. The main idea of Wenquxing 22A is to integrate the SNN calculation unit into the pipeline of a general-purpose CPU to achieve low-power computing with customized RISC-V SNN instructions version 1.0 (RV-SNN V1.0), Streamlined Leaky Integrate-and-Fire (LIF) model, and the binary stochastic Spike-timing-dependent-plasticity (STDP). The source code of Wenquxing 22A is released online on Gitee and GitHub. We apply Wenquxing 22A to the recognition of the MNIST dataset to make a comparison with other SNN systems. Our experiment results show that Wenquxing 22A improves the energy expenses by 5.13 times over the accelerator solution, ODIN, with approximately classification accuracy, 85.00% for 3-bit ODIN online learning, and 91.91% for 1-bit Wenquxing 22A.注释
6 pages, 5 figures, conference or other essential info ACM-class: B.2.0邮件日期
2022年10月04日- A Novel Explainable Out-of-Distribution Detection Approach for Spiking Neural Networks 时间：2022年09月30日 第一作者：Aitor Martinez Seras 链接.
摘要：由于与传统神经网络相比，脉冲神经网络的优势，包括其高效处理和建模复杂时间动态的固有能力，过去几年中，围绕脉冲神经网的研究已经开始。尽管存在这些差异，但当在现实环境中部署时，Spiking神经网络面临着与其他神经计算同行类似的问题。这项工作解决了可能阻碍这一系列模型可信度的实际情况之一：使用远离其训练数据分布的样本（也称为分布外或OoD数据）查询训练模型的可能性。具体地说，这项工作提出了一种新的OoD检测器，它可以识别输入到脉冲神经网络的测试示例是否属于训练数据的分布。为此，我们以脉冲计数模式的形式描述了网络隐藏层的内部激活
英文摘要
Research around Spiking Neural Networks has ignited during the last years due to their advantages when compared to traditional neural networks, including their efficient processing and inherent ability to model complex temporal dynamics. Despite these differences, Spiking Neural Networks face similar issues than other neural computation counterparts when deployed in real-world settings. This work addresses one of the practical circumstances that can hinder the trustworthiness of this family of models: the possibility of querying a trained model with samples far from the distribution of its training data (also referred to as Out-of-Distribution or OoD data). Specifically, this work presents a novel OoD detector that can identify whether test examples input to a Spiking Neural Network belong to the distribution of the data over which it was trained. For this purpose, we characterize the internal activations of the hidden layers of the network in the form of spike count patterns, which lay a basis for determining when the activations induced by a test instance is atypical. Furthermore, a local explanation method is devised to produce attribution maps revealing which parts of the input instance push most towards the detection of an example as an OoD sample. Experimental results are performed over several image classification datasets to compare the proposed detector to other OoD detection schemes from the literature. As the obtained results clearly show, the proposed detector performs competitively against such alternative schemes, and produces relevance attribution maps that conform to expectations for synthetically created OoD instances.注释
37 pages, 10 figures, under review MSC-class: 68T07 ACM-class: I.2邮件日期
2022年10月04日- Spike-based local synaptic plasticity: A survey of computational models and neuromorphic circuits 时间：2022年09月30日 第一作者：Lyes Khacef 链接.
摘要：了解生物神经网络如何利用基于脉冲的局部可塑性机制进行学习，可以开发出强大、节能和自适应的神经形态处理系统。根据不同的方法，最近提出了大量基于脉冲的学习模型。然而，很难评估它们是否以及如何映射到神经形态硬件上，也很难比较它们的特性和实现的容易性。为此，在本次调查中，我们在一个统一的框架内，全面概述了代表性的脑启发突触可塑性模型和混合信号CMOS神经形态电路。我们回顾了建模突触可塑性的历史、自下而上和自上而下方法，并确定了可以支持基于脉冲学习规则的低延迟和低功耗硬件实现的计算原语。我们提供了基于突触前和突触后神经元的局部性原则的共同定义
英文摘要
Understanding how biological neural networks carry out learning using spike-based local plasticity mechanisms can lead to the development of powerful, energy-efficient, and adaptive neuromorphic processing systems. A large number of spike-based learning models have recently been proposed following different approaches. However, it is difficult to assess if and how they could be mapped onto neuromorphic hardware, and to compare their features and ease of implementation. To this end, in this survey, we provide a comprehensive overview of representative brain-inspired synaptic plasticity models and mixed-signal \acs{CMOS} neuromorphic circuits within a unified framework. We review historical, bottom-up, and top-down approaches to modeling synaptic plasticity, and we identify computational primitives that can support low-latency and low-power hardware implementations of spike-based learning rules. We provide a common definition of a locality principle based on pre- and post-synaptic neuron information, which we propose as a fundamental requirement for physical implementations of synaptic plasticity. Based on this principle, we compare the properties of these models within the same framework, and describe the mixed-signal electronic circuits that implement their computing primitives, pointing out how these building blocks enable efficient on-chip and online learning in neuromorphic processing systems.邮件日期
2022年10月03日- Spikformer: When Spiking Neural Network Meets Transformer 时间：2022年09月29日 第一作者：Zhaokun Zhou 链接.
摘要：我们考虑两种生物学上合理的结构，脉冲神经网络（SNN）和自我注意机制。前者为深度学习提供了一种能效和事件驱动的范例，而后者能够捕获特征依赖性，使Transformer能够实现良好的性能。直觉上，探索他们之间的婚姻是有希望的。在本文中，我们考虑利用SNN的自我注意能力和生物学特性，并提出了一种新的脉冲自我注意（SSA）以及一个强大的框架，称为脉冲变压器（Spikformer）。Spikformer中的SSA机制通过使用spik-form Query、Key和Value来建模稀疏视觉特征，而不使用softmax。由于它的计算是稀疏的并且避免了乘法，因此SSA是高效的并且具有低的计算能耗。结果表明，具有SSA的Spikformer在图像分类方面，无论是在神经形态分类还是在视觉识别方面，都能优于最先进的SNN类框架
英文摘要
We consider two biologically plausible structures, the Spiking Neural Network (SNN) and the self-attention mechanism. The former offers an energy-efficient and event-driven paradigm for deep learning, while the latter has the ability to capture feature dependencies, enabling Transformer to achieve good performance. It is intuitively promising to explore the marriage between them. In this paper, we consider leveraging both self-attention capability and biological properties of SNNs, and propose a novel Spiking Self Attention (SSA) as well as a powerful framework, named Spiking Transformer (Spikformer). The SSA mechanism in Spikformer models the sparse visual feature by using spike-form Query, Key, and Value without softmax. Since its computation is sparse and avoids multiplication, SSA is efficient and has low computational energy consumption. It is shown that Spikformer with SSA can outperform the state-of-the-art SNNs-like frameworks in image classification on both neuromorphic and static datasets. Spikformer (66.3M parameters) with comparable size to SEW-ResNet-152 (60.2M,69.26%) can achieve 74.81% top1 accuracy on ImageNet using 4 time steps, which is the state-of-the-art in directly trained SNNs models.邮件日期
2022年10月03日- Evaluating the temporal understanding of neural networks on event-based action recognition with DVS-Gesture-Chain 时间：2022年09月29日 第一作者：Alex Vicente-Sola 链接.
摘要：为了实现对视频序列的完整感知，使人工神经网络（ANN）在视觉任务中具有时间理解是一项基本要求。当使用传统的基于帧的视频序列时，可以使用广泛的基准数据集来评估这种能力。相比之下，由于缺乏适当的数据集，针对神经形态数据的系统评估它们仍然是一个挑战。在这项工作中，我们为基于事件的视频序列中的动作识别定义了一个新的基准任务DVS手势链（DVS-GC），该任务基于广泛使用的DVS手势数据集中的多个手势的时间组合。这种方法允许创建在时间维度上任意复杂的数据集。使用我们新定义的任务，我们评估了不同前馈卷积神经网络和卷积脉冲神经网络（SNN）的时空理解。我们的研究证明了原始DVS
英文摘要
Enabling artificial neural networks (ANNs) to have temporal understanding in visual tasks is an essential requirement in order to achieve complete perception of video sequences. A wide range of benchmark datasets is available to allow for the evaluation of such capabilities when using conventional frame-based video sequences. In contrast, evaluating them for systems targeting neuromorphic data is still a challenge due to the lack of appropriate datasets. In this work we define a new benchmark task for action recognition in event-based video sequences, DVS-Gesture-Chain (DVS-GC), which is based on the temporal combination of multiple gestures from the widely used DVS-Gesture dataset. This methodology allows to create datasets that are arbitrarily complex in the temporal dimension. Using our newly defined task, we evaluate the spatio-temporal understanding of different feed-forward convolutional ANNs and convolutional Spiking Neural Networks (SNNs). Our study proves how the original DVS Gesture benchmark could be solved by networks without temporal understanding, unlike the new DVS-GC which demands an understanding of the ordering of events. From there, we provide a study showing how certain elements such as spiking neurons or time-dependent weights allow for temporal understanding in feed-forward networks without the need for recurrent connections. Code available at: https://github.com/VicenteAlex/DVS-Gesture-Chain邮件日期
2022年09月30日- Attention Spiking Neural Networks 时间：2022年09月28日 第一作者：Man Yao 链接.
摘要：得益于大脑的事件驱动和稀疏脉冲特性，脉冲神经网络（SNN）正成为人工神经网络（ANN）的节能替代品。然而，长期以来，SNN和ANN之间的性能差距一直是广泛部署SNN的巨大障碍。为了充分利用SNN的潜力，我们研究了SNN中注意机制的影响。我们首先用一种即插即用的工具，即多维注意力（MA）来展示我们的注意力概念。然后，提出了一种新的具有端到端训练的注意力SNN架构，称为“MA-SNN”，该架构分别或同时沿时间、信道和空间维度推断注意力权重。基于现有的神经科学理论，我们利用注意力权重来优化膜电位，从而以数据依赖的方式调节脉冲反应。以可忽略的附加参数为代价，MA促进了普通SNN的交流
英文摘要
Benefiting from the event-driven and sparse spiking characteristics of the brain, spiking neural networks (SNNs) are becoming an energy-efficient alternative to artificial neural networks (ANNs). However, the performance gap between SNNs and ANNs has been a great hindrance to deploying SNNs ubiquitously for a long time. To leverage the full potential of SNNs, we study the effect of attention mechanisms in SNNs. We first present our idea of attention with a plug-and-play kit, termed the Multi-dimensional Attention (MA). Then, a new attention SNN architecture with end-to-end training called "MA-SNN" is proposed, which infers attention weights along the temporal, channel, as well as spatial dimensions separately or simultaneously. Based on the existing neuroscience theories, we exploit the attention weights to optimize membrane potentials, which in turn regulate the spiking response in a data-dependent way. At the cost of negligible additional parameters, MA facilitates vanilla SNNs to achieve sparser spiking activity, better performance, and energy efficiency concurrently. Experiments are conducted in event-based DVS128 Gesture/Gait action recognition and ImageNet-1k image classification. On Gesture/Gait, the spike counts are reduced by 84.9%/81.6%, and the task accuracy and energy efficiency are improved by 5.9%/4.7% and 3.4$\times$/3.2$\times$. On ImageNet-1K, we achieve top-1 accuracy of 75.92% and 77.08% on single/4-step Res-SNN-104, which are state-of-the-art results in SNNs. To our best knowledge, this is for the first time, that the SNN community achieves comparable or even better performance compared with its ANN counterpart in the large-scale dataset. Our work lights up SNN's potential as a general backbone to support various applications for SNNs, with a great balance between effectiveness and efficiency.注释
18 pages, 8 figures, Under Review邮件日期
2022年09月29日- Spiking SiamFC++: Deep Spiking Neural Network for Object Tracking 时间：2022年09月24日 第一作者：Shuiying Xiang 链接.
摘要：脉冲神经网络（SNN）是一种生物学上合理的模型，具有高计算能力和低功耗的优点。然而，深度SNN的训练仍然是一个开放的问题，这限制了深度SNN在现实世界中的应用。在这里，我们提出了一种名为Spiking SiamFC++的深度SNN架构，用于通过端到端直接训练进行目标跟踪。具体地，在时域中扩展AlexNet网络以提取特征，并采用替代梯度函数来实现深度SNN的直接监督训练。为了检查脉冲SiamFC++的性能，考虑了几个跟踪基准，包括OTB2013、OTB2015、VOT2015、VOT2016和UAV123。结果表明，与原始SiamFC++相比，精度损失较小。与现有的基于SNN的目标跟踪器（例如SiamSNN）相比，所提出的脉冲SiamFC++的精度（连续性）达到85.24%（64.37%），远高于52.78%(
英文摘要
Spiking neural network (SNN) is a biologically-plausible model and exhibits advantages of high computational capability and low power consumption. While the training of deep SNN is still an open problem, which limits the real-world applications of deep SNN. Here we propose a deep SNN architecture named Spiking SiamFC++ for object tracking with end-to-end direct training. Specifically, the AlexNet network is extended in the time domain to extract the feature, and the surrogate gradient function is adopted to realize direct supervised training of the deep SNN. To examine the performance of the Spiking SiamFC++, several tracking benchmarks including OTB2013, OTB2015, VOT2015, VOT2016, and UAV123 are considered. It is found that, the precision loss is small compared with the original SiamFC++. Compared with the existing SNN-based target tracker, e.g., the SiamSNN, the precision (succession) of the proposed Spiking SiamFC++ reaches 85.24% (64.37%), which is much higher than that of 52.78% (44.32%) achieved by the SiamSNN. To our best knowledge, the performance of the Spiking SiamFC++ outperforms the existing state-of-the-art approaches in SNN-based object tracking, which provides a novel path for SNN application in the field of target tracking. This work may further promote the development of SNN algorithms and neuromorphic chips.邮件日期
2022年09月27日- Physically constrained neural networks to solve the inverse problem for neuron models 时间：2022年09月24日 第一作者：Matteo Ferrante 链接.
摘要：特别是系统生物学和系统神经生理学最近已成为生物医学科学中许多关键应用的有力工具。尽管如此，此类模型通常基于多尺度（以及可能的多物理）策略的复杂组合，这些策略需要特殊的计算策略，并提出极高的计算要求。深度神经网络领域的最新发展表明，与传统模型相比，建立非线性、通用逼近器来估计高度非线性和复杂问题的解具有显著的速度和精度优势。在合成数据验证后，我们使用所谓的物理约束神经网络（PINN）来同时求解生物学上合理的霍奇金-赫胥黎模型，并在可变和恒定电流刺激下从真实数据中推断其参数和隐藏时间过程，证明了脉冲的变异性极低
英文摘要
Systems biology and systems neurophysiology in particular have recently emerged as powerful tools for a number of key applications in the biomedical sciences. Nevertheless, such models are often based on complex combinations of multiscale (and possibly multiphysics) strategies that require ad hoc computational strategies and pose extremely high computational demands. Recent developments in the field of deep neural networks have demonstrated the possibility of formulating nonlinear, universal approximators to estimate solutions to highly nonlinear and complex problems with significant speed and accuracy advantages in comparison with traditional models. After synthetic data validation, we use so-called physically constrained neural networks (PINN) to simultaneously solve the biologically plausible Hodgkin-Huxley model and infer its parameters and hidden time-courses from real data under both variable and constant current stimulation, demonstrating extremely low variability across spikes and faithful signal reconstruction. The parameter ranges we obtain are also compatible with prior knowledge. We demonstrate that detailed biological knowledge can be provided to a neural network, making it able to fit complex dynamics over both simulated and real data.邮件日期
2022年09月27日- Neuromorphic Integrated Sensing and Communications 时间：2022年09月24日 第一作者：Jiechen Chen 链接.
摘要：神经形态计算是一种新兴技术，它为需要高效在线推理和/或控制的应用程序支持事件驱动的数据处理。最近的工作引入了神经形态通信的概念，其中神经形态计算与脉冲无线电（IR）传输集成，以在无线物联网网络中实现低能量和低延迟的远程推断。在本文中，我们介绍了神经形态集成传感和通信（N-ISAC），这是一种能够实现高效在线数据解码和雷达传感的新解决方案。N-ISAC利用公共IR波形来实现传输数字信息和检测雷达目标的存在或不存在的双重目的。在接收器处部署脉冲神经网络（SNN），以解码数字数据并直接使用接收到的信号检测雷达目标。通过平衡数据通信和雷达传感的性能指标，突出协同作用和跟踪能力，优化了SNN操作
英文摘要
Neuromorphic computing is an emerging technology that support event-driven data processing for applications requiring efficient online inference and/or control. Recent work has introduced the concept of neuromorphic communications, whereby neuromorphic computing is integrated with impulse radio (IR) transmission to implement low-energy and low-latency remote inference in wireless IoT networks. In this paper, we introduce neuromorphic integrated sensing and communications (N-ISAC), a novel solution that enables efficient online data decoding and radar sensing. N-ISAC leverages a common IR waveform for the dual purpose of conveying digital information and of detecting the presence or absence of a radar target. A spiking neural network (SNN) is deployed at the receiver to decode digital data and detect the radar target using directly the received signal. The SNN operation is optimized by balancing performance metric for data communications and radar sensing, highlighting synergies and trade-offs between the two applications.注释
Submitted邮件日期
2022年09月27日- A Spatial-channel-temporal-fused Attention for Spiking Neural Networks 时间：2022年09月22日 第一作者：Wuque Cai 链接.
摘要：脉冲神经网络（SNN）模仿大脑的计算策略，并在时空信息处理方面表现出强大的能力。视觉注意作为人类感知的一个基本因素，是指生物视觉系统中显著区域的动态选择过程。尽管视觉注意机制在计算机视觉中取得了巨大成功，但很少被引入SNN。受预测性注意力重映射的实验观察启发，我们在此提出了一种新的空间通道-时间融合注意力（SCTFA）模块，该模块可以引导SNN通过利用历史积累的空间通道信息来有效捕获潜在的目标区域。通过对三个事件流数据集（DVS手势、SL动物DVS和MNIST-DVS）的系统评估，我们证明了具有SCTFA模块的SNN（SCTFA-SNN）不仅显著优于基线SNN（BL-SNN）和具有退化注意力模型的其他两个SNN模型
英文摘要
Spiking neural networks (SNNs) mimic brain computational strategies, and exhibit substantial capabilities in spatiotemporal information processing. As an essential factor for human perception, visual attention refers to the dynamic selection process of salient regions in biological vision systems. Although mechanisms of visual attention have achieved great success in computer vision, they are rarely introduced into SNNs. Inspired by experimental observations on predictive attentional remapping, we here propose a new spatial-channel-temporal-fused attention (SCTFA) module that can guide SNNs to efficiently capture underlying target regions by utilizing historically accumulated spatial-channel information. Through a systematic evaluation on three event stream datasets (DVS Gesture, SL-Animals-DVS and MNIST-DVS), we demonstrate that the SNN with the SCTFA module (SCTFA-SNN) not only significantly outperforms the baseline SNN (BL-SNN) and other two SNN models with degenerated attention modules, but also achieves competitive accuracy with existing state-of-the-art methods. Additionally, our detailed analysis shows that the proposed SCTFA-SNN model has strong robustness to noise and outstanding stability to incomplete data, while maintaining acceptable complexity and efficiency. Overall, these findings indicate that appropriately incorporating cognitive mechanisms of the brain may provide a promising approach to elevate the capability of SNNs.注释
12 pages, 8 figures, 5 tabes; This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible邮件日期
2022年09月23日- Population-coding and Dynamic-neurons improved Spiking Actor Network for Reinforcement Learning 时间：2022年09月22日 第一作者：Duzhen Zhang 链接.
邮件日期
2022年09月23日- Adaptive-SpikeNet: Event-based Optical Flow Estimation using Spiking Neural Networks with Learnable Neuronal Dynamics 时间：2022年09月21日 第一作者：Adarsh Kumar Kosta 链接.
摘要：基于事件的摄像机由于其异步捕获时间丰富的信息的能力，最近显示出高速运动估计的巨大潜力。脉冲神经网络（SNN）以其神经启发的事件驱动处理可以有效地处理此类异步数据，而泄漏积分和激发（LIF）等神经元模型可以跟踪输入中包含的典型时序信息。SNN通过在神经元记忆中保持动态状态，保留重要信息，同时随着时间的推移忘记冗余数据来实现这一点。因此，我们假设，与类似大小的模拟神经网络（ANN）相比，SNN将允许在序列回归任务中具有更好的性能。然而，深层SNN很难训练，因为后期的脉冲会消失。为此，我们提出了一种具有可学习神经元动力学的自适应完全脉冲框架，以缓解脉冲消失问题。我们使用基于替代梯度的backpro
英文摘要
Event-based cameras have recently shown great potential for high-speed motion estimation owing to their ability to capture temporally rich information asynchronously. Spiking Neural Networks (SNNs), with their neuro-inspired event-driven processing can efficiently handle such asynchronous data, while neuron models such as the leaky-integrate and fire (LIF) can keep track of the quintessential timing information contained in the inputs. SNNs achieve this by maintaining a dynamic state in the neuron memory, retaining important information while forgetting redundant data over time. Thus, we posit that SNNs would allow for better performance on sequential regression tasks compared to similarly sized Analog Neural Networks (ANNs). However, deep SNNs are difficult to train due to vanishing spikes at later layers. To that effect, we propose an adaptive fully-spiking framework with learnable neuronal dynamics to alleviate the spike vanishing problem. We utilize surrogate gradient-based backpropagation through time (BPTT) to train our deep SNNs from scratch. We validate our approach for the task of optical flow estimation on the Multi-Vehicle Stereo Event-Camera (MVSEC) dataset and the DSEC-Flow dataset. Our experiments on these datasets show an average reduction of 13% in average endpoint error (AEE) compared to state-of-the-art ANNs. We also explore several down-scaled models and observe that our SNN models consistently outperform similarly sized ANNs offering 10%-16% lower AEE. These results demonstrate the importance of SNNs for smaller models and their suitability at the edge. In terms of efficiency, our SNNs offer substantial savings in network parameters (48x) and computational energy (51x) while attaining ~10% lower EPE compared to the state-of-the-art ANN implementations.邮件日期
2022年09月26日- Learning Sparse Latent Representations for Generator Model 时间：2022年09月20日 第一作者：Hanao Li 链接.
摘要：稀疏是一个理想的属性。与密集模型相比，它可以导致更高效和更有效的表示。同时，由于稀疏潜在表示的复杂性，学习稀疏潜在表示一直是计算机视觉和机器学习领域的一个挑战性问题。在本文中，我们提出了一种新的无监督学习方法，以增强生成器模型的潜在空间上的稀疏性，并将逐渐稀疏的脉冲和板条分布作为我们的先验。我们的模型仅由一个自上而下的生成器网络组成，它将潜在变量映射到观测数据。使用基于非持久梯度的方法，可以根据生成器后向推断潜在变量。推理步骤中的Spike和Slab正则化可以将非信息潜在维度推向零，从而导致稀疏性。广泛的实验表明，该模型可以保留原始图像中的大部分信息，并具有稀疏表示，同时显示了改进的结果
英文摘要
Sparsity is a desirable attribute. It can lead to more efficient and more effective representations compared to the dense model. Meanwhile, learning sparse latent representations has been a challenging problem in the field of computer vision and machine learning due to its complexity. In this paper, we present a new unsupervised learning method to enforce sparsity on the latent space for the generator model with a gradually sparsified spike and slab distribution as our prior. Our model consists of only one top-down generator network that maps the latent variable to the observed data. Latent variables can be inferred following generator posterior direction using non-persistent gradient based method. Spike and Slab regularization in the inference step can push non-informative latent dimensions towards zero to induce sparsity. Extensive experiments show the model can preserve majority of the information from original images with sparse representations while demonstrating improved results compared to other existing methods. We observe that our model can learn disentangled semantics and increase explainability of the latent codes while boosting the robustness in the task of classification and denoising.邮件日期
2022年09月22日- A Spiking Neural Network Learning Markov Chain 时间：2022年09月20日 第一作者：Mikhail Kiselev 链接.
摘要：本文探讨了脉冲神经网络（SNN）如何在其内部结构中学习和固定外部世界动力学模型的问题。这个问题对于基于模型的强化学习（RL）的实施很重要，这是一种现实的强化学习机制，在该机制中，SNN做出的决策及其在奖惩信号方面的评估可能会被重要的时间间隔和中间评估中立世界状态的序列分开。在目前的工作中，我将世界动力学形式化为具有未知先验状态转移概率的马尔可夫链，这应该由网络学习。为了使这个问题公式更现实，我在连续时间内解决它，这样马尔可夫链中每个状态的持续时间可能是不同的，并且是未知的。它证明了如何通过具有特殊设计结构和局部突触可塑性规则的SNN来完成这一任务。作为一个例子，我们展示了这个网络主题如何在简单但没有
英文摘要
In this paper, the question how spiking neural network (SNN) learns and fixes in its internal structures a model of external world dynamics is explored. This question is important for implementation of the model-based reinforcement learning (RL), the realistic RL regime where the decisions made by SNN and their evaluation in terms of reward/punishment signals may be separated by significant time interval and sequence of intermediate evaluation-neutral world states. In the present work, I formalize world dynamics as a Markov chain with unknown a priori state transition probabilities, which should be learnt by the network. To make this problem formulation more realistic, I solve it in continuous time, so that duration of every state in the Markov chain may be different and is unknown. It is demonstrated how this task can be accomplished by an SNN with specially designed structure and local synaptic plasticity rules. As an example, we show how this network motif works in the simple but non-trivial world where a ball moves inside a square box and bounces from its walls with a random new direction and velocity.邮件日期
2022年09月21日- A Synapse-Threshold Synergistic Learning Approach for Spiking Neural Networks 时间：2022年09月20日 第一作者：Hongze Sun 链接.
注释
13 pages, 9 figures, submitted for publication邮件日期
2022年09月21日- Ensembles of Compact, Region-specific & Regularized Spiking Neural Networks for Scalable Place Recognition 时间：2022年09月19日 第一作者：Somayeh Hussaini 链接.
摘要：脉冲神经网络由于其在专用硬件上的高能效，在机器人技术中具有重要的潜在用途，但概念验证实现通常还没有实现与传统方法相比具有竞争力的性能或能力。在本文中，我们通过引入一种新的模块化集成网络方法来解决可扩展性的关键实际挑战之一，在这种方法中，紧凑的局部脉冲网络每个都学习并仅负责识别环境的局部区域中的位置。这种模块化方法创建了一个高度可扩展的系统。然而，它带来了高性能的代价，在部署时缺乏全局规则化会导致过度活跃的神经元错误地对学习区域以外的地方做出反应。我们的第二个贡献介绍了一种正则化方法，在初始环境学习阶段检测并去除这些有问题的过度活跃神经元。我们评估了这个新的sca
英文摘要
Spiking neural networks have significant potential utility in robotics due to their high energy efficiency on specialized hardware, but proof-of-concept implementations have not yet typically achieved competitive performance or capability with conventional approaches. In this paper, we tackle one of the key practical challenges of scalability by introducing a novel modular ensemble network approach, where compact, localized spiking networks each learn and are solely responsible for recognizing places in a local region of the environment only. This modular approach creates a highly scalable system. However, it comes with a high-performance cost where a lack of global regularization at deployment time leads to hyperactive neurons that erroneously respond to places outside their learned region. Our second contribution introduces a regularization approach that detects and removes these problematic hyperactive neurons during the initial environmental learning phase. We evaluate this new scalable modular system on benchmark localization datasets Nordland and Oxford RobotCar, with comparisons to both standard techniques NetVLAD and SAD, and a previous spiking neural network system. Our system substantially outperforms the previous SNN system on its small dataset, but also maintains performance on 27 times larger benchmark datasets where the operation of the previous system is computationally infeasible, and performs competitively with the conventional localization systems.注释
8 pages, 6 figures, under review邮件日期
2022年09月20日- SpikeSEE: An Energy-Efficient Dynamic Scenes Processing Framework for Retinal Prostheses 时间：2022年09月16日 第一作者：Chuanqing Wang 链接.
摘要：在这个时代，智能和低功耗的视网膜假体被高度要求，可穿戴和可植入设备被用于许多医疗应用。在本文中，我们提出了一种节能的动态场景处理框架（SpikeSEE），该框架结合了脉冲表示编码技术和仿生脉冲递归神经网络（SRNN）模型，以实现视网膜假体的智能处理和极低功耗计算。脉冲表示编码技术可以解释具有稀疏脉冲序列的动态场景，从而减少数据量。受人类视网膜特殊结构和棘波处理方法启发，采用SRNN模型预测神经节细胞对动态场景的反应。实验结果表明，所提出的SRNN模型的皮尔逊相关系数达到0.93，这优于视网膜假体的现有技术处理框架。由于脉冲表示和SRNN
英文摘要
Intelligent and low-power retinal prostheses are highly demanded in this era, where wearable and implantable devices are used for numerous healthcare applications. In this paper, we propose an energy-efficient dynamic scenes processing framework (SpikeSEE) that combines a spike representation encoding technique and a bio-inspired spiking recurrent neural network (SRNN) model to achieve intelligent processing and extreme low-power computation for retinal prostheses. The spike representation encoding technique could interpret dynamic scenes with sparse spike trains, decreasing the data volume. The SRNN model, inspired by the human retina special structure and spike processing method, is adopted to predict the response of ganglion cells to dynamic scenes. Experimental results show that the Pearson correlation coefficient of the proposed SRNN model achieves 0.93, which outperforms the state of the art processing framework for retinal prostheses. Thanks to the spike representation and SRNN processing, the model can extract visual features in a multiplication-free fashion. The framework achieves 12 times power reduction compared with the convolutional recurrent neural network (CRNN) processing-based framework. Our proposed SpikeSEE predicts the response of ganglion cells more accurately with lower energy consumption, which alleviates the precision and power issues of retinal prostheses and provides a potential solution for wearable or implantable prostheses.邮件日期
2022年09月19日- Astromorphic Self-Repair of Neuromorphic Hardware Systems 时间：2022年09月15日 第一作者：Zhuangyu Han 链接.
摘要：尽管基于脉冲神经网络（SNN）的神经形态计算体系结构越来越受到人们的关注，作为一种通往生物似是而非的机器学习的途径，但人们的注意力仍然集中在神经元和突触等计算单元上。从神经突触的角度出发，本文试图探索神经胶质细胞，特别是星形胶质细胞的自我修复作用。这项工作研究了与星形胶质细胞计算神经科学模型的更强相关性，以开发具有更高生物保真度的宏观模型，准确捕捉自我修复过程的动态行为。硬件-软件协同设计分析表明，生物形态星形细胞调节具有自我修复神经形态硬件系统中硬件现实故障的潜力，对于MNIST和F-MNIST数据集上的无监督学习任务，具有显著更好的准确性和修复收敛性。
英文摘要
While neuromorphic computing architectures based on Spiking Neural Networks (SNNs) are increasingly gaining interest as a pathway toward bio-plausible machine learning, attention is still focused on computational units like the neuron and synapse. Shifting from this neuro-synaptic perspective, this paper attempts to explore the self-repair role of glial cells, in particular, astrocytes. The work investigates stronger correlations with astrocyte computational neuroscience models to develop macro-models with a higher degree of bio-fidelity that accurately captures the dynamic behavior of the self-repair process. Hardware-software co-design analysis reveals that bio-morphic astrocytic regulation has the potential to self-repair hardware realistic faults in neuromorphic hardware systems with significantly better accuracy and repair convergence for unsupervised learning tasks on the MNIST and F-MNIST datasets.邮件日期
2022年09月16日- Sketch of a novel approach to a neural model 时间：2022年09月14日 第一作者：Gabriele Scheler 链接.
摘要：在本文中，我们提出了一种新的神经可塑性模型，其形式是神经加工的水平-垂直集成模型。我们相信，一种新的神经建模方法将有利于第三波人工智能。水平面由一个由传输链路连接的自适应神经元网络组成，该网络产生时空脉冲模式。这符合标准的计算神经科学方法。此外，对于每个单独的神经元，都有一个由内部自适应参数组成的垂直部分，该内部自适应参数控制与神经传递有关的外部膜表达参数。每个神经元都有一个垂直的模块化参数系统，这些参数对应于（a）膜层的外部参数，分为隔室（棘、骨）（b）膜下带和细胞质中的内部参数及其蛋白信号网络，以及（c）细胞核中的遗传和表观遗传信息的核心参数。在这种模型中，每个节点（=神经
英文摘要
In this paper, we lay out a novel model of neuroplasticity in the form of a horizontal-vertical integration model of neural processing. We believe a new approach to neural modeling will benefit the 3rd wave of AI. The horizontal plane consists of an adaptive network of neurons connected by transmission links which generates spatio-temporal spike patterns. This fits with standard computational neuroscience approaches. Additionally for each individual neuron there is a vertical part consisting of internal adaptive parameters steering the external membrane-expressed parameters which are involved in neural transmission. Each neuron has a vertical modular system of parameters corresponding to (a) external parameters at the membrane layer, divided into compartments (spines, boutons) (b) internal parameters in the submembrane zone and the cytoplasm with its protein signaling network and (c) core parameters in the nucleus for genetic and epigenetic information. In such models, each node (=neuron) in the horizontal network has its own internal memory. Neural transmission and information storage are systematically separated, an important conceptual advance over synaptic weight models. We discuss the membrane-based (external) filtering and selection of outside signals for processing vs. signal loss by fast fluctuations and the neuron-internal computing strategies from intracellular protein signaling to the nucleus as the core system. We want to show that the individual neuron has an important role in the computation of signals and that many assumptions derived from the synaptic weight adjustment hypothesis of memory may not hold in a real brain. Not every transmission event leaves a trace and the neuron is a self-programming device, rather than passively determined by current input. Ultimately we strive to build a flexible memory system that processes facts and events automatically.邮件日期
2022年09月16日- Spiking GATs: Learning Graph Attentions via Spiking Neural Network 时间：2022年09月05日 第一作者：Beibei Wang 链接.
摘要：图形注意力网络（GAT）已被深入研究并广泛应用于图形数据学习任务中。现有的GAT通常采用自注意机制来进行图边缘注意学习，需要昂贵的计算。已知脉冲神经网络（SNN）可以通过将输入信号数据传输成离散脉冲序列来执行廉价的计算，并且还可以返回稀疏输出。受SNN优点的启发，在这项工作中，我们提出了一种用于图数据表示和学习的新型图脉冲注意力网络（GSAT）。与现有GAT中的自我关注机制相比，所提出的GSAT采用了明显节能的SNN模块架构。此外，GSAT可以返回自然的稀疏关注系数，从而可以对选择性邻居执行特征聚合，这使得GSAT能够针对图边缘噪声执行鲁棒性。在多个数据集上的实验结果表明
英文摘要
Graph Attention Networks (GATs) have been intensively studied and widely used in graph data learning tasks. Existing GATs generally adopt the self-attention mechanism to conduct graph edge attention learning, requiring expensive computation. It is known that Spiking Neural Networks (SNNs) can perform inexpensive computation by transmitting the input signal data into discrete spike trains and can also return sparse outputs. Inspired by the merits of SNNs, in this work, we propose a novel Graph Spiking Attention Network (GSAT) for graph data representation and learning. In contrast to self-attention mechanism in existing GATs, the proposed GSAT adopts a SNN module architecture which is obvious energy-efficient. Moreover, GSAT can return sparse attention coefficients in natural and thus can perform feature aggregation on the selective neighbors which makes GSAT perform robustly w.r.t graph edge noises. Experimental results on several datasets demonstrate the effectiveness, energy efficiency and robustness of the proposed GSAT model.邮件日期
2022年09月28日- Data Augmentation in Temporal and Polar Domains for Event-Based Learning 时间：2022年07月24日 第一作者：Haibo Shen 链接.
注释
7+2pages, 6figures License: http://creativecommons.org/licenses/by/4.0/邮件日期
2022年09月14日- Data Augmentation in Temporal and Polar Domains for Event-Based Learning 时间：2022年07月24日 第一作者：Haibo Shen 链接.
注释
7+2pages, 6figures License: http://creativecommons.org/licenses/by/4.0/邮件日期
2022年09月13日- Securing the Spike: On the Transferabilty and Security of Spiking Neural Networks to Adversarial Examples 时间：2022年09月07日 第一作者：Nuo Xu 链接.
摘要：脉冲神经网络（SNN）由于其高能量效率和分类性能的最新进展而引起了广泛关注。然而，与传统的深度学习方法不同，SNN对对抗性示例的鲁棒性的分析和研究仍然相对不足。在这项工作中，我们通过实验和分析三个重要的SNN安全属性，推进了对抗性机器学习领域。首先，我们表明，对SNN的成功白盒对抗性攻击高度依赖于潜在的替代梯度技术。其次，我们分析了SNN和其他最先进的架构（如视觉变换器和大传输CNN）生成的对抗性示例的可传输性。我们证明，SNN通常不会被视觉变换器和某些类型的CNN生成的对抗性示例所欺骗。最后，我们开发了一种新的白盒攻击，它可以生成能够愚弄的对手示例
英文摘要
Spiking neural networks (SNNs) have attracted much attention for their high energy efficiency and for recent advances in their classification performance. However, unlike traditional deep learning approaches, the analysis and study of the robustness of SNNs to adversarial examples remains relatively underdeveloped. In this work we advance the field of adversarial machine learning through experimentation and analyses of three important SNN security attributes. First, we show that successful white-box adversarial attacks on SNNs are highly dependent on the underlying surrogate gradient technique. Second, we analyze the transferability of adversarial examples generated by SNNs and other state-of-the-art architectures like Vision Transformers and Big Transfer CNNs. We demonstrate that SNNs are not often deceived by adversarial examples generated by Vision Transformers and certain types of CNNs. Lastly, we develop a novel white-box attack that generates adversarial examples capable of fooling both SNN models and non-SNN models simultaneously. Our experiments and analyses are broad and rigorous covering two datasets (CIFAR-10 and CIFAR-100), five different white-box attacks and twelve different classifier models.邮件日期
2022年09月09日- Making a Spiking Net Work: Robust brain-like unsupervised machine learning 时间：2022年09月01日 第一作者：Peter G. Stratton 链接.
注释
12 pages (manuscript), 5 figures, 10 pages (appendix), 11 pages (extended data)邮件日期
2022年09月02日- Bayesian Continual Learning via Spiking Neural Networks 时间：2022年08月29日 第一作者：Nicolas Skatchkovsky 链接.
摘要：生物智能的主要特征包括能源效率、持续适应能力以及通过不确定性量化进行风险管理。迄今为止，神经形态工程主要是由实现节能机器的目标驱动的，这些机器的灵感来自生物大脑的基于时间的计算范式。在本文中，我们采取步骤设计能够适应变化学习任务的神经形态系统，同时产生校准良好的不确定性量化估计。为此，我们在贝叶斯连续学习框架内导出了脉冲神经网络（SNN）的在线学习规则。在该模型中，每个突触权重由参数表示，这些参数量化了由先验知识和观察数据产生的当前认知不确定性。所提出的在线规则在观察到数据时以流方式更新分布参数。我们为实际值和
英文摘要
Among the main features of biological intelligence are energy efficiency, capacity for continual adaptation, and risk management via uncertainty quantification. Neuromorphic engineering has been thus far mostly driven by the goal of implementing energy-efficient machines that take inspiration from the time-based computing paradigm of biological brains. In this paper, we take steps towards the design of neuromorphic systems that are capable of adaptation to changing learning tasks, while producing well-calibrated uncertainty quantification estimates. To this end, we derive online learning rules for spiking neural networks (SNNs) within a Bayesian continual learning framework. In it, each synaptic weight is represented by parameters that quantify the current epistemic uncertainty resulting from prior knowledge and observed data. The proposed online rules update the distribution parameters in a streaming fashion as data are observed. We instantiate the proposed approach for both real-valued and binary synaptic weights. Experimental results using Intel's Lava platform show the merits of Bayesian over frequentist learning in terms of capacity for adaptation and uncertainty quantification.注释
Submitted for journal publication邮件日期
2022年08月30日- Uncertainty Guided Depth Fusion for Spike Camera 时间：2022年08月29日 第一作者：Jianing Li 链接.
注释
18 pages, 11 figures ACM-class: I.2.10邮件日期
2022年08月30日- Scalable Nanophotonic-Electronic Spiking Neural Networks 时间：2022年08月28日 第一作者：Luis El Srouji 链接.
摘要：脉冲神经网络（SNN）提供了一种能够高度并行化、实时处理的新计算范式。光子器件是设计与SNN计算范式相匹配的高带宽并行架构的理想选择。CMOS和光子元件的共集成允许低损耗光子器件与模拟电子器件相结合，以提高非线性计算元件的灵活性。因此，我们设计并模拟了单片硅光子学（SiPh）工艺上的光电脉冲神经元电路，该工艺复制了漏积分和激发（LIF）之外的有用脉冲行为。此外，我们还探索了两种具有片上学习潜力的学习算法，使用Mach-Zehnder干涉（MZI）网格作为突触互连。随机反向传播（RPB）的变化在芯片上进行了实验演示，并与简单分类任务的标准线性回归的性能相匹配。同时，对比
英文摘要
Spiking neural networks (SNN) provide a new computational paradigm capable of highly parallelized, real-time processing. Photonic devices are ideal for the design of high-bandwidth, parallel architectures matching the SNN computational paradigm. Co-integration of CMOS and photonic elements allow low-loss photonic devices to be combined with analog electronics for greater flexibility of nonlinear computational elements. As such, we designed and simulated an optoelectronic spiking neuron circuit on a monolithic silicon photonics (SiPh) process that replicates useful spiking behaviors beyond the leaky integrate-and-fire (LIF). Additionally, we explored two learning algorithms with the potential for on-chip learning using Mach-Zehnder Interferometric (MZI) meshes as synaptic interconnects. A variation of Random Backpropagation (RPB) was experimentally demonstrated on-chip and matched the performance of a standard linear regression on a simple classification task. Meanwhile, the Contrastive Hebbian Learning (CHL) rule was applied to a simulated neural network composed of MZI meshes for a random input-output mapping task. The CHL-trained MZI network performed better than random guessing but does not match the performance of the ideal neural network (without the constraints imposed by the MZI meshes). Through these efforts, we demonstrate that co-integrated CMOS and SiPh technologies are well-suited to the design of scalable SNN computing architectures.邮件日期
2022年08月30日- Sub-mW Neuromorphic SNN audio processing applications with Rockpool and Xylo 时间：2022年08月27日 第一作者：Hannah Bos 链接.
摘要：脉冲神经网络（SNN）为时间信号处理提供了一种有效的计算机制，特别是当与低功率SNN推理ASIC耦合时。SNN在历史上很难配置，缺乏为任意任务找到解决方案的通用方法。近年来，梯度下降优化方法越来越容易地应用于SNN。因此，SNN和SNN推理处理器为能源受限环境中的商业低功耗信号处理提供了良好的平台，而无需依赖云。然而，到目前为止，行业中的ML工程师还无法使用这些方法，需要研究生级别的培训才能成功配置单个SNN应用程序。在这里，我们演示了一种方便的高级流水线，用于设计、训练和部署任意时间信号处理应用程序到子mW SNN推理硬件。我们采用了一种新的直接SNN架构，用于时间信号处理
英文摘要
Spiking Neural Networks (SNNs) provide an efficient computational mechanism for temporal signal processing, especially when coupled with low-power SNN inference ASICs. SNNs have been historically difficult to configure, lacking a general method for finding solutions for arbitrary tasks. In recent years, gradient-descent optimization methods have been applied to SNNs with increasing ease. SNNs and SNN inference processors therefore offer a good platform for commercial low-power signal processing in energy constrained environments without cloud dependencies. However, to date these methods have not been accessible to ML engineers in industry, requiring graduate-level training to successfully configure a single SNN application. Here we demonstrate a convenient high-level pipeline to design, train and deploy arbitrary temporal signal processing applications to sub-mW SNN inference hardware. We apply a new straightforward SNN architecture designed for temporal signal processing, using a pyramid of synaptic time constants to extract signal features at a range of temporal scales. We demonstrate this architecture on an ambient audio classification task, deployed to the Xylo SNN inference processor in streaming mode. Our application achieves high accuracy (98%) and low latency (100ms) at low power (<4muW inference power). Our approach makes training and deploying SNN applications available to ML engineers with general NN backgrounds, without requiring specific prior experience with spiking NNs. We intend for our approach to make Neuromorphic hardware and SNNs an attractive choice for commercial low-power and edge signal processing applications.邮件日期
2022年08月30日- Neuromorphic Visual Scene Understanding with Resonator Networks 时间：2022年08月26日 第一作者：Alpha Renner 链接.
摘要：推断对象的位置及其刚性变换仍然是视觉场景理解中的一个开放问题。在这里，我们提出了一种神经形态解，该解利用基于三个关键概念的有效因子分解网络：（1）基于具有复值向量的向量符号架构（VSA）的计算框架；（2） 设计分层谐振器网络（HRN），以处理视觉场景中平移和旋转的非交换性质，当两者结合使用时；（3） 用于在神经形态硬件上实现复值向量绑定的多室脉冲相量神经元模型的设计。VSA框架使用向量绑定操作生成生成图像模型，其中绑定充当几何变换的等变操作。因此，场景可以被描述为向量乘积的和，而向量乘积又可以被谐振器网络有效地分解以推断对象及其姿态。
英文摘要
Inferring the position of objects and their rigid transformations is still an open problem in visual scene understanding. Here we propose a neuromorphic solution that utilizes an efficient factorization network which is based on three key concepts: (1) a computational framework based on Vector Symbolic Architectures (VSA) with complex-valued vectors; (2) the design of Hierarchical Resonator Networks (HRN) to deal with the non-commutative nature of translation and rotation in visual scenes, when both are used in combination; (3) the design of a multi-compartment spiking phasor neuron model for implementing complex-valued vector binding on neuromorphic hardware. The VSA framework uses vector binding operations to produce generative image models in which binding acts as the equivariant operation for geometric transformations. A scene can therefore be described as a sum of vector products, which in turn can be efficiently factorized by a resonator network to infer objects and their poses. The HRN enables the definition of a partitioned architecture in which vector binding is equivariant for horizontal and vertical translation within one partition, and for rotation and scaling within the other partition. The spiking neuron model allows to map the resonator network onto efficient and low-power neuromorphic hardware. In this work, we demonstrate our approach using synthetic scenes composed of simple 2D shapes undergoing rigid geometric transformations and color changes. A companion paper demonstrates this approach in real-world application scenarios for machine vision and robotics.注释
15 pages, 6 figures ACM-class: I.4.8邮件日期
2022年08月30日- Unsupervised Spike Depth Estimation via Cross-modality Cross-domain Knowledge Transfer 时间：2022年08月26日 第一作者：Jiaming Liu 链接.
摘要：神经形态spike摄像机以生物启发的方式生成具有高时间分辨率的数据流，在自动驾驶等现实世界应用中具有巨大潜力。与RGB流相比，脉冲流具有克服运动模糊的固有优势，从而为高速对象提供更精确的深度估计。然而，以有监督的方式训练脉冲深度估计网络几乎是不可能的，因为对于时间密集的脉冲流获得成对的深度标签是极其困难和具有挑战性的。在本文中，我们从开源RGB数据集（如KITTI）转移知识，并以无监督的方式估计脉冲深度，而不是构建具有全深度标签的脉冲流数据集。这类问题的关键挑战在于RGB和棘波模式之间的模态间隙，以及标记源RGB和未标记目标棘波域之间的域间隙。为了克服这些挑战，我们引入了cross-m
英文摘要
The neuromorphic spike camera generates data streams with high temporal resolution in a bio-inspired way, which has vast potential in the real-world applications such as autonomous driving. In contrast to RGB streams, spike streams have an inherent advantage to overcome motion blur, leading to more accurate depth estimation for high-velocity objects. However, training the spike depth estimation network in a supervised manner is almost impossible since it is extremely laborious and challenging to obtain paired depth labels for temporally intensive spike streams. In this paper, instead of building a spike stream dataset with full depth labels, we transfer knowledge from the open-source RGB datasets (e.g., KITTI) and estimate spike depth in an unsupervised manner. The key challenges for such problem lie in the modality gap between RGB and spike modalities, and the domain gap between labeled source RGB and unlabeled target spike domains. To overcome these challenges, we introduce a cross-modality cross-domain (BiCross) framework for unsupervised spike depth estimation. Our method narrows the enormous gap between source RGB and target spike by introducing the mediate simulated source spike domain. To be specific, for the cross-modality phase, we propose a novel Coarse-to-Fine Knowledge Distillation (CFKD), which transfers the image and pixel level knowledge from source RGB to source spike. Such design leverages the abundant semantic and dense temporal information of RGB and spike modalities respectively. For the cross-domain phase, we introduce the Uncertainty Guided Mean-Teacher (UGMT) to generate reliable pseudo labels with uncertainty estimation, alleviating the shift between the source spike and target spike domains. Besides, we propose a Global-Level Feature Alignment method (GLFA) to align the feature between two domains and generate more reliable pseudo labels.邮件日期
2022年08月29日- Uncertainty Guided Depth Fusion for Spike Camera 时间：2022年08月26日 第一作者：Jianing Li 链接.
摘要：深度估计对于各种重要的实际应用（如自动驾驶）至关重要。然而，由于传统摄像机只能捕获模糊图像，因此在高速场景下，它的性能会严重下降。为了解决这个问题，spike摄像机被设计为在高帧速率下捕获像素级亮度强度。然而，使用基于光度一致性的传统单目或立体深度估计算法，使用spike相机进行深度估计仍然非常具有挑战性。在本文中，我们提出了一种新的不确定性引导深度融合（UGDF）框架，用于融合单目和立体深度估计网络的预测。我们的框架是基于这样一个事实，即立体脉冲深度估计在近距离获得更好的结果，而单目脉冲深度估算在远距离获得更好的效果。因此，我们引入了一种具有联合训练的双任务深度估计架构
英文摘要
Depth estimation is essential for various important real-world applications such as autonomous driving. However, it suffers from severe performance degradation in high-velocity scenario since traditional cameras can only capture blurred images. To deal with this problem, the spike camera is designed to capture the pixel-wise luminance intensity at high frame rate. However, depth estimation with spike camera remains very challenging using traditional monocular or stereo depth estimation algorithms, which are based on the photometric consistency. In this paper, we propose a novel Uncertainty-Guided Depth Fusion (UGDF) framework to fuse the predictions of monocular and stereo depth estimation networks for spike camera. Our framework is motivated by the fact that stereo spike depth estimation achieves better results at close range while monocular spike depth estimation obtains better results at long range. Therefore, we introduce a dual-task depth estimation architecture with a joint training strategy and estimate the distributed uncertainty to fuse the monocular and stereo results. In order to demonstrate the advantage of spike depth estimation over traditional camera depth estimation, we contribute a spike-depth dataset named CitySpike20K, which contains 20K paired samples, for spike depth estimation. UGDF achieves state-of-the-art results on CitySpike20K, surpassing all monocular or stereo spike depth estimation baselines. We conduct extensive experiments to evaluate the effectiveness and generalization of our method on CitySpike20K. To the best of our knowledge, our framework is the first dual-task fusion framework for spike camera depth estimation. Code and dataset will be released.注释
18 pages, 11 figures, submitted to AAAI 2023 ACM-class: I.2.10邮件日期
2022年08月29日- CMOS-based area-and-power-efficient neuron and synapse circuits for time-domain analog spiking neural networks 时间：2022年08月25日 第一作者：Xiangyu Chen 链接.
摘要：传统的神经结构倾向于通过模拟量（例如电流或电压）进行通信，然而，随着CMOS器件的缩小和电源电压的降低，电压/电流域模拟电路的动态范围变得更窄，可用裕度变得更小，并且噪声抗扰度降低。除此之外，在传统设计中使用运算放大器（运算放大器）和时钟或异步比较器导致高能耗和大芯片面积，这将不利于建立脉冲神经网络。鉴于此，我们提出了一种用于生成和传输时域信号的神经结构，包括神经元模块、突触模块和两个权重模块。所提出的神经结构由晶体管三极管区域中的漏电流驱动，并且不使用运算放大器和比较器，因此与传统设计相比提供了更高的能量和面积效率。此外，该结构提供了更大的抗噪声能力
英文摘要
Conventional neural structures tend to communicate through analog quantities such as currents or voltages, however, as CMOS devices shrink and supply voltages decrease, the dynamic range of voltage/current-domain analog circuits becomes narrower, the available margin becomes smaller, and noise immunity decreases. More than that, the use of operational amplifiers (op-amps) and clocked or asynchronous comparators in conventional designs leads to high energy consumption and large chip area, which would be detrimental to building spiking neural networks. In view of this, we propose a neural structure for generating and transmitting time-domain signals, including a neuron module, a synapse module, and two weight modules. The proposed neural structure is driven by leakage currents in the transistor triode region and does not use op-amps and comparators, thus providing higher energy and area efficiency compared to conventional designs. In addition, the structure provides greater noise immunity due to internal communication via time-domain signals, which simplifies the wiring between the modules. The proposed neural structure is fabricated using TSMC 65 nm CMOS technology. The proposed neuron and synapse occupy an area of 127 um2 and 231 um2, respectively, while achieving millisecond time constants. Actual chip measurements show that the proposed structure successfully implements the temporal signal communication function with millisecond time constants, which is a critical step toward hardware reservoir computing for human-computer interaction.邮件日期
2022年08月26日- Human-Level Control through Directly-Trained Deep Spiking Q-Networks 时间：2022年08月25日 第一作者：Guisong Liu 链接.
邮件日期
2022年08月26日- ReckOn: A 28nm Sub-mm2 Task-Agnostic Spiking Recurrent Neural Network Processor Enabling On-Chip Learning over Second-Long Timescales 时间：2022年08月20日 第一作者：Charlotte Frenkel 链接.
摘要：自主边缘设备的强大现实部署需要片上自适应，以适应用户、环境和任务引起的变化。由于芯片内存限制，先前的学习设备仅限于没有时间内容的静态刺激。我们提出了一个0.45毫米$^2$的峰值RNN处理器，使任务无关的在线学习在几秒钟内实现，我们演示了在0.8%的内存开销和<150-$\mu$W的训练功率预算内进行导航、手势识别和关键字识别。
英文摘要
A robust real-world deployment of autonomous edge devices requires on-chip adaptation to user-, environment- and task-induced variability. Due to on-chip memory constraints, prior learning devices were limited to static stimuli with no temporal contents. We propose a 0.45-mm$^2$ spiking RNN processor enabling task-agnostic online learning over seconds, which we demonstrate for navigation, gesture recognition, and keyword spotting within a 0.8-% memory overhead and a <150-$\mu$W training power budget.注释
Published in the 2022 IEEE International Solid-State Circuits Conference (ISSCC), 2022 DOI: 10.1109/ISSCC42614.2022.9731734邮件日期
2022年08月23日- Combinatorial optimization solving by coherent Ising machines based on spiking neural networks 时间：2022年08月16日 第一作者：Bo Lu 链接.
摘要：脉冲神经网络是一种神经形态计算，被认为可以提高智能水平，为量子计算提供优势。在这项工作中，我们通过设计一个光学脉冲神经网络来解决这个问题，并证明它可以用来加快计算速度，特别是在组合优化问题上。这里，脉冲神经网络由反对称耦合简并光学参量振荡器脉冲和耗散脉冲构成。根据脉冲神经元的动态行为，选择非线性传递函数来缓解振幅不均匀性并使产生的局部极小值不稳定。数值结果表明，脉冲神经网络相干伊辛机在组合优化问题上具有优异的性能，有望为神经计算和光学计算提供新的应用。
英文摘要
Spiking neural network is a kind of neuromorphic computing which is believed to improve on the level of intelligence and provide advabtages for quantum computing. In this work, we address this issue by designing an optical spiking neural network and prove that it can be used to accelerate the speed of computation, especially on the combinatorial optimization problems. Here the spiking neural network is constructed by the antisymmetrically coupled degenerate optical parametric oscillator pulses and dissipative pulses. A nonlinear transfer function is chosen to mitigate amplitude inhomogeneities and destabilize the resulting local minima according to the dynamical behavior of spiking neurons. It is numerically proved that the spiking neural network-coherent Ising machines has excellent performance on combinatorial optimization problems, for which is expected to offer a new applications for neural computing and optical computing.注释
5 pages, 4 figures, comments are welcome邮件日期
2022年08月17日- Scaling Up Dynamic Graph Representation Learning via Spiking Neural Networks 时间：2022年08月15日 第一作者：Jintang Li 链接.
摘要：近年来，动态图表示学习的研究出现了激增，其目的是对动态且随时间不断演化的时态图建模。然而，当前的工作通常使用递归神经网络（RNN）来建模图动态，这使得它们在大型时态图上的计算和内存开销严重。到目前为止，大型时态图的动态图表示学习的可扩展性仍然是主要挑战之一。在本文中，我们提出了一个可扩展的框架，即SpikeNet，以有效地捕获时态图的时态和结构模式。我们探索了一个新的方向，即我们可以用脉冲神经网络（SNN）而不是RNN来捕捉时态图的演化动态。作为RNN的低功耗替代方案，SNN明确地将图动力学建模为神经元种群的脉冲序列，并以有效的方式实现基于脉冲的传播。三个大型实时图数据集的实验
英文摘要
Recent years have seen a surge in research on dynamic graph representation learning, which aims to model temporal graphs that are dynamic and evolving constantly over time. However, current work typically models graph dynamics with recurrent neural networks (RNNs), making them suffer seriously from computation and memory overheads on large temporal graphs. So far, scalability of dynamic graph representation learning on large temporal graphs remains one of the major challenges. In this paper, we present a scalable framework, namely SpikeNet, to efficiently capture the temporal and structural patterns of temporal graphs. We explore a new direction in that we can capture the evolving dynamics of temporal graphs with spiking neural networks (SNNs) instead of RNNs. As a low-power alternative to RNNs, SNNs explicitly model graph dynamics as spike trains of neuron populations and enable spike-based propagation in an efficient way. Experiments on three large real-world temporal graph datasets demonstrate that SpikeNet outperforms strong baselines on the temporal node classification task with lower computational costs. Particularly, SpikeNet generalizes to a large temporal graph (2M nodes and 13M edges) with significantly fewer parameters and computation overheads. Our code is publicly available at https://github.com/EdisonLeeeee/SpikeNet注释
Preprint; Code available at https://github.com/EdisonLeeeee/SpikeNet邮件日期
2022年08月23日- Encoding Integers and Rationals on Neuromorphic Computers using Virtual Neuron 时间：2022年08月15日 第一作者：Prasanna Date 链接.
摘要：神经形态计算机通过模拟人脑进行计算，并使用极低的功耗。预计它们在未来的节能计算中是不可或缺的。虽然它们主要用于基于脉冲神经网络的机器学习应用，但已知神经形态计算机是图灵完备的，因此能够进行通用计算。然而，为了充分实现其通用、节能计算的潜力，设计高效的数字编码机制非常重要。当前的编码方法具有有限的适用性，并且可能不适合于通用计算。在本文中，我们提出了虚拟神经元作为整数和有理数的编码机制。我们评估了虚拟神经元在物理和模拟神经形态硬件上的性能，并表明它可以使用基于混合信号忆阻器的神经形态过程平均使用23 nJ的能量执行加法运算
英文摘要
Neuromorphic computers perform computations by emulating the human brain, and use extremely low power. They are expected to be indispensable for energy-efficient computing in the future. While they are primarily used in spiking neural network-based machine learning applications, neuromorphic computers are known to be Turing-complete, and thus, capable of general-purpose computation. However, to fully realize their potential for general-purpose, energy-efficient computing, it is important to devise efficient mechanisms for encoding numbers. Current encoding approaches have limited applicability and may not be suitable for general-purpose computation. In this paper, we present the virtual neuron as an encoding mechanism for integers and rational numbers. We evaluate the performance of the virtual neuron on physical and simulated neuromorphic hardware and show that it can perform an addition operation using 23 nJ of energy on average using a mixed-signal memristor-based neuromorphic processor. We also demonstrate its utility by using it in some of the mu-recursive functions, which are the building blocks of general-purpose computation.邮件日期
2022年08月17日- Convolutional Spiking Neural Networks for Detecting Anticipatory Brain Potentials Using Electroencephalogram 时间：2022年08月14日 第一作者：Nathan Lutes 链接.
摘要：脉冲神经网络（SNN）作为一种开发“生物学上合理的”机器学习模型的手段，正受到越来越多的关注。这些网络模拟人脑中的突触连接并产生脉冲序列，可以用二进制值近似，从而避免了浮点运算电路的高计算成本。最近，引入了卷积层，将卷积网络的特征提取能力与SNN的计算效率结合起来。本文研究了使用卷积脉冲神经网络（CSNN）作为分类器，使用脑电图（EEG）检测与人类参与者制动意图相关的预期慢皮层电位的可行性。EEG数据是在一项实验中收集的，其中参与者在设计用于模拟城市环境的试验台上操作遥控车辆。通过音频提醒参与者即将发生的制动事件
英文摘要
Spiking neural networks (SNNs) are receiving increased attention as a means to develop "biologically plausible" machine learning models. These networks mimic synaptic connections in the human brain and produce spike trains, which can be approximated by binary values, precluding high computational cost with floating-point arithmetic circuits. Recently, the addition of convolutional layers to combine the feature extraction power of convolutional networks with the computational efficiency of SNNs has been introduced. In this paper, the feasibility of using a convolutional spiking neural network (CSNN) as a classifier to detect anticipatory slow cortical potentials related to braking intention in human participants using an electroencephalogram (EEG) was studied. The EEG data was collected during an experiment wherein participants operated a remote controlled vehicle on a testbed designed to simulate an urban environment. Participants were alerted to an incoming braking event via an audio countdown to elicit anticipatory potentials that were then measured using an EEG. The CSNN's performance was compared to a standard convolutional neural network (CNN) and three graph neural networks (GNNs) via 10-fold cross-validation. The results showed that the CSNN outperformed the other neural networks.注释
10 pages, 5 figures, IEEE transaction on Neural Networks submission邮件日期
2022年08月16日- A Time-to-first-spike Coding and Conversion Aware Training for Energy-Efficient Deep Spiking Neural Network Processor Design 时间：2022年08月09日 第一作者：Dongwoo Lew 链接.
摘要：在本文中，我们提出了一种能量高效的SNN架构，它可以无缝运行深度脉冲神经网络（SNN），并提高精度。首先，我们提出了一种转换感知训练（CAT），以减少ANN到SNN的转换损失，而无需硬件实现开销。在所提出的CAT中，有效地利用了为模拟ANN训练期间的SNN而开发的激活函数，以减少转换后的数据表示误差。基于CAT技术，我们还提出了一种时间到第一脉冲编码，该编码允许利用脉冲时间信息进行轻量级对数计算。支持所提出技术的SNN处理器设计已使用28nm CMOS工艺实现。当运行具有5位对数权重的VGG-16时，处理器分别以486.7uJ、503.6uJ和1426uJ的推理能量处理CIFAR-10、CIFAR-100和Tiny ImageNet，达到了91.7%、67.9%和57.4%的顶级精度。
英文摘要
In this paper, we present an energy-efficient SNN architecture, which can seamlessly run deep spiking neural networks (SNNs) with improved accuracy. First, we propose a conversion aware training (CAT) to reduce ANN-to-SNN conversion loss without hardware implementation overhead. In the proposed CAT, the activation function developed for simulating SNN during ANN training, is efficiently exploited to reduce the data representation error after conversion. Based on the CAT technique, we also present a time-to-first-spike coding that allows lightweight logarithmic computation by utilizing spike time information. The SNN processor design that supports the proposed techniques has been implemented using 28nm CMOS process. The processor achieves the top-1 accuracies of 91.7%, 67.9% and 57.4% with inference energy of 486.7uJ, 503.6uJ, and 1426uJ to process CIFAR-10, CIFAR-100, and Tiny-ImageNet, respectively, when running VGG-16 with 5bit logarithmic weights.注释
Accepted to Design Automation Conference 2022 DOI: 10.1145/3489517.3530457邮件日期
2022年08月10日- Denoising Induction Motor Sounds Using an Autoencoder 时间：2022年08月08日 第一作者：Thanh Tran 链接.
摘要：去噪是在改善声音信号的质量和充分性的同时从声音信号中去除噪声的过程。声音去噪在语音处理、声音事件分类和机器故障检测系统中有许多应用。本文描述了一种创建自动编码器的方法，用于将有噪声的机器声音映射到干净的声音，以达到去噪目的。声音中有几种类型的噪声，例如环境噪声和信号处理方法产生的频率相关噪声。环境活动产生的噪声为环境噪声。在工厂内，车辆、钻井、在测量区域工作或谈话的人员、风和流水会产生环境噪声。这些噪音在录音中表现为脉冲。在本文的范围内，我们以感应电机的水槽水龙头噪声为例，演示了高斯分布产生的噪声和环境噪声的去除
英文摘要
Denoising is the process of removing noise from sound signals while improving the quality and adequacy of the sound signals. Denoising sound has many applications in speech processing, sound events classification, and machine failure detection systems. This paper describes a method for creating an autoencoder to map noisy machine sounds to clean sounds for denoising purposes. There are several types of noise in sounds, for example, environmental noise and generated frequency-dependent noise from signal processing methods. Noise generated by environmental activities is environmental noise. In the factory, environmental noise can be created by vehicles, drilling, people working or talking in the survey area, wind, and flowing water. Those noises appear as spikes in the sound record. In the scope of this paper, we demonstrate the removal of generated noise with Gaussian distribution and the environmental noise with a specific example of the water sink faucet noise from the induction motor sounds. The proposed method was trained and verified on 49 normal function sounds and 197 horizontal misalignment fault sounds from the Machinery Fault Database (MAFAULDA). The mean square error (MSE) was used as the assessment criteria to evaluate the similarity between denoised sounds using the proposed autoencoder and the original sounds in the test set. The MSE is below or equal to 0.14 when denoise both types of noises on 15 testing sounds of the normal function category. The MSE is below or equal to 0.15 when denoising 60 testing sounds on the horizontal misalignment fault category. The low MSE shows that both the generated Gaussian noise and the environmental noise were almost removed from the original sounds with the proposed trained autoencoder.注释
9 pages, 10 figures, conference邮件日期
2022年08月10日- Spiking Neural Predictive Coding for Continual Learning from Data Streams 时间：2022年08月08日 第一作者：Alex 链接.
注释
Newest revised version of manuscript邮件日期
2022年08月09日- Neuro-symbolic computing with spiking neural networks 时间：2022年08月04日 第一作者：Dominik Dold 链接.
摘要：知识图是一种表现力强、使用广泛的数据结构，因为它们能够以合理和机器可读的方式集成来自不同领域的数据。因此，它们可以用于模拟各种系统，如分子和社交网络。然而，如何在脉冲系统中实现符号推理，以及如何将脉冲神经网络应用于此类图形数据，仍然是一个悬而未决的问题。在这里，我们通过演示如何使用脉冲神经元对符号和多关系信息进行编码，扩展了先前关于基于脉冲的图算法的工作，允许使用脉冲神经网络对符号结构（如知识图）进行推理。引入的框架是通过结合图嵌入范式和使用误差反向传播训练脉冲神经网络的最新进展而实现的。所提出的方法适用于各种脉冲神经元模型，并可与其他方法相结合进行端到端训练
英文摘要
Knowledge graphs are an expressive and widely used data structure due to their ability to integrate data from different domains in a sensible and machine-readable way. Thus, they can be used to model a variety of systems such as molecules and social networks. However, it still remains an open question how symbolic reasoning could be realized in spiking systems and, therefore, how spiking neural networks could be applied to such graph data. Here, we extend previous work on spike-based graph algorithms by demonstrating how symbolic and multi-relational information can be encoded using spiking neurons, allowing reasoning over symbolic structures like knowledge graphs with spiking neural networks. The introduced framework is enabled by combining the graph embedding paradigm and the recent progress in training spiking neural networks using error backpropagation. The presented methods are applicable to a variety of spiking neuron models and can be trained end-to-end in combination with other differentiable network architectures, which we demonstrate by implementing a spiking relational graph neural network.注释
Accepted for publication at the International Conference on Neuromorphic Systems (ICONS) 2022邮件日期
2022年08月05日- Style Transfer of Black and White Silhouette Images using CycleGAN and a Randomly Generated Dataset 时间：2022年08月03日 第一作者：Worasait Suwannik 链接.
摘要：CycleGAN可用于将艺术风格转换为图像。它不需要成对的源图像和样式化图像来训练模型。利用这一优势，我们建议使用随机生成的数据来训练机器学习模型，该模型可以将传统艺术风格转换为黑白轮廓图像。该结果明显优于先前的神经类型转移方法。然而，还有一些方面需要改进，例如从变换图像中去除伪影和脉冲。
英文摘要
CycleGAN can be used to transfer an artistic style to an image. It does not require pairs of source and stylized images to train a model. Taking this advantage, we propose using randomly generated data to train a machine learning model that can transfer traditional art style to a black and white silhouette image. The result is noticeably better than the previous neural style transfer methods. However, there are some areas for improvement, such as removing artifacts and spikes from the transformed image.邮件日期
2022年08月09日- LaneSNNs: Spiking Neural Networks for Lane Detection on the Loihi Neuromorphic Processor 时间：2022年08月03日 第一作者：Alberto Viale 链接.
摘要：与自动驾驶（AD）相关的功能是下一代移动机器人和自动车辆的重要组成部分，其重点是日益智能、自主和互联的系统。根据定义，涉及使用这些功能的应用程序必须提供实时决策，而这一特性是避免灾难性事故的关键。此外，所有决策过程必须要求低功耗，以增加电池驱动系统的寿命和自主性。这些挑战可以通过在神经形态芯片上有效实现脉冲神经网络（SNN）和使用基于事件的摄像机而不是传统的基于帧的摄像机来解决。在本文中，我们提出了一种新的基于SNN的方法，称为LaneSNN，用于使用基于事件的摄像机输入检测街道上标记的车道。我们开发了四种具有低复杂度和快速响应特性的新型SNN模型，并使用离线监督学习对其进行训练
英文摘要
Autonomous Driving (AD) related features represent important elements for the next generation of mobile robots and autonomous vehicles focused on increasingly intelligent, autonomous, and interconnected systems. The applications involving the use of these features must provide, by definition, real-time decisions, and this property is key to avoid catastrophic accidents. Moreover, all the decision processes must require low power consumption, to increase the lifetime and autonomy of battery-driven systems. These challenges can be addressed through efficient implementations of Spiking Neural Networks (SNNs) on Neuromorphic Chips and the use of event-based cameras instead of traditional frame-based cameras. In this paper, we present a new SNN-based approach, called LaneSNN, for detecting the lanes marked on the streets using the event-based camera input. We develop four novel SNN models characterized by low complexity and fast response, and train them using an offline supervised learning rule. Afterward, we implement and map the learned SNNs models onto the Intel Loihi Neuromorphic Research Chip. For the loss function, we develop a novel method based on the linear composition of Weighted binary Cross Entropy (WCE) and Mean Squared Error (MSE) measures. Our experimental results show a maximum Intersection over Union (IoU) measure of about 0.62 and very low power consumption of about 1 W. The best IoU is achieved with an SNN implementation that occupies only 36 neurocores on the Loihi processor while providing a low latency of less than 8 ms to recognize an image, thereby enabling real-time performance. The IoU measures provided by our networks are comparable with the state-of-the-art, but at a much low power consumption of 1 W.注释
To appear at the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022)邮件日期
2022年08月05日- Making a Spiking Net Work: Robust brain-like unsupervised machine learning 时间：2022年08月02日 第一作者：Peter G. Stratton 链接.
摘要：在过去十年中，人工智能（AI）的兴趣激增几乎完全是由人工神经网络（ANN）的进步推动的。虽然人工神经网络为许多以前难以解决的问题提供了最先进的性能，但它们需要大量的数据和计算资源用于训练，并且由于它们采用监督学习，它们通常需要知道每个训练示例的正确标记响应，从而限制了它们在现实领域的可扩展性。脉冲神经网络（SNN）是神经网络的一种替代方案，它使用更多类似大脑的人工神经元，并且可以使用无监督学习来发现输入数据中的可识别特征，而无需知道正确的响应。然而，SNN难以保持动态稳定性，无法与ANN的精度相匹配。在这里，我们展示了SNN如何克服文献中确定的许多缺点，包括为消失脉冲问题提供一个原则性解决方案
英文摘要
The surge in interest in Artificial Intelligence (AI) over the past decade has been driven almost exclusively by advances in Artificial Neural Networks (ANNs). While ANNs set state-of-the-art performance for many previously intractable problems, they require large amounts of data and computational resources for training, and since they employ supervised learning they typically need to know the correctly labelled response for every training example, limiting their scalability for real-world domains. Spiking Neural Networks (SNNs) are an alternative to ANNs that use more brain-like artificial neurons and can use unsupervised learning to discover recognizable features in the input data without knowing correct responses. SNNs, however, struggle with dynamical stability and cannot match the accuracy of ANNs. Here we show how an SNN can overcome many of the shortcomings that have been identified in the literature, including offering a principled solution to the vanishing spike problem, to outperform all existing shallow SNNs and equal the performance of an ANN. It accomplishes this while using unsupervised learning with unlabeled data and only 1/50th of the training epochs (labelled data is used only for a final simple linear readout layer). This result makes SNNs a viable new method for fast, accurate, efficient, explainable, and re-deployable machine learning with unlabeled datasets.注释
12 pages (manuscript), 10 pages (appendix), 10 pages (extended data)邮件日期
2022年08月03日- MT-SNN: Spiking Neural Network that Enables Single-Tasking of Multiple Tasks 时间：2022年08月02日 第一作者：Paolo G. Cachi 链接.
摘要：在本文中，我们探索了脉冲神经网络在解决多任务分类问题中使用多任务单任务方法的能力。我们设计并实现了一个多任务脉冲神经网络（MT-SNN），它可以在一次执行一个任务的同时学习两个或多个分类任务。通过调节本工作中使用的漏积分和激发神经元的激发阈值来选择要执行的任务。该网络使用Intel的Loihi2神经形态芯片的Lava平台实现。对NMNIST数据的动态多任务分类进行了测试。结果表明，MT-SNN通过修改其动力学，即脉冲神经元的放电阈值，有效地学习多任务。
英文摘要
In this paper we explore capabilities of spiking neural networks in solving multi-task classification problems using the approach of single-tasking of multiple tasks. We designed and implemented a multi-task spiking neural network (MT-SNN) that can learn two or more classification tasks while performing one task at a time. The task to perform is selected by modulating the firing threshold of leaky integrate and fire neurons used in this work. The network is implemented using Intel's Lava platform for the Loihi2 neuromorphic chip. Tests are performed on dynamic multitask classification for NMNIST data. The results show that MT-SNN effectively learns multiple tasks by modifying its dynamics, namely, the spiking neurons' firing threshold.注释
4 pages, 2 figures邮件日期
2022年08月03日- Spiking Graph Convolutional Networks 时间：2022年08月02日 第一作者：Zulun Zhu 链接.
注释
Accepted by IJCAI 2022; Code available at https://github.com/ZulunZhu/SpikingGCN邮件日期
2022年08月03日- enpheeph: A Fault Injection Framework for Spiking and Compressed Deep Neural Networks 时间：2022年07月31日 第一作者：Alessio Colucci 链接.
摘要：深度神经网络（DNN）的研究侧重于提高实际部署的性能和准确性，从而产生了新的模型，如脉冲神经网络（SNN），以及优化技术，如压缩网络的量化和修剪。然而，这些创新模型和优化技术的部署引入了可能的可靠性问题，这是DNN广泛用于安全关键应用（如自动驾驶）的支柱。此外，扩展技术节点具有同时发生多个故障的相关风险，这一可能性在最先进的弹性分析中没有解决。为了更好地分析DNN的可靠性，我们提出了enpheeph，这是一种用于脉冲和压缩DNN的故障注入框架。enpheeph框架支持在专用硬件设备（如GPU）上优化执行，同时提供完全的可定制性，以调查不同的故障模型，模拟各种可靠性测试
英文摘要
Research on Deep Neural Networks (DNNs) has focused on improving performance and accuracy for real-world deployments, leading to new models, such as Spiking Neural Networks (SNNs), and optimization techniques, e.g., quantization and pruning for compressed networks. However, the deployment of these innovative models and optimization techniques introduces possible reliability issues, which is a pillar for DNNs to be widely used in safety-critical applications, e.g., autonomous driving. Moreover, scaling technology nodes have the associated risk of multiple faults happening at the same time, a possibility not addressed in state-of-the-art resiliency analyses. Towards better reliability analysis for DNNs, we present enpheeph, a Fault Injection Framework for Spiking and Compressed DNNs. The enpheeph framework enables optimized execution on specialized hardware devices, e.g., GPUs, while providing complete customizability to investigate different fault models, emulating various reliability constraints and use-cases. Hence, the faults can be executed on SNNs as well as compressed networks with minimal-to-none modifications to the underlying code, a feat that is not achievable by other state-of-the-art tools. To evaluate our enpheeph framework, we analyze the resiliency of different DNN and SNN models, with different compression techniques. By injecting a random and increasing number of faults, we show that DNNs can show a reduction in accuracy with a fault rate as low as 7 x 10 ^ (-7) faults per parameter, with an accuracy drop higher than 40%. Run-time overhead when executing enpheeph is less than 20% of the baseline execution time when executing 100 000 faults concurrently, at least 10x lower than state-of-the-art frameworks, making enpheeph future-proof for complex fault injection scenarios. We release enpheeph at https://github.com/Alexei95/enpheeph.注释
Source code: https://github.com/Alexei95/enpheeph To appear at 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October, 2022邮件日期
2022年08月02日- Ultra-low Latency Adaptive Local Binary Spiking Neural Network with Accuracy Loss Estimator 时间：2022年07月31日 第一作者：Changqing Xu 链接.
摘要：脉冲神经网络（SNN）是一种受大脑启发的模型，具有更强的时空信息处理能力和计算能量效率。然而，随着SNN深度的增加，SNN权重引起的记忆问题逐渐引起关注。受人工神经网络（ANN）量化技术的启发，引入二值化SNN（BSNN）来解决记忆问题。由于缺乏合适的学习算法，BSNN通常通过ANN到SNN的转换获得，其精度将受到训练的ANN的限制。在本文中，我们提出了一种具有精度损失估计器的超低延迟自适应局部二进制脉冲神经网络（ALBSNN），该网络通过评估网络学习过程中二值化权重引起的误差，动态选择要二值化的网络层，以确保网络的精度。实验结果表明，该方法可以在不损失网络的情况下，将存储空间减少20%以上
英文摘要
Spiking neural network (SNN) is a brain-inspired model which has more spatio-temporal information processing capacity and computational energy efficiency. However, with the increasing depth of SNNs, the memory problem caused by the weights of SNNs has gradually attracted attention. Inspired by Artificial Neural Networks (ANNs) quantization technology, binarized SNN (BSNN) is introduced to solve the memory problem. Due to the lack of suitable learning algorithms, BSNN is usually obtained by ANN-to-SNN conversion, whose accuracy will be limited by the trained ANNs. In this paper, we propose an ultra-low latency adaptive local binary spiking neural network (ALBSNN) with accuracy loss estimators, which dynamically selects the network layers to be binarized to ensure the accuracy of the network by evaluating the error caused by the binarized weights during the network learning process. Experimental results show that this method can reduce storage space by more than 20 % without losing network accuracy. At the same time, in order to accelerate the training speed of the network, the global average pooling(GAP) layer is introduced to replace the fully connected layers by the combination of convolution and pooling, so that SNNs can use a small number of time steps to obtain better recognition accuracy. In the extreme case of using only one time step, we still can achieve 92.92 %, 91.63 % ,and 63.54 % testing accuracy on three different datasets, FashionMNIST, CIFAR-10, and CIFAR-100, respectively.邮件日期
2022年08月02日- Text Classification in Memristor-based Spiking Neural Networks 时间：2022年07月31日 第一作者：Jinqi Huang 链接.
注释
23 pages, 5 figures邮件日期
2022年08月02日- Spiking neural networks trained via proxy 时间：2022年07月30日 第一作者：Saeed Reza Kheradpisheh 链接.
邮件日期
2022年08月02日- Text Classification in Memristor-based Spiking Neural Networks 时间：2022年07月27日 第一作者：Jinqi Huang 链接.
摘要：忆阻器是一种新兴的非易失性存储器件，在神经形态硬件设计中显示出了巨大的潜力，特别是在脉冲神经网络（SNN）硬件实现中。基于忆阻器的SNN已成功应用于广泛的各种应用，包括图像分类和模式识别。然而，在文本分类中实现基于记忆的SNN仍在探索中。其中一个主要原因是，由于缺乏有效的学习规则和记忆器的非理想性，训练用于文本分类的基于记忆器的SNN代价高昂。为了解决这些问题并加快在文本分类应用中探索基于忆阻器的脉冲神经网络的研究，我们使用经验忆阻器模型开发了虚拟忆阻器阵列的仿真框架。我们使用这个框架来演示IMDB电影评论数据集中的情感分析任务。我们采用两种方法获得训练的脉冲神经网络
英文摘要
Memristors, emerging non-volatile memory devices, have shown promising potential in neuromorphic hardware designs, especially in spiking neural network (SNN) hardware implementation. Memristor-based SNNs have been successfully applied in a wide range of various applications, including image classification and pattern recognition. However, implementing memristor-based SNNs in text classification is still under exploration. One of the main reasons is that training memristor-based SNNs for text classification is costly due to the lack of efficient learning rules and memristor non-idealities. To address these issues and accelerate the research of exploring memristor-based spiking neural networks in text classification applications, we develop a simulation framework with a virtual memristor array using an empirical memristor model. We use this framework to demonstrate a sentiment analysis task in the IMDB movie reviews dataset. We take two approaches to obtain trained spiking neural networks with memristor models: 1) by converting a pre-trained artificial neural network (ANN) to a memristor-based SNN, or 2) by training a memristor-based SNN directly. These two approaches can be applied in two scenarios: offline classification and online training. We achieve the classification accuracy of 85.88% by converting a pre-trained ANN to a memristor-based SNN and 84.86% by training the memristor-based SNN directly, given that the baseline training accuracy of the equivalent ANN is 86.02%. We conclude that it is possible to achieve similar classification accuracy in simulation from ANNs to SNNs and from non-memristive synapses to data-driven memristive synapses. We also investigate how global parameters such as spike train length, the read noise, and the weight updating stop conditions affect the neural networks in both approaches.注释
23 pages, 5 figures邮件日期
2022年07月29日- Towards the Neuroevolution of Low-level Artificial General Intelligence 时间：2022年07月27日 第一作者：Sidney Pontes-Filho 链接.
摘要：在这项工作中，我们认为人工通用智能（AGI）的搜索应该从比人类智能低得多的水平开始。自然界中智能行为的环境是由有机体与其周围环境相互作用产生的，随着时间的推移，可能会发生变化，并对有机体施加压力，以允许学习新的行为或环境模型。我们的假设是，当一个主体在一个环境中行动时，学习是通过解释感觉反馈发生的。要做到这一点，需要一个机构和一个反应环境。我们评估了一种进化从环境反应中学习的生物启发人工神经网络的方法，称为人工通用智能（NAGI）的神经进化，这是一种低水平AGI框架。该方法允许具有自适应突触的随机初始化脉冲神经网络的进化复杂化，该神经网络控制在可变环境中实例化的代理。这种结构
英文摘要
In this work, we argue that the search for Artificial General Intelligence (AGI) should start from a much lower level than human-level intelligence. The circumstances of intelligent behavior in nature resulted from an organism interacting with its surrounding environment, which could change over time and exert pressure on the organism to allow for learning of new behaviors or environment models. Our hypothesis is that learning occurs through interpreting sensory feedback when an agent acts in an environment. For that to happen, a body and a reactive environment are needed. We evaluate a method to evolve a biologically-inspired artificial neural network that learns from environment reactions named Neuroevolution of Artificial General Intelligence (NAGI), a framework for low-level AGI. This method allows the evolutionary complexification of a randomly-initialized spiking neural network with adaptive synapses, which controls agents instantiated in mutable environments. Such a configuration allows us to benchmark the adaptivity and generality of the controllers. The chosen tasks in the mutable environments are food foraging, emulation of logic gates, and cart-pole balancing. The three tasks are successfully solved with rather small network topologies and therefore it opens up the possibility of experimenting with more complex tasks and scenarios where curriculum learning is beneficial.注释
18 pages, 14 figures MSC-class: 68T05 ACM-class: I.2.6邮件日期
2022年07月28日- SPAIC: A Spike-based Artificial Intelligence Computing Framework 时间：2022年07月26日 第一作者：Chaofei Hong 链接.
摘要：神经形态计算是一个新兴的研究领域，旨在通过整合神经科学和深度学习等多学科的理论和技术来开发新的智能系统。目前，已经为相关领域开发了各种软件框架，但缺乏专门用于基于峰值的计算模型和算法的有效框架。在这项工作中，我们提出了一个基于Python的脉冲神经网络（SNN）模拟和训练框架，又名SPAIC，旨在支持结合深度学习和神经科学特征的脑启发模型和算法研究。为了整合这两个压倒性学科的不同方法，平衡灵活性和效率，SPAIC设计了神经科学风格的前端和深度学习后端结构。我们提供了广泛的示例，包括神经电路仿真、深度SNN学习和神经形态应用，演示了
英文摘要
Neuromorphic computing is an emerging research field that aims to develop new intelligent systems by integrating theories and technologies from multi-disciplines such as neuroscience and deep learning. Currently, there have been various software frameworks developed for the related fields, but there is a lack of an efficient framework dedicated for spike-based computing models and algorithms. In this work, we present a Python based spiking neural network (SNN) simulation and training framework, aka SPAIC that aims to support brain-inspired model and algorithm researches integrated with features from both deep learning and neuroscience. To integrate different methodologies from the two overwhelming disciplines, and balance between flexibility and efficiency, SPAIC is designed with neuroscience-style frontend and deep learning backend structure. We provide a wide range of examples including neural circuits Simulation, deep SNN learning and neuromorphic applications, demonstrating the concise coding style and wide usability of our framework. The SPAIC is a dedicated spike-based artificial intelligence computing platform, which will significantly facilitate the design, prototype and validation of new models, theories and applications. Being user-friendly, flexible and high-performance, it will help accelerate the rapid growth and wide applicability of neuromorphic computing research.注释
This Paper has been submitted to IEEE computational intelligence magazine邮件日期
2022年07月27日- Static Hand Gesture Recognition for American Sign Language using Neuromorphic Hardware 时间：2022年07月25日 第一作者：MohammedReza Mohammadi 链接.
摘要：在本文中，我们为两个静态美国手语（ASL）手势分类任务（即ASL字母表和ASL数字）开发了四个脉冲神经网络（SNN）模型。SNN模型部署在Intel的神经形态平台Loihi上，然后与部署在边缘计算设备Intel神经计算棒2（NCS2）上的等效深度神经网络（DNN）模型进行比较。我们在准确性、延迟、功耗和能量方面对两个系统进行了全面比较。最佳DNN模型在ASL字母表数据集上的准确率达到99.6%，而最佳SNN模型的准确率为99.44%。对于ASL数字数据集，最佳SNN模型以99.52%的准确率优于所有DNN模型。此外，我们获得的实验结果表明，与NCS2相比，Loihi神经形态硬件实现的功耗和能量分别减少了14.67x和4.09x。
英文摘要
In this paper, we develop four spiking neural network (SNN) models for two static American Sign Language (ASL) hand gesture classification tasks, i.e., the ASL Alphabet and ASL Digits. The SNN models are deployed on Intel's neuromorphic platform, Loihi, and then compared against equivalent deep neural network (DNN) models deployed on an edge computing device, the Intel Neural Compute Stick 2 (NCS2). We perform a comprehensive comparison between the two systems in terms of accuracy, latency, power consumption, and energy. The best DNN model achieves an accuracy of 99.6% on the ASL Alphabet dataset, whereas the best performing SNN model has an accuracy of 99.44%. For the ASL-Digits dataset, the best SNN model outperforms all of its DNN counterparts with 99.52% accuracy. Moreover, our obtained experimental results show that the Loihi neuromorphic hardware implementations achieve up to 14.67x and 4.09x reduction in power consumption and energy, respectively, when compared to NCS2.注释
Authors MohammedReza Mohammadi, and Peyton Chandarana contributed equally邮件日期
2022年07月27日- Modeling Associative Plasticity between Synapses to Enhance Learning of Spiking Neural Networks 时间：2022年07月24日 第一作者：Haibo Shen 链接.
摘要：脉冲神经网络（SNN）是第三代人工神经网络，能够在神经形态硬件上实现节能。然而，脉冲的离散传输给鲁棒和高性能学习机制带来了重大挑战。现有的大多数工作只关注神经元之间的学习，而忽略了突触之间的影响，导致鲁棒性和准确性的损失。为了解决这个问题，我们提出了一种鲁棒有效的学习机制，通过模拟突触间的联想可塑性（APB），从联想长时程增强（ALTP）的生理现象中观察到。在所提出的APBS方法中，当其他神经元同时刺激时，同一神经元的突触通过共享因子相互作用。此外，我们提出了一种时空裁剪和翻转（STCF）方法来提高网络的泛化能力。大量实验表明，我们的方法实现了
英文摘要
Spiking Neural Networks (SNNs) are the third generation of artificial neural networks that enable energy-efficient implementation on neuromorphic hardware. However, the discrete transmission of spikes brings significant challenges to the robust and high-performance learning mechanism. Most existing works focus solely on learning between neurons but ignore the influence between synapses, resulting in a loss of robustness and accuracy. To address this problem, we propose a robust and effective learning mechanism by modeling the associative plasticity between synapses (APBS) observed from the physiological phenomenon of associative long-term potentiation (ALTP). With the proposed APBS method, synapses of the same neuron interact through a shared factor when concurrently stimulated by other neurons. In addition, we propose a spatiotemporal cropping and flipping (STCF) method to improve the generalization ability of our network. Extensive experiments demonstrate that our approaches achieve superior performance on static CIFAR-10 datasets and state-of-the-art performance on neuromorphic MNIST-DVS, CIFAR10-DVS datasets by a lightweight convolution network. To our best knowledge, this is the first time to explore a learning method between synapses and an extended approach for neuromorphic data.注释
Submitted to ijcai2022, rejected邮件日期
2022年07月26日- Event-Driven Tactile Learning with Location Spiking Neurons 时间：2022年07月23日 第一作者：Peng Kang 链接.
摘要：触觉对于各种日常任务至关重要。基于事件的触觉传感器和脉冲神经网络（SNN）的新进展推动了事件驱动触觉学习的研究。然而，由于现有脉冲神经元的代表性能力有限以及数据的时空复杂性，SNN激活的事件驱动触觉学习仍处于初级阶段。在本文中，为了提高现有脉冲神经元的代表性能力，我们提出了一种新的神经元模型，称为“位置脉冲神经元”，它使我们能够以新的方式提取基于事件的数据的特征。此外，在经典的时间脉冲响应模型（TSRM）的基础上，我们开发了一种特定的位置脉冲神经元模型-位置脉冲响应（LSRM），作为SNN的新构建块。此外，我们提出了一种混合模型，将SNN与TSRM神经元和SNN与LSRM神经元相结合，以捕获数据中复杂的时空相关性。广阔的
英文摘要
The sense of touch is essential for a variety of daily tasks. New advances in event-based tactile sensors and Spiking Neural Networks (SNNs) spur the research in event-driven tactile learning. However, SNN-enabled event-driven tactile learning is still in its infancy due to the limited representative abilities of existing spiking neurons and high spatio-temporal complexity in the data. In this paper, to improve the representative capabilities of existing spiking neurons, we propose a novel neuron model called "location spiking neuron", which enables us to extract features of event-based data in a novel way. Moreover, based on the classical Time Spike Response Model (TSRM), we develop a specific location spiking neuron model - Location Spike Response Model (LSRM) that serves as a new building block of SNNs. Furthermore, we propose a hybrid model which combines an SNN with TSRM neurons and an SNN with LSRM neurons to capture the complex spatio-temporal dependencies in the data. Extensive experiments demonstrate the significant improvements of our models over other works on event-driven tactile learning and show the superior energy efficiency of our models and location spiking neurons, which may unlock their potential on neuromorphic hardware.注释
accepted by IJCNN 2022 (oral), the source code is available at https://github.com/pkang2017/TactileLocNeurons邮件日期
2022年09月05日- NeuroHSMD: Neuromorphic Hybrid Spiking Motion Detector 时间：2022年07月22日 第一作者：Pedro Machado 链接.
邮件日期
2022年07月25日- Exploring Lottery Ticket Hypothesis in Spiking Neural Networks 时间：2022年07月20日 第一作者：Youngeun Kim 链接.
注释
Accepted to European Conference on Computer Vision (ECCV) 2022邮件日期
2022年07月22日- Neural Architecture Search for Spiking Neural Networks 时间：2022年07月20日 第一作者：Youngeun Kim 链接.
注释
Accepted to European Conference on Computer Vision (ECCV) 2022邮件日期
2022年07月22日- A Temporally and Spatially Local Spike-based Backpropagation Algorithm to Enable Training in Hardware 时间：2022年07月20日 第一作者：Anmol Biswas 链接.
摘要：脉冲神经网络（SNN）已成为分类任务的硬件高效架构。基于脉冲的编码的缺点是缺乏完全使用脉冲执行的通用训练机制。已经有几次尝试采用非脉冲人工神经网络（ANN）中使用的强大反向传播（BP）技术：（1）SNN可以通过外部计算的数值梯度进行训练。（2） 基于自然脉冲的学习的一个主要进展是使用了近似反向传播，使用脉冲时间相关塑性（STDP）和分阶段的前/后向传递。然而，这些阶段之间的信息传输需要外部存储器和计算访问。这是神经形态硬件实现的挑战。在本文中，我们提出了一种基于随机SNN的反向支持（SSNN-BP）算法，该算法利用一个复合神经元同时计算前向通过激活和后向通过梯度
英文摘要
Spiking Neural Networks (SNNs) have emerged as a hardware efficient architecture for classification tasks. The penalty of spikes-based encoding has been the lack of a universal training mechanism performed entirely using spikes. There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANN): (1) SNNs can be trained by externally computed numerical gradients. (2) A major advancement toward native spike-based learning has been the use of approximate Backpropagation using spike-time-dependent plasticity (STDP) with phased forward/backward passes. However, the transfer of information between such phases necessitates external memory and computational access. This is a challenge for neuromorphic hardware implementations. In this paper, we propose a stochastic SNN-based Back-Prop (SSNN-BP) algorithm that utilizes a composite neuron to simultaneously compute the forward pass activations and backward pass gradients explicitly with spikes. Although signed gradient values are a challenge for spike-based representation, we tackle this by splitting the gradient signal into positive and negative streams. The composite neuron encodes information in the form of stochastic spike-trains and converts Backpropagation weight updates into temporally and spatially local discrete STDP-like spike coincidence updates compatible with hardware-friendly Resistive Processing Units (RPUs). Furthermore, our method approaches BP ANN baseline with sufficiently long spike-trains. Finally, we show that softmax cross-entropy loss function can be implemented through inhibitory lateral connections enforcing a Winner Take All (WTA) rule. Our SNN shows excellent generalization through comparable performance to ANNs on the MNIST, Fashion-MNIST and Extended MNIST datasets. Thus, SSNN-BP enables BP compatible with purely spike-based neuromorphic hardware.邮件日期
2022年07月21日- Neuromorphic Data Augmentation for Training Spiking Neural Networks 时间：2022年07月20日 第一作者：Yuhang Li 链接.
注释
Accepted to the 17th European Conference on Computer Vision (ECCV 2022)邮件日期
2022年07月21日- The Brain-Inspired Decoder for Natural Visual Image Reconstruction 时间：2022年07月18日 第一作者：Wenyi Li 链接.
摘要：从大脑活动中解码图像一直是一个挑战。由于深度学习的发展，有可用的工具来解决这个问题。解码图像，其目的是将神经脉冲训练映射到低级视觉特征和高级语义信息空间。最近，有一些关于从棘波序列解码的研究，然而，这些研究较少关注神经科学的基础，并且很少有研究将感受野合并到视觉图像重建中。在本文中，我们提出了一种具有生物学特性的深度学习神经网络结构，用于从脉冲序列重建视觉图像。据我们所知，我们第一次实现了将接收场特性矩阵整合到损失函数中的方法。我们的模型是从神经脉冲序列到图像的端到端解码器。我们不仅将Gabor滤波器合并到用于生成图像的自动编码器中，还提出了一种具有感受野特性的损失函数。
英文摘要
Decoding images from brain activity has been a challenge. Owing to the development of deep learning, there are available tools to solve this problem. The decoded image, which aims to map neural spike trains to low-level visual features and high-level semantic information space. Recently, there are a few studies of decoding from spike trains, however, these studies pay less attention to the foundations of neuroscience and there are few studies that merged receptive field into visual image reconstruction. In this paper, we propose a deep learning neural network architecture with biological properties to reconstruct visual image from spike trains. As far as we know, we implemented a method that integrated receptive field property matrix into loss function at the first time. Our model is an end-to-end decoder from neural spike trains to images. We not only merged Gabor filter into auto-encoder which used to generate images but also proposed a loss function with receptive field properties. We evaluated our decoder on two datasets which contain macaque primary visual cortex neural spikes and salamander retina ganglion cells (RGCs) spikes. Our results show that our method can effectively combine receptive field features to reconstruct images, providing a new approach to visual reconstruction based on neural information.邮件日期
2022年07月19日- BrainCog: A Spiking Neural Network based Brain-inspired Cognitive Intelligence Engine for Brain-inspired AI and Brain Simulation 时间：2022年07月18日 第一作者：Yi Zeng 链接.
摘要：脉冲神经网络（SNN）在脑启发人工智能和计算神经科学中引起了广泛关注。它们可以用于在多个尺度上模拟大脑中的生物信息处理。更重要的是，SNN作为一个适当的抽象层次，将大脑和认知的灵感引入人工智能。在本文中，我们提出了脑启发认知智能引擎（BrainCog），用于创建脑启发人工智能和脑模拟模型。BrainCog整合了不同类型的脉冲神经元模型、学习规则、大脑区域等，作为平台提供的基本模块。基于这些易于使用的模块，BrainCog支持各种大脑启发的认知功能，包括感知和学习、决策、知识表示和推理、运动控制和社会认知。这些受大脑启发的人工智能模型已在各种有监督、无监督、有监督和无监督的情况下得到有效验证
英文摘要
Spiking neural networks (SNNs) have attracted extensive attentions in Brain-inspired Artificial Intelligence and computational neuroscience. They can be used to simulate biological information processing in the brain at multiple scales. More importantly, SNNs serve as an appropriate level of abstraction to bring inspirations from brain and cognition to Artificial Intelligence. In this paper, we present the Brain-inspired Cognitive Intelligence Engine (BrainCog) for creating brain-inspired AI and brain simulation models. BrainCog incorporates different types of spiking neuron models, learning rules, brain areas, etc., as essential modules provided by the platform. Based on these easy-to-use modules, BrainCog supports various brain-inspired cognitive functions, including Perception and Learning, Decision Making, Knowledge Representation and Reasoning, Motor Control, and Social Cognition. These brain-inspired AI models have been effectively validated on various supervised, unsupervised, and reinforcement learning tasks, and they can be used to enable AI models to be with multiple brain-inspired cognitive functions. For brain simulation, BrainCog realizes the function simulation of decision-making, working memory, the structure simulation of the Neural Circuit, and whole brain structure simulation of Mouse brain, Macaque brain, and Human brain. An AI engine named BORN is developed based on BrainCog, and it demonstrates how the components of BrainCog can be integrated and used to build AI models and applications. To enable the scientific quest to decode the nature of biological intelligence and create AI, BrainCog aims to provide essential and easy-to-use building blocks, and infrastructural support to develop brain-inspired spiking neural network based AI, and to simulate the cognitive brains at multiple scales. The online repository of BrainCog can be found at https://github.com/braincog-x.邮件日期
2022年07月19日- Efficient spike encoding algorithms for neuromorphic speech recognition 时间：2022年07月14日 第一作者：Sidi Yaya Arnaud Yarga 链接.
摘要：已知脉冲神经网络（SNN）对于神经形态处理器的实现非常有效，与传统深度学习方法相比，在能量效率和计算延迟方面实现了数量级的改进。最近，随着监督训练算法适应SNN环境，可比较的算法性能也成为可能。然而，包括音频、视频和其他传感器衍生数据的信息通常被编码为不适合SNN的实值信号，从而防止网络利用脉冲定时信息。因此，从实值信号到脉冲的有效编码是关键的，并且显著影响整个系统的性能。为了有效地将信号编码成脉冲，必须考虑与手头任务相关的信息的保存以及编码脉冲的密度。在本文中，我们研究了说话人背景下的四种脉冲编码方法
英文摘要
Spiking Neural Networks (SNN) are known to be very effective for neuromorphic processor implementations, achieving orders of magnitude improvements in energy efficiency and computational latency over traditional deep learning approaches. Comparable algorithmic performance was recently made possible as well with the adaptation of supervised training algorithms to the context of SNN. However, information including audio, video, and other sensor-derived data are typically encoded as real-valued signals that are not well-suited to SNN, preventing the network from leveraging spike timing information. Efficient encoding from real-valued signals to spikes is therefore critical and significantly impacts the performance of the overall system. To efficiently encode signals into spikes, both the preservation of information relevant to the task at hand as well as the density of the encoded spikes must be considered. In this paper, we study four spike encoding methods in the context of a speaker independent digit classification system: Send on Delta, Time to First Spike, Leaky Integrate and Fire Neuron and Bens Spiker Algorithm. We first show that all encoding methods yield higher classification accuracy using significantly fewer spikes when encoding a bio-inspired cochleagram as opposed to a traditional short-time Fourier transform. We then show that two Send On Delta variants result in classification results comparable with a state of the art deep convolutional neural network baseline, while simultaneously reducing the encoded bit rate. Finally, we show that several encoding methods result in improved performance over the conventional deep learning baseline in certain cases, further demonstrating the power of spike encoding algorithms in the encoding of real-valued signals and that neuromorphic implementation has the potential to outperform state of the art techniques.注释
Accepted to International Conference on Neuromorphic Systems (ICONS 2022) DOI: 10.1145/3546790.3546803邮件日期
2022年07月15日- A Macrocolumn Architecture Implemented with Temporal (Spiking) Neurons 时间：2022年07月11日 第一作者：James E. Smith 链接.
摘要：由于长期目标是自下而上逆向架构计算大脑，因此本文的重点是宏列抽象层。通过首先用状态机模型描述其操作，开发了基本的宏列架构。然后用支持时间计算的脉冲神经元实现状态机函数。神经元模型基于活跃的脉冲树突，反映了Hawkins/Numenta神经元模型。该架构通过一个研究基准进行了演示，其中代理使用宏列首先学习，然后导航包含随机放置特征的二维环境。环境在宏列中表示为带标签的有向图，其中边连接特征，标签表示它们之间的相对位移。
英文摘要
With the long-term goal of reverse-architecting the computational brain from the bottom up, the focus of this document is the macrocolumn abstraction layer. A basic macrocolumn architecture is developed by first describing its operation with a state machine model. Then state machine functions are implemented with spiking neurons that support temporal computation. The neuron model is based on active spiking dendrites and mirrors the Hawkins/Numenta neuron model. The architecture is demonstrated with a research benchmark in which an agent uses a macrocolumn to first learn and then navigate 2-d environments containing randomly placed features. Environments are represented in the macrocolumn as labeled directed graphs where edges connect features and labels indicate the relative displacements between them.邮件日期
2022年07月13日- Brain-inspired Graph Spiking Neural Networks for Commonsense Knowledge Representation and Reasoning 时间：2022年07月11日 第一作者：Hongjian Fang 链接.
摘要：人脑中的神经网络如何代表常识知识，并完成相关推理任务，是神经科学、认知科学、心理学和人工智能领域的重要研究课题。尽管使用固定长度向量表示符号的传统人工神经网络在某些特定任务中取得了良好的性能，但它仍然是一个缺乏可解释性的黑盒子，与人类如何感知世界相去甚远。受神经科学中祖母细胞假说的启发，这项工作研究了群体编码和脉冲时间依赖性可塑性（STDP）机制如何整合到脉冲神经网络的学习中，以及神经元群体如何通过引导不同神经元群体之间的顺序放电完成来表示符号。不同群体的神经元群体共同构成了整个常识知识图，形成了一个巨大的图脉冲神经网络。此外，我们和我
英文摘要
How neural networks in the human brain represent commonsense knowledge, and complete related reasoning tasks is an important research topic in neuroscience, cognitive science, psychology, and artificial intelligence. Although the traditional artificial neural network using fixed-length vectors to represent symbols has gained good performance in some specific tasks, it is still a black box that lacks interpretability, far from how humans perceive the world. Inspired by the grandmother-cell hypothesis in neuroscience, this work investigates how population encoding and spiking timing-dependent plasticity (STDP) mechanisms can be integrated into the learning of spiking neural networks, and how a population of neurons can represent a symbol via guiding the completion of sequential firing between different neuron populations. The neuron populations of different communities together constitute the entire commonsense knowledge graph, forming a giant graph spiking neural network. Moreover, we introduced the Reward-modulated spiking timing-dependent plasticity (R-STDP) mechanism to simulate the biological reinforcement learning process and completed the related reasoning tasks accordingly, achieving comparable accuracy and faster convergence speed than the graph convolutional artificial neural networks. For the fields of neuroscience and cognitive science, the work in this paper provided the foundation of computational modeling for further exploration of the way the human brain represents commonsense knowledge. For the field of artificial intelligence, this paper indicated the exploration direction for realizing a more robust and interpretable neural network by constructing a commonsense knowledge representation and reasoning spiking neural networks with solid biological plausibility.邮件日期
2022年07月13日- BioLCNet: Reward-modulated Locally Connected Spiking Neural Networks 时间：2022年07月07日 第一作者：Hafez Ghaemi 链接.
注释
15 pages, 6 figures ACM-class: I.2.6; I.5.1邮件日期
2022年07月08日- Spike Calibration: Fast and Accurate Conversion of Spiking Neural Network for Object Detection and Segmentation 时间：2022年07月06日 第一作者：Yang Li 链接.
摘要：脉冲神经网络（SNN）由于在神经形态硬件上具有高生物似然性和低能量消耗的特性而受到高度重视。作为获得深度SNN的有效方法，该转换方法在各种大规模数据集上表现出了高性能。然而，它通常遭受严重的性能降级和高时间延迟。特别是，以前的大多数工作集中于简单的分类任务，而忽略了神经网络输出的精确近似。在本文中，我们首先从理论上分析了转换误差，并推导了时变极值对突触电流的有害影响。我们提出了脉冲校准（SpiCalib）以消除离散脉冲对输出分布的损害，并修改Lipoooling以允许任意最大池层的无损转换。此外，提出了最佳归一化参数的贝叶斯优化，以避免经验设置。经验
英文摘要
Spiking neural network (SNN) has been attached to great importance due to the properties of high biological plausibility and low energy consumption on neuromorphic hardware. As an efficient method to obtain deep SNN, the conversion method has exhibited high performance on various large-scale datasets. However, it typically suffers from severe performance degradation and high time delays. In particular, most of the previous work focuses on simple classification tasks while ignoring the precise approximation to ANN output. In this paper, we first theoretically analyze the conversion errors and derive the harmful effects of time-varying extremes on synaptic currents. We propose the Spike Calibration (SpiCalib) to eliminate the damage of discrete spikes to the output distribution and modify the LIPooling to allow conversion of the arbitrary MaxPooling layer losslessly. Moreover, Bayesian optimization for optimal normalization parameters is proposed to avoid empirical settings. The experimental results demonstrate the state-of-the-art performance on classification, object detection, and segmentation tasks. To the best of our knowledge, this is the first time to obtain SNN comparable to ANN on these tasks simultaneously. Moreover, we only need 1/50 inference time of the previous work on the detection task and can achieve the same performance under 0.492$\times$ energy consumption of ANN on the segmentation task.邮件日期
2022年07月07日- An Unsupervised Spiking Neural Network Inspired By Biologically Plausible Learning Rules and Connections 时间：2022年07月06日 第一作者：Yiting Dong 链接.
摘要：反向传播算法促进了深度学习的快速发展，但它依赖于大量的标记数据，与人类的学习方式还有很大差距。人脑可以以自组织和无监督的方式快速学习各种概念知识，这是通过协调人脑中的多个学习规则和结构来实现的。脉冲时间依赖性可塑性（STDP）是大脑中广泛存在的学习规则，但单独使用STDP训练的脉冲神经网络效率低且性能差。本文受短期突触可塑性的启发，设计了一种自适应突触滤波器，并引入自适应阈值平衡作为神经元可塑性，以丰富SNN的表达能力。我们还引入了自适应横向抑制连接来动态调整脉冲平衡，以帮助网络学习更丰富的特征。加快和稳定
英文摘要
The backpropagation algorithm has promoted the rapid development of deep learning, but it relies on a large amount of labeled data, and there is still a large gap with the way the human learns. The human brain can rapidly learn various concept knowledge in a self-organized and unsupervised way, which is accomplished through the coordination of multiple learning rules and structures in the human brain. Spike-timing-dependent plasticity (STDP) is a widespread learning rule in the brain, but spiking neural network trained using STDP alone are inefficient and performs poorly. In this paper, taking inspiration from the short-term synaptic plasticity, we design an adaptive synaptic filter, and we introduce the adaptive threshold balance as the neuron plasticity to enrich the representation ability of SNNs. We also introduce an adaptive lateral inhibitory connection to dynamically adjust the spikes balance to help the network learn richer features. To accelerate and stabilize the training of the unsupervised spiking neural network, we design a sample temporal batch STDP which update the weight based on multiple samples and multiple moments. We have conducted experiments on MNIST and FashionMNIST, and have achieved state-of-the-art performance of the current unsupervised spiking neural network based on STDP. And our model also shows strong superiority in small samples learning.邮件日期
2022年07月07日- Lottery Ticket Hypothesis for Spiking Neural Networks 时间：2022年07月04日 第一作者：Youngeun Kim 链接.
摘要：脉冲神经网络（SNN）最近作为新一代低功耗深度神经网络出现，其中二进制脉冲在多个时间步长上传递信息。当SNN部署在资源受限的移动/边缘设备上时，对SNN的修剪非常重要。以前的SNN修剪工作集中于浅SNN（2~6层），然而，最先进的SNN工作提出了更深的SNN（>16层），这很难与当前的修剪工作兼容。为了向深度SNN扩展剪枝技术，我们研究了彩票假设（LTH），该假设指出，密集网络包含较小的子网络（即中奖彩票），其性能与密集网络相当。我们对LTH的研究表明，中奖彩票始终存在于各种数据集和架构的深度SNN中，提供了高达97%的稀疏性，而不会出现巨大的性能下降。然而，LTH的迭代搜索过程带来了巨大的训练成本
英文摘要
Spiking Neural Networks (SNNs) have recently emerged as a new generation of low-power deep neural networks where binary spikes convey information across multiple timesteps. Pruning for SNNs is highly important as they become deployed on a resource-constraint mobile/edge device. The previous SNN pruning works focus on shallow SNNs (2~6 layers), however, deeper SNNs (>16 layers) are proposed by state-of-the-art SNN works, which is difficult to be compatible with the current pruning work. To scale up a pruning technique toward deep SNNs, we investigate Lottery Ticket Hypothesis (LTH) which states that dense networks contain smaller subnetworks (i.e., winning tickets) that achieve comparable performance to the dense networks. Our studies on LTH reveal that the winning tickets consistently exist in deep SNNs across various datasets and architectures, providing up to 97% sparsity without huge performance degradation. However, the iterative searching process of LTH brings a huge training computational cost when combined with the multiple timesteps of SNNs. To alleviate such heavy searching cost, we propose Early-Time (ET) ticket where we find the important weight connectivity from a smaller number of timesteps. The proposed ET ticket can be seamlessly combined with common pruning techniques for finding winning tickets, such as Iterative Magnitude Pruning (IMP) and Early-Bird (EB) tickets. Our experiment results show that the proposed ET ticket reduces search time by up to 38% compared to IMP or EB methods.注释
Accepted to European Conference on Computer Vision (ECCV) 2022邮件日期
2022年07月05日- Simple and complex spiking neurons: perspectives and analysis in a simple STDP scenario 时间：2022年06月28日 第一作者：Davide Liberato Manna 链接.
摘要：脉冲神经网络（SNN）在很大程度上受到生物学和神经科学的启发，并利用**和理论来创建快速高效的学习系统。脉冲神经元模型被用作神经形态系统的核心处理单元，因为它们支持基于事件的处理。通常采用集成和火灾（I&F）模型，其中最常用的是简单泄漏I&F（LIF）。采用这种模型的原因是它们的效率和/或生物学合理性。然而，在人工学习系统中采用LIF优于其他神经元模型的严格理由尚未研究。这项工作考虑了文献中的各种神经元模型，然后选择单变量、高效且显示不同类型复杂性的计算神经元模型。从这一选择中，我们对三种简单的I&F神经元模型，即LIF、二次I&F（QIF）和指数I&F，进行了比较研究，以了解是否使用
英文摘要
Spiking neural networks (SNNs) are largely inspired by biology and neuroscience and leverage ideas and theories to create fast and efficient learning systems. Spiking neuron models are adopted as core processing units in neuromorphic systems because they enable event-based processing. The integrate-and-fire (I&F) models are often adopted, with the simple Leaky I&F (LIF) being the most used. The reason for adopting such models is their efficiency and/or biological plausibility. Nevertheless, rigorous justification for adopting LIF over other neuron models for use in artificial learning systems has not yet been studied. This work considers various neuron models in the literature and then selects computational neuron models that are single-variable, efficient, and display different types of complexities. From this selection, we make a comparative study of three simple I&F neuron models, namely the LIF, the Quadratic I&F (QIF) and the Exponential I&F (EIF), to understand whether the use of more complex models increases the performance of the system and whether the choice of a neuron model can be directed by the task to be completed. Neuron models are tested within an SNN trained with Spike-Timing Dependent Plasticity (STDP) on a classification task on the N-MNIST and DVS Gestures datasets. Experimental results reveal that more complex neurons manifest the same ability as simpler ones to achieve high levels of accuracy on a simple dataset (N-MNIST), albeit requiring comparably more hyper-parameter tuning. However, when the data possess richer Spatio-temporal features, the QIF and EIF neuron models steadily achieve better results. This suggests that accurately selecting the model based on the richness of the feature spectrum of the data could improve the whole system's performance. Finally, the code implementing the spiking neurons in the SpykeTorch framework is made publicly available.邮件日期
2022年07月12日- Structural Stability of Spiking Neural Networks 时间：2022年06月21日 第一作者：G. Zhang 链接.
摘要：在过去的几十年中，由于对时间相关数据建模的巨大潜力，人们对脉冲神经网络（SNN）越来越感兴趣。已经开发了许多算法和技术；然而，对脉冲神经网络的许多方面的理论理解仍然模糊。最近的一项研究[Zhang等人，2021]揭示，由于其分叉动力学，典型SNN很难承受内部和外部扰动，并建议必须添加自连接。在本文中，我们研究了具有自连接的SNN的理论性质，并通过指定最大分岔解数的下界和上界来深入分析结构稳定性。在模拟和实际任务上进行的数值实验证明了所提出结果的有效性。
英文摘要
The past decades have witnessed an increasing interest in spiking neural networks (SNNs) due to their great potential of modeling time-dependent data. Many algorithms and techniques have been developed; however, theoretical understandings of many aspects of spiking neural networks are still cloudy. A recent work [Zhang et al. 2021] disclosed that typical SNNs could hardly withstand both internal and external perturbations due to their bifurcation dynamics and suggested that self-connection has to be added. In this paper, we investigate the theoretical properties of SNNs with self-connection, and develop an in-depth analysis on structural stability by specifying the lower and upper bounds of the maximum number of bifurcation solutions. Numerical experiments conducted on simulation and practical tasks demonstrate the effectiveness of the proposed results.邮件日期
2022年07月12日- Linear Leaky-Integrate-and-Fire Neuron Model Based Spiking Neural Networks and Its Mapping Relationship to Deep Neural Networks 时间：2022年05月31日 第一作者：Sijia Lu 链接.
摘要：脉冲神经网络（SNN）是一种受大脑启发的机器学习算法，具有生物学合理性和无监督学习能力等优点。先前的工作已经表明，将人工神经网络（ANN）转换为SNN是实现SNN的实用和有效的方法。然而，缺乏训练非精确损失SNN的基本原理和理论基础。本文建立了线性泄漏积分和火灾模型（LIF）/SNN的生物参数与ReLU AN/Deep神经网络（DNN）参数之间的精确数学映射。这种映射关系在一定条件下得到了分析证明，并通过模拟和实际数据实验进行了验证。它可以作为两类神经网络各自优点的潜在组合的理论基础。
英文摘要
Spiking neural networks (SNNs) are brain-inspired machine learning algorithms with merits such as biological plausibility and unsupervised learning capability. Previous works have shown that converting Artificial Neural Networks (ANNs) into SNNs is a practical and efficient approach for implementing an SNN. However, the basic principle and theoretical groundwork are lacking for training a non-accuracy-loss SNN. This paper establishes a precise mathematical mapping between the biological parameters of the Linear Leaky-Integrate-and-Fire model (LIF)/SNNs and the parameters of ReLU-AN/Deep Neural Networks (DNNs). Such mapping relationship is analytically proven under certain conditions and demonstrated by simulation and real data experiments. It can serve as the theoretical basis for the potential combination of the respective merits of the two categories of neural networks.邮件日期
2022年07月12日- Why we do need Explainable AI for Healthcare 时间：2022年06月30日 第一作者：Giovanni Cin`a 链接.
摘要：最近，用于医疗保健的人工智能（AI）认证工具激增，重新引发了关于采用这项技术的辩论。这种争论的一个线索涉及可解释的人工智能及其使人工智能设备更透明和更可信的承诺。一些活跃在医学人工智能领域的声音对可解释人工智能技术的可靠性表示担忧，质疑其使用和纳入指南和标准。回顾这些批评，本文就可解释人工智能的效用提供了一个平衡和全面的视角，重点关注人工智能临床应用的特殊性，并将其放在医疗干预的背景下。针对其批评者，尽管存在合理的担忧，我们认为可解释的人工智能研究项目仍然是人机交互的核心，最终是我们防止失控的主要工具，这种危险仅靠严格的临床验证是无法预防的。
英文摘要
The recent spike in certified Artificial Intelligence (AI) tools for healthcare has renewed the debate around adoption of this technology. One thread of such debate concerns Explainable AI and its promise to render AI devices more transparent and trustworthy. A few voices active in the medical AI space have expressed concerns on the reliability of Explainable AI techniques, questioning their use and inclusion in guidelines and standards. Revisiting such criticisms, this article offers a balanced and comprehensive perspective on the utility of Explainable AI, focusing on the specificity of clinical applications of AI and placing them in the context of healthcare interventions. Against its detractors and despite valid concerns, we argue that the Explainable AI research program is still central to human-machine interaction and ultimately our main tool against loss of control, a danger that cannot be prevented by rigorous clinical validation alone.邮件日期
2022年07月01日- CIRDataset: A large-scale Dataset for Clinically-Interpretable lung nodule Radiomics and malignancy prediction 时间：2022年06月29日 第一作者：Wookjin Choi 链接.
摘要：针状/分叶状、肺结节表面尖锐/弯曲的脉冲是肺癌恶性的良好预测因子，因此，放射科医生定期评估和报告，作为标准化肺RADS临床评分标准的一部分。考虑到结节的三维几何形状和放射科医生逐层二维评估，手动针状/分叶注释是一项繁琐的任务，因此目前还没有公共数据集用于探讨这些临床报告特征在SOTA恶性肿瘤预测算法中的重要性。作为本文的一部分，我们发布了一个大规模临床可解释的放射组学数据集CIRDataset，其中包含来自两个公共数据集LIDC-IDRI（N=883）和LUNGx（N=73）的956个放射科医生对分割肺结节的QA/QC'ed针状/分叶注释。我们还提出了一种基于多类体素网格扩展的端到端深度学习模型，用于分割结节（同时保留脉冲），分类脉冲（尖锐/脉冲和弯曲/分叶状），并进行恶性肿瘤预测。以前的方法已经对LIDC和LUNGx数据集进行了恶性肿瘤预测，但没有对任何临床报告/可操作的特征进行可靠归因（由于一般归因方案存在已知的超参数敏感性问题）。随着这一全面注释的CIRDataset和端到端深度学习基线的发布，我们希望恶性肿瘤预测方法能够验证其解释，对照我们的基线进行基准测试，并提供临床可操作的见解。数据集、代码、预训练模型和docker容器可在https://github.com/nadeemlab/CIR.
英文摘要
Spiculations/lobulations, sharp/curved spikes on the surface of lung nodules, are good predictors of lung cancer malignancy and hence, are routinely assessed and reported by radiologists as part of the standardized Lung-RADS clinical scoring criteria. Given the 3D geometry of the nodule and 2D slice-by-slice assessment by radiologists, manual spiculation/lobulation annotation is a tedious task and thus no public datasets exist to date for probing the importance of these clinically-reported features in the SOTA malignancy prediction algorithms. As part of this paper, we release a large-scale Clinically-Interpretable Radiomics Dataset, CIRDataset, containing 956 radiologist QA/QC'ed spiculation/lobulation annotations on segmented lung nodules from two public datasets, LIDC-IDRI (N=883) and LUNGx (N=73). We also present an end-to-end deep learning model based on multi-class Voxel2Mesh extension to segment nodules (while preserving spikes), classify spikes (sharp/spiculation and curved/lobulation), and perform malignancy prediction. Previous methods have performed malignancy prediction for LIDC and LUNGx datasets but without robust attribution to any clinically reported/actionable features (due to known hyperparameter sensitivity issues with general attribution schemes). With the release of this comprehensively-annotated CIRDataset and end-to-end deep learning baseline, we hope that malignancy prediction methods can validate their explanations, benchmark against our baseline, and provide clinically-actionable insights. Dataset, code, pretrained models, and docker containers are available at https://github.com/nadeemlab/CIR.注释
MICCAI 2022邮件日期
2022年07月01日- The Case for RISP: A Reduced Instruction Spiking Processor 时间：2022年06月28日 第一作者：James S. Plank 链接.
摘要：本文介绍了精简指令脉冲处理器RISP。虽然大多数脉冲神经处理器基于大脑，或来自大脑的概念，但我们提出了一种简化而非复杂的脉冲处理器。因此，它具有离散集成周期、可配置泄漏等特点。我们提出了RISP的计算模型，并强调了其简单性的优点。我们演示了它如何帮助开发用于简单计算任务的手工构建的神经网络，详细介绍了如何使用它来简化使用更复杂的机器学习技术构建的神经网络，并演示了它的性能如何与其他脉冲神经处理器类似。
英文摘要
In this paper, we introduce RISP, a reduced instruction spiking processor. While most spiking neuroprocessors are based on the brain, or notions from the brain, we present the case for a spiking processor that simplifies rather than complicates. As such, it features discrete integration cycles, configurable leak, and little else. We present the computing model of RISP and highlight the benefits of its simplicity. We demonstrate how it aids in developing hand built neural networks for simple computational tasks, detail how it may be employed to simplify neural networks built with more complicated machine learning techniques, and demonstrate how it performs similarly to other spiking neurprocessors.注释
5 pages, 5 figures邮件日期
2022年06月29日- Short-Term Plasticity Neurons Learning to Learn and Forget 时间：2022年06月28日 第一作者：Hector Garcia Rodriguez 链接.
摘要：短时可塑性（STP）是一种在大脑皮层突触中储存衰退记忆的机制。在计算实践中，虽然理论预测STP是某些动态任务的最佳解决方案，但它主要用于脉冲神经元的小生境。在这里，我们提出了一种新型的递归神经单元，STP神经元（STPN），它确实非常强大。其关键机制是突触具有一种状态，通过突触内的自循环连接在时间中传播。这种公式可以通过时间反向传播来训练可塑性，从而形成一种短期内学会学习和忘记的形式。STPN优于所有测试的替代方案，即RNN、LSTM、其他具有快速权重和可微塑性的模型。我们在监督学习和强化学习（RL）以及联想检索、迷宫探索、雅达利视频游戏和MuJoCo机器人等任务中都证实了这一点。此外，我们计算出，在神经形态或生物电路中，STPN最大限度地减少了跨模型的能量消耗，因为它动态抑制了单个突触。基于这些，生物STP可能是一个强大的进化吸引子，可以最大限度地提高效率和计算能力。STPN现在也为广泛的机器学习实践带来了这些神经形态的优势。代码位于https://github.com/NeuromorphicComputing/stpn
英文摘要
Short-term plasticity (STP) is a mechanism that stores decaying memories in synapses of the cerebral cortex. In computing practice, STP has been used, but mostly in the niche of spiking neurons, even though theory predicts that it is the optimal solution to certain dynamic tasks. Here we present a new type of recurrent neural unit, the STP Neuron (STPN), which indeed turns out strikingly powerful. Its key mechanism is that synapses have a state, propagated through time by a self-recurrent connection-within-the-synapse. This formulation enables training the plasticity with backpropagation through time, resulting in a form of learning to learn and forget in the short term. The STPN outperforms all tested alternatives, i.e. RNNs, LSTMs, other models with fast weights, and differentiable plasticity. We confirm this in both supervised and reinforcement learning (RL), and in tasks such as Associative Retrieval, Maze Exploration, Atari video games, and MuJoCo robotics. Moreover, we calculate that, in neuromorphic or biological circuits, the STPN minimizes energy consumption across models, as it depresses individual synapses dynamically. Based on these, biological STP may have been a strong evolutionary attractor that maximizes both efficiency and computational power. The STPN now brings these neuromorphic advantages also to a broad spectrum of machine learning practice. Code is available at https://github.com/NeuromorphicComputing/stpn注释
Accepted at ICML 2022 Journal-ref: Proceedings of the 39th International Conference on Machine Learning, Baltimore, Maryland, USA, PMLR 162, 2022邮件日期
2022年06月29日- Learning to learn online with neuromodulated synaptic plasticity in spiking neural networks 时间：2022年06月28日 第一作者：Samuel Schmidgall 链接.
邮件日期
2022年06月29日- Energy-efficient Knowledge Distillation for Spiking Neural Networks 时间：2022年06月27日 第一作者：Dongjin Lee 链接.
注释
The manuscript was withdrawn because it contains inappropriate content for posting邮件日期
2022年06月28日- Gradient-based Neuromorphic Learning on Dynamical RRAM Arrays 时间：2022年06月26日 第一作者：Peng Zhou 链接.
摘要：我们提出了MEMprop，即采用基于梯度的学习来训练全记忆脉冲神经网络（MSNN）。我们的方法利用固有的器件动态来触发自然产生的电压脉冲。忆阻动力学发出的这些脉冲本质上是模拟的，因此是完全可微的，这消除了对脉冲神经网络（SNN）文献中流行的替代梯度方法的需要。忆阻神经网络通常要么将忆阻器集成为映射离线训练网络的突触，要么依赖联想学习机制来训练忆阻神经元网络。相反，我们将时间反向传播（BPTT）训练算法直接应用于记忆神经元和突触的模拟SPICE模型。我们的实现是完全记忆的，因为突触权重和脉冲神经元都集成在电阻RAM（RRAM）阵列上，而不需要额外的电路来实现脉冲动态，例如模数转换器（ADC）或阈值比较器。因此，高阶电生理效应被充分利用，以在运行时使用记忆神经元的状态驱动动力学。通过转向基于非近似梯度的学习，我们在之前报告的轻量级密集完全多通道神经网络中获得了在多个基准上具有高度竞争力的准确性。
英文摘要
We present MEMprop, the adoption of gradient-based learning to train fully memristive spiking neural networks (MSNNs). Our approach harnesses intrinsic device dynamics to trigger naturally arising voltage spikes. These spikes emitted by memristive dynamics are analog in nature, and thus fully differentiable, which eliminates the need for surrogate gradient methods that are prevalent in the spiking neural network (SNN) literature. Memristive neural networks typically either integrate memristors as synapses that map offline-trained networks, or otherwise rely on associative learning mechanisms to train networks of memristive neurons. We instead apply the backpropagation through time (BPTT) training algorithm directly on analog SPICE models of memristive neurons and synapses. Our implementation is fully memristive, in that synaptic weights and spiking neurons are both integrated on resistive RAM (RRAM) arrays without the need for additional circuits to implement spiking dynamics, e.g., analog-to-digital converters (ADCs) or thresholded comparators. As a result, higher-order electrophysical effects are fully exploited to use the state-driven dynamics of memristive neurons at run time. By moving towards non-approximate gradient-based learning, we obtain highly competitive accuracy amongst previously reported lightweight dense fully MSNNs on several benchmarks.邮件日期
2022年06月28日- Learning to learn online with neuromodulated synaptic plasticity in spiking neural networks 时间：2022年06月25日 第一作者：Samuel Schmidgall 链接.
摘要：我们提出，为了利用我们对神经科学的理解来进行机器学习，我们必须首先拥有强大的工具来训练类似大脑的学习模型。虽然在理解大脑学习动态方面取得了实质性进展，但神经科学衍生的学习模型尚未证明与梯度下降等深度学习方法具有相同的性能。受使用梯度下降的机器学习成功的启发，我们证明了神经科学中的神经调制突触可塑性模型可以在脉冲神经网络（SNN）中训练，其框架是通过梯度下降学习，以解决具有挑战性的在线学习问题。该框架为开发神经科学启发的在线学习算法开辟了一条新途径。
英文摘要
We propose that in order to harness our understanding of neuroscience toward machine learning, we must first have powerful tools for training brain-like models of learning. Although substantial progress has been made toward understanding the dynamics of learning in the brain, neuroscience-derived models of learning have yet to demonstrate the same performance capabilities as methods in deep learning such as gradient descent. Inspired by the successes of machine learning using gradient descent, we demonstrate that models of neuromodulated synaptic plasticity from neuroscience can be trained in Spiking Neural Networks (SNNs) with a framework of learning to learn through gradient descent to address challenging online learning problems. This framework opens a new path toward developing neuroscience inspired online learning algorithms.邮件日期
2022年06月28日- Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks 时间：2022年06月23日 第一作者：Alex Vicente-Sola 链接.
注释
17 pages, 6 figures, 17 tables ACM-class: I.2.6; I.2.10; I.4.8; I.5.2; D.2.13邮件日期
2022年06月24日- Artificial optoelectronic spiking neuron based on a resonant tunnelling diode coupled to a vertical cavity surface emitting laser 时间：2022年06月22日 第一作者：Mat\v{e}j Hejda 链接.