1.

INTRODUCTION

Spiking neural networks (SNNs) establish interoperability properties at the artificial neural network level by simulating the communication mechanism of firing spike trains in real time^1,2. Biological neuron signals can be converted into continuous functions by SNNs^2,3, which are widely used for spatial-temporal data analysis in research fields. Several categories of SNNs architecture have been developed in recent years based on different applicability, including single-layer SNNs, multi-layer feed-forward SNNs, and recurrent SNNs. Compared with the other two network structures, recurrent SNNs (RSNNs) are more biologically realistic, and it has the ability to simulate the actual biological activities of biological pulse issuance, transmission, processing and feedback in real time. At the same time, RSNNs are the artificially constructed structures that are closest to the real biological complex network node links. An artificial neural network with recurrent connections is a type of machine learning model with a feedback loop structure that can be used to assist in solving parallel processes, such as machine automation⁴, intelligent recognition⁵, and textual analysis⁶.

A key component of learning and memory is synaptic plasticity. As part of his initial synaptic plasticity hypothesis, Hebb presented that synergistic activity between presynaptic and postsynaptic neurons played a vital role in increasing synaptic potentiation⁷. With the in-depth exploration of researchers, excellent results have emerged, including strengthening synaptic weights using spike-timing dependent plasticity (STDP)⁸, weakening them using anti-STDP⁹, threshold driven plasticity rules (TDP)¹⁰, precise-spike-driven synaptic plasticity¹¹ and somato-dendritic synaptic plasticity¹². Consequently, the supervised synaptic plasticity learning mechanism, such as STDP and TDP, missing the characteristic of the delay in the propagation of impulse signals in biological synapses, cannot be directly used to express the biological communication mechanism of RSNNs. From this point of view, it focuses more on the exploration of neural dynamics rather than just analyzing the macro variables of neural networks¹³. On the basis of the comparative study, scientists do not emphasize the importance of developing synaptic weights between neuronal connections, which calls for developing sophisticated algorithms for RSNNs.

The aim of this paper is to propose a supervised learning rule based on synaptic delay plasticity with a spike train weighting adjustment operation in all layers, which can improve the learning ability for complex spatiotemporal patterns. The update method of synaptic weights is defined in combination with the synaptic delay, and then a supervised learning rule is constructed to realize the spatiotemporal communication of spike train. In Section 2 we mainly introduce the network structure of RSNNs and the simplified neural information processing flow by introducing the context layer. In Section 3 we give the construction process of learning rules for each layer of the network. In Section 4 we show the learning performance of the spike train and the effect of different element variations. A summary of the work in paper is presented in Section 5.

2.

RELATED WORK

This section mainly describes the linear network structure and conceptual network introduction of RSNNs in detail. Besides, spike train function is introduced through mathematical expressions.

2.1

The network structure

In this study, a multi-layer spiking neural network is recursively constructed using random coupling and intrinsic connectivity. As shown in Figure 1, the network structure consists of three main layers, namely input, hidden and output, and each layer consists of neuronal connections. It is obvious that the processing process between the layers for spike train is a sequential recursive feed-forward structure, while the randomly coupled real-time communication of the neurons within the hidden layers forms recurrent connections. To achieve a recurrent connection approach as shown in Figure 1 by the dotted line, a connectivity degree (C_d) is applied to indicate the tightness of the coupling within the hidden neurons. In fact, the hidden neurons are connected in an intricate way, which is more reflected by random coupling, so we just artificially abstract to facilitate the construction of network model.

Figure 1.

The constructed network structure.

As mentioned above, RSNNs are complex in structure, so it is difficult to develop an algorithm for spike train learning in terms of synaptic delay-weight plasticity. In order to follow the structural principle and facilitate program design, a copy of the recurrent layer (called the context layer) is placed at the bottom with the aim of converting RSNNs into an equivalent feed-forward structure, as shown in Figure 2.

Figure 2.

The network structure conversion with context layer.

2.2

The spike train function

After the biological neuron is stimulated by the presynaptic a spike train s_i, it will generate the stress response output spike train s_o. A continuous-time linear Poisson neuron model is used to establish the relationship between input and output spike trains, which can be shown as:

where the number of presynaptic neurons is denoted by A, and an important part of the spike train processing between neuron i and another neuron o is the synaptic weight w_oi. Based on the results in References^14,15, learning rules can be constructed using a simplified linear summation mechanism of smooth spike trains.

In addition, a smoothing function h is chosen for mathematical convenience, which allows discrete spike trains to be converted into continuous functions. On R₂(ϖ) space, s_i and s_j are defined as follows¹⁶:

where an autocorrelation between smoothing function u and the kernel function Θ, and the kernel can be defined as Θ(t^m,tⁿ) = ∫_Γu(t – t^m)u(t – tⁿ) dt.

3.

SUPERVISED LEARNING ALGORITHM WITH SYNAPTIC DELAY-WEIGHT PLASTICITY

In this case, we can define , as the desired spike train, and , as the actual spike train, respectively. By combining equations (1) and (2), we can interpret the total error of RSNNs at time t as the error train in output spikes:

For all output neurons, the error function can be transformed into the inner product of spike trains, facilitating the adjustment of real-time spike train for an efficient learning process.

By backpropaguing network errors to synaptic weights through error backpropagation, a generalized delta update rule is used to modify synaptic weights. As a result of synaptic connections, the synaptic weight w is calculated as follows:

where η is the learning rate, ∇E is the error function for adjusting the learning efficiency of spike train, and d_i is synaptic delay, which can be expressed as:

where d_ϕ is the synapse ϕ that need to be adjusted, and denotes the distance between the time interval when the maximum postsynaptic potential occurs between the input and the desired after the generation of spike trains, which can be represented as:

If , the synaptic delay d_i is assigned as . Otherwise, d_i is assigned as The mathematical expression for the gradient ∇E is:

As demonstrated in Figure 2, synaptic weight learning rules for the network layer can be separated into three layers: output, context, and input.

3.1

Learning rule in output layer (N_O)

Using equation (7) and the corresponding partial derivative transformation operation, the gradient ∇E_oh is associated with the recurrent neurons at time t, which can be shown:

Accordingly, the rule for updating the synaptic weight ∆w_oh based on the inner products of adjacent layer relationships and spike train conversion transitions is:

where the spike train counts of the actual and the expected are denoted by and , both as continuous values.

3.2

Learning rule in context layer (N_C)

From equation (7) and the partial derivative operation, the gradient ∇E_hr is associated with the recurrent neurons at time t, which can be represented as:

The rule for updating the synaptic weight ∆w_hr of adjacent layer relationships and spike train conversion transitions is:

3.3

Learning rule in input layer (N_I)

In the same way, the gradient ∇E_oh is associated with the recurrent neurons at time t, which can be show as:

Similarly, the rule of the synaptic weight ∆w_hi updated is:

Finally, a summary of the learning rules for synaptic weights is given in equations (9), (11), and (13) corresponding to each layer of the RSNNs.

4.

EXPERIMENTS

This section shows the learning capabilities of RSNNs when trained with a supervised learning algorithm with spike weights. The learning algorithm is set up by defining the initial parameters to facilitate testing and verification, and then the emulating performance of spike train based on learning rules is analyzed.

4.2

Elemental analysis

Figure 3 shows the simulation threads of spike train based learning rules with synaptic delay. Figure 3a illustrates an overview of the learning process of the pulse sequence, which is divided into three phases: initial, desired and actual. Through supervised learning over about 250 epochs, the learning algorithm produces the desired output spike train in 150 ms intervals based on the initial output spike train. As shown in Figure 3b, the total network error E evolves over time. Following 250 learning epochs, the error function drops to 0 as the network is iteratively learned. This means that the spike train achieves the actual-to-expected identicality.

Figure 3.

The simulation threads of spike train based learning rules with synaptic delay for RSNNs. (a): The efficient iterative process. Δ, ∇ and ∙ all denote spike trains, each specific to the initial, desired and actual; (b): The errors on the network evolve over time.

Figure 4 shows the variation of network errors based on desired output spike train of different lengths. Figure 4a illustrates that the optimal expected output spike train length of the algorithm is 150 ms, then high-precision autonomous learning can be performed, and the error increases significantly with the length of spike train. It is intuitively evident from Figure 4b that the learning epoch number for the algorithm with synaptic delay incorporated is about 250 epochs. The main reason is that the occurrence of negative feedback regarding learning accuracy and spike train length, where the error rises significantly when the length increases.

Figure 4.

The spike trains with varying lengths when iterating through 500 learning epochs. (a): The differences in network errors due to spike train lengths; (b): The effect of spike train lengths on errors and learning epochs.

Figure 5 shows the variation of network errors based on desired output spike train of different frequencies. Figure 5a illustrates that the network error grows significantly as the frequency of desired output spike train increases, and the algorithm can perform high-precision autonomous learning when the output frequency is set to 50 Hz. From Figure 5b, the learning epoch of constructed rules is approximately in equilibrium as output spike frequency varies.

Figure 5.

The spike trains with varying frequencies when iterating through 500 learning epochs. (a): The differences in network errors due to spike train frequencies; (b): The effect of spike train frequencies on errors and learning epochs.

5.

CONCLUSIONS

In this work, we propose a multi-spike learning rule with synaptic delay-weight plasticity based on weighting adjustment operation of spike train for RSNNs. The introduction of synaptic delay makes the adjustment of synaptic weights more similar to the processing process of biological neurons for spike train. The synaptic weight update rules combined with synaptic delay are given mathematical representations on N_O, N_C, and N_I, respectively. The high-precision experimental results are obtained in the learning process by integrating the learning algorithm with synaptic delay, including the analysis of the optimal learning performance for spike train, the influence of different spike train lengths, and the change of the different output frequency for desired spike train. Therefore, the updating of synaptic weights based on the delay characteristic may be utilized in various neuron learning rules, neuron models and complex network structures.