Parameters for producing the comparison graph of Figure 3.

Parameter is the learning rate, turns the model from a strict policy gradient rule to naive Hebbian, is the time constant used to estimate the firing rate of the action cells, is the time constant of the eligibility trace, is the reward baseline, the width of the averaging window of the reward, is the height of the postsynaptic pulse produced by the arrival of a spike and determines the width of the threshold region (escape noise). For C–E integration stops as soon as the total mean firing rate of all action cells , calculated by , see equation (15), exceeds 200 spikes/ms, i.e. the activity bump is well formed. For panels where two alternative parameter sets are given, both sets give very similar results, and hence we only depict one of them.

各参数的定义如下：学习率，可将模型从严格的策略梯度规则转化为朴素赫布学习规则；用于估计动作神经元放电率的时间常数；资格迹的时间常数；奖励基线；奖励平均窗口的宽度；由尖峰抵达时产生的突触后脉冲峰值幅度，该参数同时决定了阈值区域的宽度（逃逸噪声）。针对C-E场景，当所有动作神经元的总平均放电率（通过公式(15)计算得到）超过200尖峰/毫秒时，积分过程将立即停止，此时尖峰活动簇已充分形成。对于包含两组可选参数的子图，两组参数均可得到极为相近的结果，因此我们仅绘制其中一组即可。