Parameters for producing the comparison graph of Figure 3.

Parameter is the learning rate, turns the model from a strict policy gradient rule to naive Hebbian, is the time constant used to estimate the firing rate of the action cells, is the time constant of the eligibility trace, is the reward baseline, the width of the averaging window of the reward, is the height of the postsynaptic pulse produced by the arrival of a spike and determines the width of the threshold region (escape noise). For C–E integration stops as soon as the total mean firing rate of all action cells , calculated by , see equation (15), exceeds 200 spikes/ms, i.e. the activity bump is well formed. For panels where two alternative parameter sets are given, both sets give very similar results, and hence we only depict one of them.