A Reinforcement-Learning Model of Active Avoidance in Wistar-Kyoto and Sprague Dawley Rats: Experimental Dataset with Model Code and Output

This study used a reinforcement learning (RL) model to examine strain differences in rats learning an active avoidance task. The empirical data were previously published in Spiegler et al. 2019; preprocessed data are included here. In brief, 40 male Wistar-Kyoto (WKY) and 40 male Sprague Dawley (SD) rats were trained to make an operant lever press response to avoid or escape from aversive footshocks. Responses made during a danger period that preceded shock onset were scored as avoidance responses and averted the shock; responses made during the shock period were scored as escape responses and terminated the shock. Each rat received 12 acquisition sessions of 25 trials each. For the current study, raw data from the empirical study were preprocessed by discretizing data from each rat (SD rats S09-S48 and WKY rats W09-W48) to 12-second periods (“timesteps”), and noting presence or absence of experimental stimuli (danger signal, safety signal, and/or shock), the context (experimental chamber or home cage), and whether the rat emitted at least one lever press within that timestep. These can be found in the “ratRL_datafiles_Model_InputOutput” folder (e.g. S09.csv, S10.csv, etc.). A reinforcement learning (RL) model was applied to these preprocessed empirical data to estimate model parameters for each rat that best reproduced that rat’s behavior. Code can be found in the “ratRL_ModelFitting_Code” folder. The program estimates model parameters and also generates a summary file for each rat (e.g. S09sum.csv, S10sum.csv, etc.) noting whether an escape or avoidance response occurred on each trial. Given estimated parameter values for each rat, behavioral recovery simulations were conducted, by constructing “simulated rats” based on the parameter values, which were then trained on the avoidance acquisition task. For each simulated rat, results were averaged over 100 runs. Code can be found in the “ratRL_Simulation_Code” folder. Example inputs to this simulation are found in the “ratRL_codedTrials_Sim_InputOutput” folder, and include both a list of parameter values for each simulated rat (“parm_listfile.csv”) as well as the summary file for each rat (e.g. “S09sum.csv”) generated by the model fitting program. The results of the parameter estimation can be used to examine differences within and across strains; for example, whether there are differences in the estimated value of a particular parameter, such as the subjective reinforcement value of shock, for SD vs. WKY rats. The results of the behavioral recovery simulations can be analyzed in the same way as empirical data; for example, after averaging across 100 simulation runs to get an average learning curve for each rat, the performance of simulated rats of each strain can be compared to determine if there is a main effect of trial/session (indicating learning), a main effect of strain (e.g. simulated WKY show more avoidance responses than simulated SD), and/or an interaction.