Data from: Simple visual stimuli are sufficient to drive responses in action observation and execution neurons in the macaque ventral premotor cortex

# Simple visual stimuli are sufficient to drive responses in action observation and execution neurons in the macaque ventral premotor cortex This README file was generated on 2024-04-04 by Sofie De Schrijver ## GENERAL INFORMATION 1. Title of Dataset: Dataset for 'Simple visual stimuli are sufficient to drive responses in action observation and execution neurons in the macaque ventral premotor cortex' 2. Author Information Principal Investigator (Research Facilitites) Information Name: Peter Janssen ORCID:0000-0002-8463-5577 Institution: KU Leuven Address: Laboratory for Neuro- and Psychophysiology Department of Neurosciences ON2 - Herestraat 49 3000 Leuven (Belgium) Email: [[e-mail]](mailto:[e-mail]) First Author Information Name: Sofie De Schrijver ORCID: 0000-0002-9063-8790 Institution: KU Leuven Address: Laboratory for Neuro- and Psychophysiology Department of Neurosciences ON2 -Herestraat 49 3000 Leuven (Belgium) Email: [[e-mail]](mailto:[e-mail]) 3. Date of data collection: 2019-02-21 to 2023-05-11 4. Geographic location of data collection: Leuven/Flemish Brabant, Belgium 5. Information about funding sources that supported the collection of the data: Fonds Wetenschappelijk Onderzoek (FWO): G.097422N KU Leuven: C14/18/100 KU Leuven: C14/22/134 The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication. ## SHARING/ACCESS INFORMATION 1. Licenses/restrictions placed on the data: N/A 2. Links to publications that cite or use the data: N/A yet 3. Links to other publicly accessible locations of the data: N/A 4. Links/relationships to ancillary data sets: N/A Was data derived from another source? No 5. Recommended citation for this dataset: De Schrijver S; Decramer T; Janssen Peter (2024), Simple visual stimuli are sufficient to drive responses in action observation and execution neurons in the macaque ventral premotor cortex. ## DATA & FILE OVERVIEW 1. Description of dataset The following .mat files correspond to the filtered data acquired along the study and organized per figure. The name of each file includes the task, the animal name, date of collection and hour of collection. Monkey1 = Isaac, Monkey2 Right = Sky1, Monkey2 Left = Sky2, Monkey3 = Loki, Monkey4 = Vino For each .mat file, there is a corresponding .nex5 file that contains the sorted spikes. For files that were generated after 2020, there is an extra .mat file named *_ReplayedMUA.mat. This file contains the complete multi-unit activity, since only a part was recorded and stored in the original .mat file due to a change in the settings during recording of the signal. Addionally, .mat files were generated starting from June 2020 with the name 'mgrasp_dark'. This task refers to the same grasping task but performed in the dark. This data was not used to generate figures. 1. File List: Figure 2 and supplementary Figures 1,5: Isaac_vgrasp_20190221_1121_B Isaac_vgrasp_20190225_1101_B Isaac_vgrasp_20190226_1052_D vgrasp_Sky1_20200107_1143_C vgrasp_Sky1_20200109_1032_A vgrasp_Sky1_20200110_1007_A vgrasp_Sky2_20200615_1015_A vgrasp_Sky2_20200616_0925_A vgrasp_Sky2_20200617_0924_A vgrasp_Loki_20210525_1000_B vgrasp_Loki_20210528_0929_A vgrasp_Loki_20210531_0938_A vgrasp_Vino_20221014_0955_A vgrasp_Vino_20221017_0940_A vgrasp_Vino_20221027_1052_B Figure 3: Isaac_mirrormovies_20190226_1059_A Figures 4,5,6,7 and supplementary Figures 1,2,3,4,5: Isaac_mirrormovies_20190221_1127_A Isaac_mirrormovies_20190225_1107_B Isaac_mirrormovies_20190226_1059_A mirrormovies_Sky1_20200107_1103_B mirrormovies_Sky1_20200109_1044_A mirrormovies_Sky1_20200110_1038_B mirrormovies_Sky2_20200615_1048_A mirrormovies_Sky2_20200616_0958_A mirrormovies_Sky2_20200617_1006_A mirrormovies_Loki_20210525_1037_B mirrormovies_Loki_20210528_0950_A mirrormovies_Loki_20210531_1004_A mirrormovies_Vino_20221014_1028_A mirrormovies_Vino_20221017_1014_A mirrormovies_Vino_20221027_1127_A Supplementary Figure 3: mirrormovies_Loki_20230511_0940_A ## METHODOLOGICAL INFORMATION 1. Description of methods used for collection/generation of data Surgery and Recording Procedures: Four male rhesus monkey (Macaca mulatta, 8 kg) were implanted with a titanium head post that was fixed to the skull with dental acrylic and titanium screws. After training in a passive fixation task and a grasping task, a 96-channel microelectrode Utah array with 1.5 mm electrode length and an electrode spacing of 400 µm (4x4 mm; Blackrock Neurotech, UT, USA) was inserted during general anesthesia and guided by stereotactic coordinates and anatomical landmarks. We inserted the arrays using a pneumatic inserter (Blackrock Neurotech) with a pressure of 1.034 bar and an implantation depth of 1 mm. During all surgical procedures, the monkey was kept under propofol anesthesia (10 mg/kg/h) and strict aseptic conditions. Postoperative anatomical scans (Siemens 3T scanner, 0.6 mm resolution) verified the position of the Utah array in ventral premotor area F5c (Figure 1A top left panel), contralateral to the monkey’s working hand. The remaining three panels of Figure 1A show the exact location of the five implantations in the four monkeys. The posterior edge of the 4x4 mm arrays was located 0 to 5 mm anterior to a vertical line extending down from the spur of the arcuate sulcus. Thus, our recording sites covered a substantial part of the inferior frontal convexity. Due to an implant failure in the second monkey, another Utah array was implanted in area F5c in the other hemisphere (‘Monkey 2 Left’ in Figure 1A). All surgical and experimental procedures were approved by the ethical committee on animal experiments of the KU Leuven and performed according to the National Institute of Health’s Guide for the Care and Use of Laboratory Animals and the EU Directive 2010/63/EU. During a recording session, data were collected using a 96-channel digital headstage (Cereplex M) connected to a digital neural processor and sent to a Cerebus data acquisition system (Blackrock Neurotech, UT, USA). Single- and multiunit signals were high pass filtered (750 Hz) and sampled at 30 kHz. The threshold to detect multiunit activity was set to 95% of the average noise level. The data were subsequently sorted offline with a refractory period of 1 ms to isolate single units, using the Offline Spike sorter software (Plexon, Inc., Dallas, TX, USA). Overall, we had a good yield over the implanted array in each monkey with approximately 60 channels with detectable single units in every session. Experimental setup: The monkey was trained to sit upright in a primate chair with his head fixed during all experimental procedures. During the recording of neuronal activity, the monkey had to perform two different tasks sequentially in blocks (typically 20 trials per condition): a grasping task and a passive fixation task. For the grasping task, a custom-built object containing three identical small spheres was placed in front of the monkey at a 28 cm viewing distance 55. The spheres (2.5 cm diameter) were attached to a disk (15 cm diameter) with springs, allowing the monkey to pull the spheres. Each sphere contained a blue LED that could be turned on and off individually, and was positioned at an angle of 120 degree relative to the other two spheres. In the center of the disk, a green LED served as the fixation point, and the dimming of this green LED was the go-signal for grasping. During the grasping task (Figure 1B), the monkey had to grasp one of the three identical spheres (indicated with the blue LED) in a pseudorandom order. The position of the hand was monitored using infrared laser beams, which were interrupted when the hand was positioned on the resting position. An infrared-based camera system (Eyelink 1000; SR Research, Ontario, Canada) monitored the eye movements to ensure fixation on the object during the duration of each trial. For the passive fixation task, a display (17.3 inch) was placed in front of the monkey at the same viewing distance as the object in the grasping task. The monkey had to maintain fixation on a red dot in the center of the screen during the presentation of different videos. Eye movements were monitored to ensure fixation inside a ~2 degree fixation window using an the same infrared-based camera system as in the VGG task. A photodiode attached to the lower right corner of the screen registered the onset of each video by the detection of a bright square (not visible to the monkey) that appeared simultaneously with the onset of the video. Photodiode pulses were sampled at 30 kHz on the Cerebus data acquisition system to allow synchronization with the neural data. Visually guided and memory-guided grasping task (VGG and MGG): In every recording session, the monkey had to perform a delayed visually guided reach-to-grasp task (Figure 1B). To start a trial, the monkey had to place its hand on a resting position in complete darkness. After 500 ms of fixation on a green LED in the center of the disk, an external light illuminated the object. At the same time, a blue LED appeared on one of the three spheres, indicating the sphere to-be-grasped. After a variable time (700-1000 ms), the green LED dimmed (i.e. the go cue), instructing the monkey to release the resting position, grasp the object with the illuminated blue LED, and pull it to obtain a juice reward. The complete movement, from releasing the resting position to pulling the object, could maximally last 1000 ms to ensure the shortest and most efficient reach trajectory. During the grasping task, the opposite hand was gently restrained to avoid movement. In the memory-guided version of the grasping task (MGG), all events were identical to the VGG task, with the exception of the light above the object and the blue LED on the target, which both went off after 300 ms, so that the animal had to grasp and pull the object in the dark after a delay of 700 to 1000 ms. To avoid any influence of the reward on the activity around the pull of the object, reward was administered at least 40 ms after the detection of the pull of the object. Action observation task: To initiate a trial, the monkey had to fixate on a small red dot that appeared in the center of the screen. After 300 ms of passive fixation, a video started. The monkey had to maintain its gaze on the fixation point during the presentation of the stimulus (15.5 x 10.3 visual degrees). In total, twelve conditions (one video per condition) were shown in pseudorandom order during the task (Figure 1C, supplemental material). In brief, the stimulus set included six videos filmed from the point of view (Viewpoint 1) of the monkey and six videos filmed from the side (Viewpoint 2). Both viewpoints included a monkey and a human performing the VGG task (‘Monkey grasp’ and ‘Human grasp’, respectively), a human performing the same task without pulling the sphere (‘Human touch’) and a video without any movement and no human or monkey hand visible (‘Static’). After pulling the sphere, the hand moved back to the starting position. Additionally, four videos were shown in which an ellipse (a scrambled version of the monkey hand, major axis ± 95 mm, minor axis ± 43 mm, as in 12) moved towards the object with the same kinetic parameters as the hand in the action videos. Since the period before movement onset in the ellipse videos differed from the one in the action videos, the total length of the ellipse videos was slightly different compared to the action videos. The background of the video was either the natural (‘Ellipse’) or a scrambled version of the natural background (‘SCR background’). We also presented videos of a static frame of the Human Grasp action video in which the hand was halfway towards the object and in which the hand was interacting with the object. All videos were made using the object with the three spheres from the grasping task and lasted between 2.6 and 3.5 sec. Both arms of the monkey were restrained during the action observation task to prevent movement. 1. Methods for processing the data All data were analyzed using custom written Matlab scripts (the MathWorks R2019b, MA, USA). For each trial, we calculated the net spike rate in 50 ms bins by subtracting the baseline activity (average spike rate of 200 ms interval before object onset or before onset of the action video) from the spike rate after stimulus start (either object or action video). We analyzed three recording sessions for each implantation. Because the recording signal was unstable in the first weeks after implantation, we considered all spikes recorded on different days as different units. However, we verified that the results were essentially the same when analyzing a single recording session for each of the three implantations. All analyses were calculated on SUA and MUA independently, and averaged across the three spheres that had to be grasped. Task-related neurons were significantly positively modulated (at least five spikes/s, and three standard errors above baseline activity for at least 150 ms) during the VGG task in any of three epochs of the task (Go cue, Lift of the hand, and Pull). Action Observation/Execution Neurons (AOENs) were defined as task-related (VGG) and significantly positively modulated (at least five spikes/s, and three standard errors above baseline activity for at least 200 ms) during passive viewing of any of the action videos. Likewise, neurons were considered significantly negatively modulated during a task when the minimal spike rate was no more than five spikes/s and the average activity was at least three standard errors below the baseline activity for at least 200ms. The average net spike rate was calculated for 15 to 35 repetitions per condition for each task. To assess the selectivity of our AOEN sample, we calculated the average responses to each action video as the average spike rate in a 200 ms interval around the maximum, and then calculated a two-way analysis of variance on these average responses with factors viewpoint (Viewpoint 1 and Viewpoint 2) and action type (Human touch, Human Grasp, and Monkey Grasp). This way, we tested for viewpoint selectivity and congruence of the action during execution and observation. Additionally, we calculated the d’ selectivity index for each neuron to quantify viewpoint selectivity: d^'=(mean(VP1)-mean(VP2))/√((var(VP1)+var(VP2))/2) with VP1 = viewpoint 1 and VP2 = viewpoint 2. We determined the preferred video as the one eliciting the highest firing rate for each SUA or MUA site. To capture the phasic nature of the responses during action observation, we used the matlab function ‘findpeaks’ detecting peaks in the normalized (by dividing by the maximum) net firing rate to each video with a minimal prominence (i.e., the decline in spike rate on either side of the peak) of 0.8, discarding sites that had more than three peaks due to noisy responses. We analyzed the peaks that were identified by the Matlab function regardless of the time epoch to account for the variable length of the videos. To characterize the degree of tuning during passive viewing of the action video, we calculated the full width at half maximum (FWHM) of the spike rate around the peak response to the preferred action video. For all AOENs, we then plotted the x and y position of the hand with respect to the object and the Euclidean distance (in pixels) to the object 50 ms before the peak response occurred (to account for the latency of the neuronal response). A Kruskal-Wallis one-way ANOVA was used to test whether the neurons showed a significant preference for one of the three movement intervals: approaching the object, interacting with the object, and receding from the object. Additionally, to test whether static frames of the action video could account for the observed responses, we plotted the peak firing rate during the preferred action video against the peak firing rate during the static frame videos (in which the hand was either touching the object or halfway its trajectory towards the object). We then calculated the Pearson correlation coefficient between the two peak firing rates (action video and static frame video) for each static frame video. This control experiment was performed in a subset of the recorded F5c sample. Because almost no AOEN responded during the entire video but rather during specific epochs, we compared the maximal spike rate during the preferred action video to the maximal spike rate during the corresponding ellipse video with the normal background. Note that our analysis ignored the exact timing of the maximal firing rate because the action videos and the ellipse videos differed in length. Ellipse neurons were defined as AOENs with a maximal spiking response to the ellipse video that was at least 50% of the maximal spike rate during viewing of the preferred action video, analogous to 12. Furthermore, to assess whether the F5c ellipse neurons were selective for the direction or the orientation of the movement, we used a Mann-Whitney U test to compare the average activity of each neuron during the approaching and the receding phase of the ellipse when it moved on the scrambled background. Since our action execution task only included object grasping and in line with 2, we defined strictly congruent AOENs as neurons that showed a significant preference for Human grasp over Human touch (based on a two-way ANOVA with factors perspective and action type, main effect of action type p < 0.05). Broadly congruent AOENs were defined as neurons in which the main effect of action type was not significant. All post-hoc tests were calculated with the Tukey’s honestly significant difference procedure. Suppression AOENs were defined as neurons that were significantly positively modulated during the action execution task and significantly negatively modulated during the action observation task, as in 8. To assess whether suppression AOENs also responded to the ellipse control video, we calculated the Pearson correlation coefficient between the average spike rate in a 200 ms interval around the most inhibitory activity during the preferred action video and the average spike rate in the corresponding interval in the ellipse video. In contrast to excitatory AOENs, suppression AOENs did not exhibit a phasic response to the videos. Therefore, we calculated the average spike rate in an interval instead of using the spike rate in one bin. For each AOEN we defined the preferred action video as the action video with the lowest net spike rate during the movement of the hand. To investigate whether muscle activity contributed to the neural responses observed during the action observation task, we measured the electromyographic (EMG) activity of the thumb and bicep muscle of the hand used in the VGG task during passive fixation of the action videos. The EMG signal was recorded in Monkey 3 using dry self-adhesive electrodes. The ground electrode was placed next to the recording electrode on the bicep muscle. Data were obtained with a multi-channel amplifier (EMG100C, BIOPAC systems Inc., CA, US) and sampled at 10000Hz with a gain of 5000. After applying a bandpass filter between 2 and 30Hz, the rectified EMG signal was aligned to the neural data. We then correlated the rectified EMG signal (in 50 ms bins) with the spiking activity of each AOEN in a 1000 ms interval (500 ms around the peak response and 500 ms one second before the peak response). 1. Instrument- or software-specific information needed to interpret the data: All data analyses were performed in MATLAB (MathWorks, MA). 2. Quality-assurance procedures performed on the data For behavioral monitoring, we have also continuously recorded the right eye position of each animal using an infrared-based camera system (Eye Link II, SR Research, Ontario, Canada), sampling the pupil position at 500 Hz. We replicated all results in 5 implantations in 4 monkeys to assure the robustness of the results. 1. People involved with sample collection, processing, analysis and/or submission: The same operator (Sofie De Schrijver) has performed all the experimental work and the data processing/analyses. ## DATA-SPECIFIC INFORMATION 1. Number of variables, label, description and units (for all the uploaded .mat files): Each .mat file contains 2 variables that are necessary for the analysis of the data: cerebusDataA and tnsTrials. The third variable 'tnsData' is only necessary when analysing the EMG data in the next file: mirrormovies_Loki_20230511_0940_A. Some .mat files have an additional variable 'cerebusDataB' that contains the same structure as cerebusDataA but for additional electrodes. When numbering electrodes, electrodes of cerebusDataB follow those of cerebusDataA. So if cerebusDataA contains 128 electrodes and cerebusDataB 64 electrodes, the electrodes of cerebusDataB will be electrode 129 until 192. A) cerebusDataA contains the neural data Action execution task (vgrasp) var1: elecSpikes; timestamps of each electrode. \_0 in the name indicates that it is multi-unit activity. Each timestep is the timing of a spike event as trespassing the set threshold (in milliseconds). var2: Cue; timing of the Cue (in milliseconds) used to align the neural data to the task data. other vars: N/A for this task. Action observation task (mirrormovies) var1: elecSpikes; timestamps of each electrode. \_0 in the name indicates that it is multi-unit activity. Each timestep is the timing of a spike event as trespassing the set threshold (in milliseconds). var2: PhotoEvents; timing of the photoevents (in milliseconds) used to align the neural data to the task data. Each photoevent is recorded when a bright white square is presented on the screen. var3: anInputData; the 4th column contains the raw EMG data other vars: N/A for this task. B) tnsTrials contains the task data Action execution task (vgrasp) var1: Index; trial number var2: Start; start of the trial var3: Stop; stop of the trial var4: Answer; answer is 1 if the animal performs the trial correctly. var5: TargetObject; sphere that needs to be grasped [0:3] var11: Light; external light turns on and illuminates the object. var12: Target; when the monkey pulls the correct sphere. var14: Cue; two values that indicate when the cue light goes on and off. So the second value is the go cue. other vars: N/A for this task Action observation task (mirrormovies) var1: Index; trial number var2: Start; start of the trial var3: Stop; stop of the trial var4: Answer; answer is 1 if the animal performs the trial correctly. var5: Stimulus; name of the video that is shown (POV = point-of-view of the monkey = Viewpoint 1; Side = filmed from the side = Viewpoint 2). Each stimulus name with its corresponding name in the paper: Human_Grip = Human Grasp; Human_Fist = Human touch; Obi_Monkey = Monkey; Human_Nogo = Static; Object_Ellipse = Ellipse; SCR_Ellipse = SCR background; Human_staticgrip = Static Interaction; Human_StaticApproach = Static Approach; Human_Disappear was not used in this study. var11: PhotoEvents; timing of photoevents (in milliseconds), 3 in a correct trial: (1) start of the trial, (2) start of the video, (3) end of the video. other vars: N/A for this task For files that were generated after 2020, there is an extra .mat file named *_ReplayedMUA.mat. This file contains 1 variable 'cerebusDataA'. Use this variable when the *_ReplayedMUA.mat file is present, otherwise the correct multi-unit activity is stored in the general .mat file. The variable 'elecSpikes' has the same structure in both .mat files. Each .mat file has a .nex5 file that contains the timesteps (in milliseconds) of the sorted spikes per electrode. The variable structure is the same for the mgrasp_dark.mat files and the vgrasp.mat files, with the exception that var11 in the tnsTrials variable contains 2 values; 1 for when the external light goes on and 1 for when the light goes off. All data shown in the paper are from the ventral premotor area F5c. However, not all electrodes that are included in the files are implanted in this area. This list lists the electrodes that are located in the ventral premotor area: The electrode numbering is the same in the action execution (vgrasp) and the action observation task (mirrormovies). 20190221 - elec 1:96 20190225 - elec 1:64 20190226 - elec 1:64 20200107 - elec 97:128 20200109 - elec 97:128 20200110 - elec 1:96 20200615 - elec 1:96 20200616 - elec 1:96 20200617 - elec 1:96 20210525 - elec 1:96 20210528 - elec 1:96 20210531 - elec 1:96 20221014 - elec 193:256 20221017 - elec 193:256 20221027 - elec 1:64 & elec 97:128 20230511 - elec 1:64 & elec 97:128 1. Missing data codes: None 2. Abbreviations used: N/A