The role of consciously timed movements in shaping and improving auditory timing

Our subjective sense of time is intertwined with a plethora of perceptual, cognitive, and motor functions, and likewise, the brain is equipped to expertly filter, weight, and combine these signals for seamless interactions with a dynamic world. Until relatively recently, the literature on time perception has excluded the influence of motor activity, yet, it has been found that motor circuits in the brain are at the core of most timing functions. Several studies have now identified that concurrent movements exert robust effects on perceptual timing estimates, but critically, have not assessed how humans consciously judge the duration of their own movements. This creates a gap in our understanding of the mechanisms driving movement-related effects on sensory timing. We sought to address this gap by administering a sensorimotor timing task in which we explicitly compared the timing of isolated auditory tones and arm movements, or both simultaneously. We contextualized our findings within a Bayesian cue combination framework, in which separate sources of temporal information are weighted by their reliability and integrated into a unitary time estimate that is more precise than either unisensory estimate. Our results revealed differences in accuracy between auditory, movement, and combined trials, and crucially, that combined trials were the most accurately timed. Under the Bayesian framework, we found that participants’ combined estimates were more precise than isolated estimates in a way that trended towards optimality, while being overall less optimal than the model’s prediction. These findings elucidate previously unknown qualities of conscious motor timing and propose computational mechanisms that can describe how movements combine with perceptual signals to create unified, multimodal experiences of time.

Methods
Participants performed the experiment using a robotic arm manipulandum (KINARM End-Point Lab, BKIN Technologies; Nguyen et al. 2019; Hosseini et al. 2017) that allowed movement along a flat workspace using the right arm. Direct viewing of the robotic arm was occluded by a flat display that allowed viewing of targets and cues via a downward-facing monitor mounted above the workspace. Motor output was sampled at 1000 Hz. Participants were free to adjust the chair so they could comfortably view the full display. 

Trials were divided into encoding and reproduction phases, and were structured as follows (see Fig. 1): first, the robotic arm guided participants to one of 16 locations in a grid-like array. Then, they experienced one of three trial conditions. In “movement” trials, subjects began moving until interrupted by an imposed brake (a 100 ms linear increase in resistive force from 0 to 50 N). In “auditory” trials, the robotic arm was locked in the random location and the participant heard an auditory tone. In “combined” trials, subjects were cued to move while timing a concurrent auditory tone; in this condition, the tone began as soon as the apparatus detected movement at the velocity threshold of 5 cm/s, and the brake was applied synchronously with the auditory tone offset. After the encoding phase, they were guided to a central target for the reproduction phase. When this target turned green, subjects reproduced the encoded duration by holding and releasing a button attached to the handle. The tested durations were 1000, 1500, 2000, 2500, 3000, 3500, and 4000 ms. Trial conditions were experienced in blocks of 14 trials (for a total of 210 trials) in a pseudorandomized order such that no condition was experienced twice in a row.

Robotic arm manipulandum data were sampled at 1000 Hz to produce vectors for position, velocity, force, and other movement parameters over the course of time for each trial. Trials were excluded if reproduced times fell outside three deviations from the mean (<1% of trials excluded). Additionally, trials with movement (movement-only and combined) were excluded if the stop latency after the brake was applied fell outside three scaled absolute deviations from the median (2.9% of trials excluded). 

Our first goal was to investigate the relationship between duration and reproduced time to determine timing accuracy. Importantly, movement-only and combined trials were analyzed with respect to time spent moving rather than the pre-specified duration. This is because in general, participants exhibited a short delay to respond to the movement brake, usually adding up to a few hundred milliseconds on most trials. We directly assessed constant error as a measure of accuracy, defined as the difference between the reproduced duration using the button press and the actual target duration.  To measure performance, we applied a linear mixed model (LMM) design, in which response error was the predicted variable, encoded duration and condition were fixed effects, and subject was treated as a random effect.  

Additionally, we examined the relationships between movement parameters and timing performance. Previous work has shown that arm movements covering a greater distance lead to longer perceived durations (De Kock et al., 2021; Wiener et al., 2019). Here, we extracted the movement distance for each trial (defined as the Euclidean distance traveled between duration onset and offset) and performed a Spearman partial correlation test between movement distance and reproduced time, controlling for target duration.  Partial correlation values were calculated for each subject separately for movement and combined trial types; each was assessed with a one-sample t-test against a baseline of zero.  
Statistical analyses were performed using R, JASP (http://www.jasp-stats.org), and Matlab. For accuracy and CV analyses, we report results from linear mixed models with subject as a random effect. Results are reported at a significance level of 0.05. This dataset includes trials for all participants.