LOOIC for each model.

Research suggests that a fast, capacity-limited working memory (WM) system and a slow, incremental reinforcement learning (RL) system jointly contribute to instrumental learning. Thus, situations that strain WM resources alter instrumental learning: under WM loads, learning becomes slow and incremental, the reliance on computationally efficient learning increases, and action selection becomes more random. It is also suggested that Pavlovian learning influences people’s behavior during instrumental learning by providing hard-wired instinctive responses including approach to reward predictors and avoidance of punishment predictors. However, it remains unknown how constraints on WM resources affect instrumental learning under Pavlovian influence. Thus, we conducted a functional magnetic resonance imaging (fMRI) study (N = 49) in which participants completed an instrumental learning task with Pavlovian–instrumental conflict (the orthogonalized go/no-go task) both with and without extra WM load. Behavioral and computational modeling analyses revealed that WM load reduced the learning rate and increased random choice, without affecting Pavlovian bias. Model-based fMRI analysis revealed that WM load strengthened RPE signaling in the striatum. Moreover, under WM load, the striatum showed weakened connectivity with the ventromedial and dorsolateral prefrontal cortex when computing reward expectations. These results suggest that the limitation of cognitive resources by WM load promotes slow and incremental learning through the weakened cooperation between WM and RL; such limitation also makes action selection more random, but it does not directly affect the balance between instrumental and Pavlovian systems.

现有研究表明，快速且容量受限的工作记忆（working memory, WM）系统与缓慢渐进的强化学习（reinforcement learning, RL）系统共同参与工具性学习（instrumental learning）过程。因此，占用工作记忆资源的情境会对工具性学习产生影响：当受试者承受工作记忆负荷时，学习进程会变得缓慢且渐进，对计算高效型学习的依赖程度提升，且动作选择更具随机性。另有研究指出，巴甫洛夫学习（Pavlovian learning）会在工具性学习过程中，通过引发先天固有的本能反应（如趋近奖赏预测线索、回避惩罚预测线索）来影响个体行为。然而，目前尚不清楚在巴甫洛夫学习的影响下，工作记忆资源受限会如何作用于工具性学习。为此，我们开展了一项功能磁共振成像（functional magnetic resonance imaging, fMRI）研究（样本量N=49）：受试者在施加额外工作记忆负荷与未施加负荷的两种条件下，分别完成了带有巴甫洛夫-工具性冲突的工具性学习任务——即正交化go/no-go任务。行为学与计算建模分析结果显示，工作记忆负荷会降低学习率并提升随机选择倾向，但不会影响巴甫洛夫偏差（Pavlovian bias）。基于模型的功能磁共振成像分析进一步发现，工作记忆负荷会增强纹状体中的奖励预测误差（Reward Prediction Error, RPE）信号。此外，在施加工作记忆负荷的条件下，纹状体在计算奖赏预期时，与腹内侧前额叶皮层（ventromedial prefrontal cortex）、背外侧前额叶皮层（dorsolateral prefrontal cortex）的功能连接显著减弱。上述结果表明，工作记忆负荷造成的认知资源受限，会通过削弱工作记忆与强化学习系统的协同作用，推动缓慢且渐进式的学习进程；此类资源受限同样会使动作选择更具随机性，但不会直接改变工具性学习与巴甫洛夫学习系统之间的平衡。