Prospective contingency explains behavior and dopamine signals during associative learning

Associative learning depends on contingency, the degree to which a stimulus predicts an outcome. Despite its importance, the neural mechanisms linking contingency to behavior remain elusive. In the present study, we examined the dopamine activity in the ventral striatum—a signal implicated in associative learning—in a Pavlovian contingency degradation task in mice. We show that both anticipatory licking and dopamine responses to a conditioned stimulus decreased when additional rewards were delivered uncued, but remained unchanged if additional rewards were cued. These results conflict with contingency-based accounts using a traditional definition of contingency or a new causal learning model (ANCCR), but can be explained by temporal difference (TD) learning models equipped with an appropriate intertrial interval state representation. Recurrent neural networks trained within a TD framework develop state representations akin to our best ‘handcrafted’ model. Our findings suggest that the TD error can be a measure that describes both contingency and dopaminergic activity.

联想学习依赖于联结依存性，即刺激对结果的预测程度。尽管该过程至关重要，但联结依存性与行为之间的神经机制仍未明晰。本研究在小鼠的巴甫洛夫联结依存性降解任务中，检测了腹侧纹状体（ventral striatum）内的多巴胺活动——该信号已被证实与联想学习密切相关。研究发现，当额外奖励以无提示方式呈现时，动物的预期舔舐行为与条件刺激（conditioned stimulus）诱发的多巴胺反应均会下降；而当额外奖励带有提示时，二者则保持不变。该结果与采用传统依存性定义的联结依存性理论，以及新型因果学习模型（ANCCR）的预测相悖，但可通过配备了合适试间间隔状态表征的时间差分（temporal difference, TD）学习模型加以解释。在TD框架下训练的循环神经网络（recurrent neural networks）所习得的状态表征，与我们最优的"手工构建"模型高度相似。本研究结果表明，TD误差可作为同时表征联结依存性与多巴胺能活动的量化指标。