Data from: Learning the sound inventory of a complex vocal skill via an intrinsic reward

Reinforcement learning (RL) is thought to underlie the acquisition of
vocal skills like birdsong and speech, where sounding like one’s “tutor”
is rewarding. But what RL strategy generates the rich sound inventories
for song or speech? We find that the standard actor-critic model of
birdsong learning fails to explain juvenile zebra finches’ efficient
learning of multiple syllables. But when we replace a single actor with
multiple independent actors that jointly maximize a common intrinsic
reward, then birds’ empirical learning trajectories are accurately
reproduced. Importantly, the influence of each actor (syllable) on the
magnitude of global reward is competitively determined by its acoustic
similarity to target syllables. This leads to each actor matching the
target it is closest to, and occasionally, to the competitive exclusion of
an actor from the learning process (i.e., the learned song). We propose
that a competitive-cooperative multi-actor (MARL) algorithm is key for the
efficient learning of the action inventory of a complex skill.