Average results across varying batch size.

In language emergence, neural agents acquire communication skills by interacting with one another and the environment. Through these interactions, agents learn to connect or ground their observations to the messages they utter, forming a shared consensus about the meaning of the messages. Such connections form what we refer to as a grounding map. However, these maps can often be complicated, unstructured, and contain redundant connections. In this paper, we introduce two novel functional pressures, modeled as differentiable auxiliary losses, to simplify and structure the grounding maps. The first pressure enforces compositionality via topological similarity, which has been previously discussed but has not been modeled or utilized as a differentiable auxiliary loss. The second functional pressure, which is conceptually novel, imposes sparsity in the grounding map by pruning weaker connections while strengthening the stronger ones. We conduct experiments in multiple value-attribute environments with varying communication channels. Our methods achieve improved out-of-domain regularization and rapid convergence over baseline approaches. Furthermore, introduced functional pressures are robust to the changes in experimental conditions and able to operate with minimum training data. We note that functional pressures cause simpler and more structured emergent languages showing distinct characteristics depending on the functional pressure employed. Enhancing grounding map sparsity yields the best performance and the languages with the most compressible grammar. In summary, our novel functional pressures, focusing on compositionality and sparse groundings, expedite the development of simpler, more structured languages while enhancing their generalization capabilities. Exploring alternative types of functional pressures and combining them in agent training may be beneficial in the ongoing quest for improved emergent languages.

在语言涌现（language emergence）领域，神经智能体（neural agents）通过彼此交互以及与环境交互来习得沟通技能。通过此类交互，智能体学会将自身的观测结果与自身产出的话语建立关联，或为之锚定（ground）意义，最终形成关于话语含义的共享共识。此类关联便构成了我们所称的锚定映射图（grounding map）。然而此类映射图往往结构复杂、缺乏规整性，且包含冗余关联。本文提出两种全新的功能约束（functional pressures），将其建模为可微辅助损失函数（differentiable auxiliary losses），用于简化并规整锚定映射图。第一种约束通过拓扑相似性强化组合性（compositionality），该思路此前已有讨论，但尚未被建模为可微辅助损失函数并加以应用。第二种功能约束在概念上为全新设计，通过剪去弱关联、强化强关联的方式，为锚定映射图引入稀疏性约束。我们在多种具备不同沟通通道的价值-属性环境（value-attribute environments）中开展了实验。相较于基线方法，本文所提方法实现了更优异的域外正则化（out-of-domain regularization）效果与更快的收敛速度。此外，所提出的功能约束对实验条件的变化具备鲁棒性，且仅需少量训练数据即可正常运行。我们观察到，功能约束可催生更简洁、更具规整性的涌现语言（emergent languages），且不同的功能约束会使语言呈现出截然不同的特性。强化锚定映射图的稀疏性约束可获得最优的实验性能，且所催生的语言拥有最具可压缩性的语法结构。综上，本文所提出的聚焦于组合性与稀疏锚定的全新功能约束，不仅加速了更简洁、更具规整性的语言的涌现进程，同时也提升了语言的泛化能力。在后续探索更优异的涌现语言的研究中，探索其他类型的功能约束并将其应用于智能体训练，或许能带来进一步的收益。