DataSheet1_Active Inference and Epistemic Value in Graphical Models.pdf

The Free Energy Principle (FEP) postulates that biological agents perceive and interact with their environment in order to minimize a Variational Free Energy (VFE) with respect to a generative model of their environment. The inference of a policy (future control sequence) according to the FEP is known as Active Inference (AIF). The AIF literature describes multiple VFE objectives for policy planning that lead to epistemic (information-seeking) behavior. However, most objectives have limited modeling flexibility. This paper approaches epistemic behavior from a constrained Bethe Free Energy (CBFE) perspective. Crucially, variational optimization of the CBFE can be expressed in terms of message passing on free-form generative models. The key intuition behind the CBFE is that we impose a point-mass constraint on predicted outcomes, which explicitly encodes the assumption that the agent will make observations in the future. We interpret the CBFE objective in terms of its constituent behavioral drives. We then illustrate resulting behavior of the CBFE by planning and interacting with a simulated T-maze environment. Simulations for the T-maze task illustrate how the CBFE agent exhibits an epistemic drive, and actively plans ahead to account for the impact of predicted outcomes. Compared to an EFE agent, the CBFE agent incurs expected reward in significantly more environmental scenarios. We conclude that CBFE optimization by message passing suggests a general mechanism for epistemic-aware AIF in free-form generative models.

自由能原理（Free Energy Principle, FEP）提出，生物智能体通过感知并与环境交互，以最小化基于环境生成模型的变分自由能（Variational Free Energy, VFE）。依据FEP推导策略（未来控制序列）的过程，被称为主动推理（Active Inference, AIF）。AIF相关文献已提出多种可催生认知（信息寻求）行为的VFE策略规划目标，但多数目标的建模灵活性较为有限。本文从约束贝塞自由能（Constrained Bethe Free Energy, CBFE）的视角出发，探讨认知行为相关问题。至关重要的是，CBFE的变分优化可通过自由格式生成模型上的消息传递得以实现。CBFE的核心直觉在于，我们对预测结果施加了点质量约束，这一设定明确编码了“智能体未来将进行观测”的假设。我们从其构成的行为驱动维度对CBFE目标进行解读。随后，通过在模拟T迷宫环境中开展规划与交互实验，展示了CBFE所产生的智能体行为。针对T迷宫任务的仿真结果表明，CBFE智能体展现出认知驱动特性，并能够主动提前规划，以考量预测结果带来的影响。相较于期望自由能（Expected Free Energy, EFE）智能体，CBFE智能体在更多环境场景中可获得预期奖励。最终我们得出结论：通过消息传递实现CBFE优化，为自由格式生成模型下具备认知意识的主动推理提供了一种通用机制。