Discounting Chain Environment

该数据集是一个旨在凸显不同折现期限影响的玩具问题，它对折现因子非常敏感。此外，该环境允许访问真实的价值函数，这有助于检查由于错误折现因子引起的偏差。任务是通过元梯度强化学习实验，演示在内层损失中使用外层批评家的影响。

This dataset is a toy problem designed to highlight the impact of different discount horizons, and it is highly sensitive to the discount factor. Furthermore, this environment enables access to the true value function, which facilitates the detection of biases caused by incorrect discount factors. The task is to demonstrate the effect of using the outer-loop critic in the inner-loop loss through meta-gradient reinforcement learning experiments.