CoPhy: Counterfactual Learning of Physical Dynamics (Benchmark Dataset)

Benchmark website: https://projet.liris.cnrs.fr/cophy/ Understanding causes and effects in mechanical systems is an essential component of reasoning in the physical world. This work poses a new problem of counterfactual learning of object mechanics from visual input. We develop the COPHY benchmark to assess the capacity of the state-of-the-art models for causal physical reasoning in a synthetic 3D environment. Having observed a mechanical experiment that involves, for example, a falling tower of blocks, a set of bouncing balls or colliding objects, we require to learn to predict how its outcome is affected by an arbitrary intervention on its initial conditions, such as displacing one of the objects in the scene. The main objective for the creation of our benchmark is (a) to focus specifically on evaluating capabilities of state of the art models for performing counterfactual reasoning, (b) to be unbiased in terms of distributions of parameters to be estimated and balanced with respect to possible outcomes, and (c) to have sufficient variety in terms of scenarios<br> and latent physical characteristics of the scene that are not visually observed and therefore can act<br> as confounders. If you use this benchmark, you need to cite the following paper: Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf. COPHY: Counterfactual Learning of Physical Dynamics. pre-print arXiv:1909.12000, 2019.