A View From Somewhere: Human-Centric Face Representations

<strong>Abstract</strong> Few datasets contain self-identified sensitive attributes, inferring attributes risks introducing additional biases, and collecting attributes can carry legal risks. Besides, categorical labels can fail to reflect the continuous nature of human phenotypic diversity, making it difficult to compare the similarity between same-labeled faces. To address these issues, we present A View From Somewhere (AVFS)---a dataset of 638,180 human judgments of face similarity. We demonstrate the utility of AVFS for learning a continuous, low-dimensional embedding space aligned with human perception. Our embedding space, induced under a novel conditional framework, not only enables the accurate prediction of face similarity, but also provides a human-interpretable decomposition of the dimensions used in the human decision-making process, and the importance distinct annotators place on each dimension. We additionally show the practicality of the dimensions for collecting continuous attributes, performing classification, and comparing dataset attribute disparities. <strong>ICLR 2023 Paper</strong> arxiv.org/abs/2303.17176 <strong>Code</strong> github.com/SonyResearch/a_view_from_somewhere <strong>Citing A View From Somewhere</strong> <pre>@inproceedings{ andrews2023avfs, title={A View From Somewhere: Human-Centric Face Representations}, author={Jerone T A Andrews and Przemyslaw Joniak and Alice Xiang}, booktitle={ICLR}, year={2023} }</pre>【摘要】 现有数据集鲜有包含自我标注的敏感属性，通过推断获取属性不仅可能引入额外偏差，还存在合规风险。此外，分类标签无法体现人类表型多样性的连续属性，导致难以对同标签人脸的相似性进行有效比对。为解决上述问题，我们提出了《A View From Somewhere（AVFS）》数据集——该数据集包含638180条人类对人脸相似性的主观标注。我们验证了AVFS在学习贴合人类感知的连续低维嵌入空间方面的应用价值。本研究基于全新条件框架生成的嵌入空间，不仅可实现人脸相似性的精准预测，还能对人类决策过程中用到的维度进行可解释性分解，并量化不同标注者对各维度的重视程度。此外，我们还证实了该嵌入维度在采集连续属性、开展分类任务以及比对数据集属性差异等场景中的实用性。 【ICLR 2023 收录论文】 arxiv.org/abs/2303.17176 【代码仓库】 github.com/SonyResearch/a_view_from_somewhere 【《A View From Somewhere》引用格式】 @inproceedings{ andrews2023avfs, title={A View From Somewhere: Human-Centric Face Representations}, author={Jerone T A Andrews and Przemyslaw Joniak and Alice Xiang}, booktitle={ICLR}, year={2023} }