Data from: An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning

PyTorch implementation of "An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning" This repository contains the PyTorch implementation for the ECCV 2020 Paper "An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning". Few-shot learning aims to train efficient predictive models with a few examples. The lack of training data leads to poor models that perform high-variance or low-confidence predictions. In this paper, we propose to meta-learn the ensemble of epoch-wise empirical Bayes models (E3BM) to achieve robust predictions. "Epoch-wise" means that each training epoch has a Bayes model whose parameters are specifically learned and deployed. "Empirical" means that the hyperparameters, e.g., used for learning and ensembling the epoch-wise models, are generated by hyperprior learners conditional on task-specific data. We introduce four kinds of hyperprior learners by considering inductive vs. transductive, and epoch-dependent vs. epoch-independent, in the paradigm of meta-learning. We conduct extensive experiments for five-class few-shot tasks on three challenging benchmarks: miniImageNet, tieredImageNet, and FC100, and achieve top performance using the epoch-dependent transductive hyperprior learner, which captures the richest information. Our ablation study shows that both "epoch-wise ensemble" and "empirical" encourage high efficiency and robustness in the model performance.

本仓库实现了ECCV 2020收录论文《面向少样本学习（Few-shot Learning）的逐轮经验贝叶斯集成方法》的PyTorch代码。少样本学习旨在利用极少量训练样本训练高效的预测模型。训练数据的匮乏会导致模型性能欠佳，预测结果存在较高方差或置信度不足的问题。本文提出通过元学习构建逐轮经验贝叶斯模型集成（E3BM，Ensemble of Epoch-wise Empirical Bayes Models），以实现鲁棒的预测效果。“逐轮（Epoch-wise）”指每个训练轮次（epoch）对应一个专属贝叶斯模型，其参数均经过专门学习并部署使用；“经验式（Empirical）”则指用于学习与集成逐轮模型的超参数，均由基于任务专属数据的超先验学习器（hyperprior learner）生成。在元学习范式下，本文通过归纳式（inductive）与转导式（transductive）、轮次依赖与轮次无关这两组维度，共提出四类超先验学习器。我们在miniImageNet、tieredImageNet与FC100三个极具挑战性的基准数据集上开展了五分类少样本任务的大量实验，并通过捕获信息最为丰富的轮次依赖转导式超先验学习器取得了最优性能。我们的消融实验结果表明，“逐轮集成”与“经验式”设计均能有效提升模型的运行效率与预测鲁棒性。