Distill-and-Select (DnS)

在这项工作中，我们提出了一个称为提取和选择 (DnS) 的知识蒸馏框架，该框架从性能良好的细粒度教师网络开始学习: a) 在不同的检索性能和计算效率权衡下的学生网络; b) 在测试时快速将样本引导到适当的学生以保持高检索性能和高计算效率的选择器网络。我们用不同的体系结构训练几个学生，并得出性能和效率的不同权衡，即速度和存储要求，包括使用二进制表示存储/索引视频的细粒度学生In this work, we propose a knowledge distillation framework named Extract and Select (DnS), which learns starting from a well-performing fine-grained teacher network: a) a student network that enables diverse trade-offs between retrieval performance and computational efficiency; b) a selector network that rapidly guides samples to appropriate students during inference to maintain high retrieval performance and computational efficiency. We train several students with different architectures, yielding diverse trade-offs between performance and efficiency, namely speed and storage requirements, including fine-grained students that store or index videos using binary representations.

搜集汇总

背景与挑战

背景概述

Distill-and-Select (DnS) 是一个基于知识蒸馏框架的数据集，旨在训练学生网络以平衡检索性能与计算效率，并支持使用二进制表示进行存储。该数据集由伦敦玛丽女王大学和希腊研究和技术中心于2021年发布。