Caltech Football Numbers (CaltechFN)

Digit datasets are widely used as compact, generalizable benchmarks for novel computer vision models. However, modern deep learning architectures have surpassed the human performance benchmark on the current state-of-the-art digit datasets. These datasets largely contain images of digits that are smooth and fully visible, which limits the variability between the digits. On the other hand, the digits on American football jerseys are highly variable due to the propensity of jerseys to be wrinkled, stretched, twisted, and otherwise distorted in live action. Given that American football is a fast-paced sport, the digits on a jersey will likely be distorted in a different way from moment to moment, making it harder for artificial vision systems to differentiate between distinct digits. Furthermore, the digits on American football jerseys will often be partially occluded in a live-action capture due to the presence of other players and props. While the human brain is able to infer the identity of partially occluded digits by "filling in" visual gaps, artificial vision systems struggle to do this. To catalyze the improvement of computer vision models in these areas, we introduce CaltechFN, an image dataset of American football numbers that will serve as a new state-of-the-art benchmark for classification, detection, and localization tasks.