Code and Dataset for Pattern Recognition Benchmarks

<p>Previous work has shown that image datasets have so much structure that a mere random projection of the dataset onto a 1D subspace is likely to uncover some of that structure. We subsequently built on this observation to propose a sequence of bounds for the error rate of a pattern recognition method based on image data. The code included computes these bounds from a training dataset. </p>

<p>The bounds correspond to the error rate that one would expect to achieve by simply selecting features at random and thresholding each randomly selected feature in order to separate the patterns (the ``TARP" approach). Our proposed bounds form a monotonically decreasing sequence that converges to a limit.  While this limit is the infimum of the sequence,  even the first few terms can be comparable or even below the error rate of more complex methods like support vector machines. The computational cost of a pattern recognition method should be offset by an accuracy gain over the TARP approach, otherwise the method does not provide any more insight into the pattern structure than random projections. This offset can be visualized in a two-dimensional space parametrized by error rate and computational cost;  only pattern recognition methods that lie on the left-hand-side of the curve defined by our proposed sequence of bounds can be considered insightful. </p>