Hathi Trust Library Vectorized features

A smaller-resolution (and therefore more portable) version of the Stable Random Projection Hathi Trust features described in my forthcoming article. The Northeastern repository is many individual files with 1280 random dimensions; this is just 640 random dimensions. The numbers are also experimentally encoded as half-precision floats, which cuts the file size by half at the cost of only being supported by my Python module. The net result is a file 1/4 the size of the full resolution ones for the paper that has, probably, something like 60-80% of the information content.

The full file is 'ht-640d-complete-half-precision.bin'. You can also download 11 smaller files organized by language.

Since these files use half-precision float encoding, to read them you must specify the precision when reading: e.g.,

from SRP import Vector_file
f = Vector_file("ita.bin", precision = "half")

Code to read these files is at https://github.com/bmschmidt/pySRP.