Numberbatch 19.08 Word Embeddings (filtered, dim50, compressed, as single files)

Numberbatch word embeddings obtained from https://github.com/commonsense/conceptnet-numberbatch
split into single language files, filtered, and reduced to 50 components per vector using PCA.

By splitting the original 20.8 GB numberbatch file in into separately downloadable, compressed files -
and thus reducing the download and in-memory-size requirements of these embeddings by several magnitudes -
it becomes feasible to work with them in highly memory-constrained environments such as Binder.
This is useful for teaching purposes and a prerequisite for upcoming interactive publications
we are working on.

PREPROCESSING

Words `w` with `w.isalpha() == False` (in Python) have been excluded, since these are
seldomly useful. This filtering step reduces the file sizes by roughly 30%.

The dimension of the embeddings has been reduced to 50 by applying a standard PCA without whitening.

ZIP files have been compressed in Python using zipfile.ZIP_LZMA. To load the uncompressed files
into gensim, use `gensim.models.KeyedVectors.load_word2vec_format(path, binary=True)`.

LICENSE

This data contains semantic vectors from ConceptNet Numberbatch, by
Luminoso Technologies, Inc. You may redistribute or modify the
data under the terms of the CC-By-SA 4.0 license.