Numberbatch 19.08 Word Embeddings (filtered, dim50, compressed, as single files)

Numberbatch word embeddings obtained from https://github.com/commonsense/conceptnet-numberbatch<br> split into single language files, filtered, and reduced to 50 components per vector using PCA. By splitting the original 20.8 GB numberbatch file in into separately downloadable, compressed files -<br> and thus reducing the download and in-memory-size requirements of these embeddings by several magnitudes -<br> it becomes feasible to work with them in highly memory-constrained environments such as Binder.<br> This is useful for teaching purposes and a prerequisite for upcoming interactive publications<br> we are working on. PREPROCESSING Words `w` with `w.isalpha() == False` (in Python) have been excluded, since these are<br> seldomly useful. This filtering step reduces the file sizes by roughly 30%. The dimension of the embeddings has been reduced to 50 by applying a standard PCA without whitening. ZIP files have been compressed in Python using zipfile.ZIP_LZMA. To load the uncompressed files<br> into gensim, use `gensim.models.KeyedVectors.load_word2vec_format(path, binary=True)`. LICENSE This data contains semantic vectors from ConceptNet Numberbatch, by<br> Luminoso Technologies, Inc. You may redistribute or modify the<br> data under the terms of the CC-By-SA 4.0 license.<br>