Tibetan for Spacy 1.1

Tibetan for SpaCy is a language model for Tibetan designed for use in the SpaCy environment. The model was trained using SpaCy. It uses an external tokenizer, Botok, to segment the Tibetan and replaces the Tibetan syllable-separator (tseg) with white spaces where it occurs as a word separator. SpaCy was then told to interpret the input as English. This produces good results with standard vocabulary, but fails with unrecognised words. The project is currently working on a more sophisticated version of Tibetan for SpaCy. The package includes a list of stop words.

Tibetan for SpaCy was developed by James Engels as part of Divergent Discourses, a joint project between SOAS University of London and Leipzig University, funded by the AHRC in the UK and the DFG in Germany.

The project developed Tibetan for SpaCy particularly for users who want to use Tibetan texts within the Leipzig Corpus Miner (iLCM), an advanced text-mining interface designed for social scientists. The instructions for using Tibetan for SpaCy within the iLCM are in the readme file below. These instructions assume the user has already downloaded and installed the iLCM, which can be found here. 

Please acknowledge the Divergent Discourses project if using this material (note that ultimately copyright belongs to the two participating universities). Contact: [e-mail]