CatCoLA - Catalan Corpus of Linguistic Acceptability

We introduce CatCoLA, the Catalan Corpus of Linguistic Acceptability that will contribute to the Catalan Language Understanding Benchmark (CLUB) to assess and compare the capabilities of language models (LM) trained with texts in Catalan. CatCoLA follows the design of the English CoLA to support the task of classifying sentences as acceptable or not. CatCoLA consists of 10,443 sentences and their acceptability judgements as found in well-known Catalan reference grammars. Additionally, all sentences have been annotated with the class of linguistic phenomenon the sentence is an example of, also following previous practices.
CatCoLA is released under a CC BY-NC-SA 4.0 licence and freely available, but the test data to avoid contamination. Please, write a message to [e-mail] if interested in getting it.