SuccubusBot/incoherent-text-dataset

---
license: cc-by-sa-4.0
task_categories:
- text-classification
language:
- en
- es
- fr
- de
- zh
- ja
- ru
- ar
- hi
size_categories:
- 10K<n<100K
---

# Incoherent Text Dataset

This dataset is designed for training models to detect incoherence in text. It includes various types of incoherence, such as grammatical errors, word soup, random words, and run-on sentences.

## Dataset Details

- **Languages:** English, Spanish, French, German, Chinese, Japanese, Russian, Arabic, Hindi
- **Size:** ~27,000 samples
- **Types of Incoherence:** Grammatical errors, word soup, random words, run-ons, random tokens, random bytes.

## Data Fields

- **text:** The text sample.
- **length:** The number of words in the text sample.
- **label:** Whether the text is "OK" (coherent) or "incoherent".
- **type:** The type of text (coherent or one of the incoherence types).

## Usage

This dataset can be used to train machine learning models for text classification tasks, specifically for detecting and categorizing different types of text incoherence.

## Limitations

The dataset has been generated by an algorithm, so care must be taken in evaluating it in the real world. More data may need to be collected before evaluating a model trained on this data in a real-world setting.