clapAI/MultiLingualSentiment

---
dataset_info:
features:
- name: text
dtype: string
- name: label
dtype: string
- name: source
dtype: string
- name: domain
dtype: string
- name: language
dtype: string
splits:
- name: train
num_bytes: 1364685913
num_examples: 3147478
- name: validation
num_bytes: 170841288
num_examples: 393435
- name: test
num_bytes: 170338153
num_examples: 393436
download_size: 988308759
dataset_size: 1705865354
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: validation
path: data/validation-*
- split: test
path: data/test-*
license: apache-2.0
task_categories:
- text-classification
language:
- ar
- de
- en
- es
- fr
- hi
- id
- it
- ko
- ms
- pt
- ru
- tr
- vi
- zh
- ja
tags:
- sentiment
- multilingual
- emotion
- review
- classification
pretty_name: text
size_categories:
- 1M<n<10M
---
## Overview
**MultilingualSentiment** is a sentiment classification dataset that encompasses three sentiment labels: **Positive**, **Neutral**, **Negative**

The dataset spans multiple languages and covers a wide range of domains, making it ideal for multilingual sentiment analysis tasks.

## Dataset Information
The dataset was meticulously collected and aggregated from various sources, including Hugging Face and Kaggle. These sources provide diverse languages and domains to ensure a comprehensive and balanced dataset.

- **Total records**: 3,934,349
- The dataset is divided into three subsets: train, validation, and test, with a ratio of 8:1:1:
+ Train: 3,147,478
+ Validation: 393,435
+ Test: 393,436

### Number of Records per Language
| Language | Count |
|---------------|---------|
| Arabic (ar) | 208,375 |
| German (de) | 212,853 |
| English (en) | 1,519,860 |
| Spanish (es) | 222,911 |
| French (fr) | 262,645 |
| Hindi (hi) | 9,423 |
| Indonesian (id) | 12,536 |
| Italian (it) | 3,020 |
| Japanese (ja) | 335,656 |
| Korean (ko) | 259,998 |
| Malay (ms) | 6,661 |
| Multilingual | 9,391 |
| Portuguese (pt) | 49,188 |
| Russian (ru) | 205,186 |
| Turkish (tr) | 44,743 |
| Vietnamese (vi) | 127,068 |
| Chinese (zh) | 444,835 |

### Number of Records per Label
| Label | Count |
|-----------|----------|
| Negative | 1,436,539 |
| Neutral | 1,041,512 |
| Positive | 1,456,298 |

## Applications
This dataset is well-suited for training and evaluating models in multilingual sentiment analysis, natural language processing (NLP), and domain-specific sentiment classification tasks.

## Loading dataset
```python
from datasets import load_dataset

# Load the MultilingualSentiment dataset
dataset = load_dataset("clapAI/MultiLingualSentiment")

print(dataset)

```

```
DatasetDict({
train: Dataset({
features: ['text', 'label', 'source', 'domain', 'language'],
num_rows: 3147478
})
validation: Dataset({
features: ['text', 'label', 'source', 'domain', 'language'],
num_rows: 393435
})
test: Dataset({
features: ['text', 'label', 'source', 'domain', 'language'],
num_rows: 393436
})
})
```

## Citation

```bibtex
@dataset{clapAI2024multilingualsentiment,
title = {MultilingualSentiment: A Multilingual Sentiment Classification Dataset},
author = {clapAI},
year = {2024},
url = {https://huggingface.co/datasets/clapAI/MultiLingualSentiment},
description = {A multilingual dataset for sentiment analysis with labels: positive, neutral, negative, covering diverse languages and domains.},
}
```

This is a multilingual sentiment classification dataset with three sentiment labels: positive, neutral, and negative, covering various languages and domains. The dataset has been meticulously collected and aggregated from multiple sources such as Hugging Face and Kaggle, consisting of 3,934,349 records divided into training, validation, and test sets with a ratio of 8:1:1. Supported languages include Arabic, German, English, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Portuguese, Russian, Turkish, Vietnamese, and Chinese.

搜集汇总

背景与挑战

背景概述

MultiLingualSentiment是一个大规模多语言情感分类数据集，包含约393万条记录，涵盖积极、中性和消极三种情感标签。该数据集覆盖17种语言和多个领域（如评论、情感分析等），适用于训练和评估多语言情感分析模型。