遇见数据集

MIRROR-EN-UA-RU: Synthetic Trilingual Minimal-Pair Sentences for Nationality and Langugae Bias Evaluation in Large Language Models

收藏
Zenodo2026-08-15 更新2026-06-21 收录
官方服务:

资源简介:

This record contains a fully synthetic trilingual (English / Russian / Ukrainian) dataset of mirrored minimal-pair sentences for measuring nationality bias in large language models between actors representing Ukraine and actors representing Russia.Contents (24 files): 22 category files (01-make_statement.csv … 20-mass_violence.csv, plus 112-accuse.csv and 11-approve.csv), each containing 1,200 rowsentity_inventory.csv — complete list of entity types per categoryprompts.txt — consolidated documentation of the generation prompts and template structure Structure. Categories follow the 20 top-level event types of the CAMEO codebook, plus subcategory 112 (Accuse) and an added Approve category. Each category file holds 100 scenario templates expanded over a fully crossed design: actor nationality (side: ua/ru) × action polarity (sentiment: pos/neg) × temporal frame (timeframe: Past/Present/Future). Columns: pair_id, category, entity_type, timeframe, side, sentiment, sentence_en, ru, uk. Within a pair_id, the 12 variants are lexically identical except for the manipulated elements. Total: 26,400 rows / 79,200 sentences. All files are UTF-8 CSV with header rows.Provenance. English sentences generated on 22 February 2026 with OpenAI GPT-5.4 (ChatGPT interface); deduplicated automatically and manually reviewed on random samples. Russian and Ukrainian translations produced with Microsoft Translator (Microsoft 365 Excel TRANSLATE) and checked by a native speaker. All sentences are fictional; actors are generic roles (president, doctor, journalist…) — no real individuals are named.Intended use. Benchmarking nationality bias in LLMs, multilingual classifiers, embedding models, and machine-translation systems; the design allows isolating the effect of actor nationality from polarity, tense, and language.

提供机构:
Zenodo
创建时间:
2026-06-18
二维码
社区交流群
二维码
科研交流群
商业服务