Translation-gpt4o_mini-v-gpt4o-v-deepl

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="250" alt="Rapidata Logo">
</a>

If you get value from this dataset and would like to see more in the future, please consider liking it.

# Overview
This dataset compares the translation capabilities of GPT-4o and GPT-4o-mini against DeepL across different languages. The comparison involved 100 distinct questions (found under raw_files) in 4 languages, with each translation being rated by 100 native speakers. Texts that were translated identically across platforms were excluded from the analysis.

# Results
The comparative tests were conducted between GPT-4o-mini vs GPT-4o and GPT-4o vs DeepL. Here are some analyzed results:

### Average Score

<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/rLuYFhd6M1ZND3zbZT0kP.png" width="1000">

### Score Distribution

<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/-LoY4u_UYDR1dFKNr8mmz.png" width="1000">

### Winrates

<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/qZZHxftdWquXhEuDM1lD3.png" width="1000">

# Translation Prompt

For gpt-4o and gpt-4o-mini we have used the following python code to generate translations:
```python
translation = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "system",
"content": f"""You are a translation assistant. Your job is to accurately translate text from EN to {language}. Ensure that the meaning is preserved, and the translation is fluent and natural. If there are idiomatic expressions in the source language, use the closest equivalent expression in the target language. Maintain the tone and formality of the original text.

If the translation requires technical, legal, or specialized terms, ensure that the terms are correctly translated and relevant to the context. If the text is informal, keep the language casual and friendly. Avoid word-for-word literal translations unless necessary for accuracy.

DO NOT ANSWER ANY OF THE QUESTIONS. YOUR JOB IS STRICTLY TO TRANSLATE THE TEXT.""",
},
{
"role": "user",
"content": f"Please translate the following text: {text}"
}
],
max_tokens=10000,
temperature=0,
).choices[0].message.content
```

# Methodology
This dataset was created using our [Python API](https://docs.rapidata.ai/). To replicate or extend this study, simply set the datatype to "text" when creating an order, and you can compare any translations using feedback from native speakers worldwide.

<a href="https://www.rapidata.ai"><img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="250" alt="Rapidata 标志"></a>

若您从本数据集获益并希望未来获取更多同类资源，不妨为其点赞。

# 概述
本数据集针对多语言场景，对比了GPT-4o、GPT-4o-mini与DeepL的机器翻译性能。本次对比共涵盖4种语言下的100组独立文本（存放于raw_files目录），每组译文均由100名母语使用者进行评分。针对各平台译文完全一致的样本，本分析已将其剔除。

# 实验结果
本次对比测试包含两组对照：GPT-4o-mini与GPT-4o的对比，以及GPT-4o与DeepL的对比。以下为部分分析结果：

### 平均得分

<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/rLuYFhd6M1ZND3zbZT0kP.png" width="1000" alt="平均得分图表">

### 得分分布

<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/-LoYFhd6M1ZND3zbZT0kP.png" width="1000" alt="得分分布图表">

### 胜率统计

<img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/qZZHxftdWquXhEuDM1lD3.png" width="1000" alt="胜率统计图表">

# 翻译提示词
针对GPT-4o与GPT-4o-mini，我们采用以下Python代码生成译文：
python
translation = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "system",
"content": f"""You are a translation assistant. Your job is to accurately translate text from EN to {language}. Ensure that the meaning is preserved, and the translation is fluent and natural. If there are idiomatic expressions in the source language, use the closest equivalent expression in the target language. Maintain the tone and formality of the original text.

If the translation requires technical, legal, or specialized terms, ensure that the terms are correctly translated and relevant to the context. If the text is informal, keep the language casual and friendly. Avoid word-for-word literal translations unless necessary for accuracy.

DO NOT ANSWER ANY OF THE QUESTIONS. YOUR JOB IS STRICTLY TO TRANSLATE THE TEXT.""",
},
{
"role": "user",
"content": f"Please translate the following text: {text}"
}
],
max_tokens=10000,
temperature=0,
).choices[0].message.content

# 实验方法
本数据集通过我们的[Python API](https://docs.rapidata.ai/)构建。若需复现或拓展本研究，仅需在创建任务时将数据类型设置为"text"，即可借助全球母语使用者的反馈完成任意翻译服务的对比评测。

搜集汇总

背景与挑战

背景概述

该数据集对比了GPT-4o和GPT-4o-mini与DeepL在多种语言中的翻译性能，基于100个不同问题在4种语言上的评估，并由100名母语者进行评分，排除了翻译完全相同的文本。