code_x_glue_cc_code_to_code_trans

# Dataset Card for "code_x_glue_cc_code_to_code_trans"

## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks and Leaderboards](#supported-tasks)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-fields)
- [Data Splits](#data-splits-sample-size)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Contributions](#contributions)

## Dataset Description

- **Homepage:** https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-to-code-trans
- **Paper:** https://arxiv.org/abs/2102.04664

### Dataset Summary

CodeXGLUE code-to-code-trans dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-to-code-trans

The dataset is collected from several public repos, including Lucene(http://lucene.apache.org/), POI(http://poi.apache.org/), JGit(https://github.com/eclipse/jgit/) and Antlr(https://github.com/antlr/).

We collect both the Java and C# versions of the codes and find the parallel functions. After removing duplicates and functions with the empty body, we split the whole dataset into training, validation and test sets.

### Supported Tasks and Leaderboards

- `machine-translation`: The dataset can be used to train a model for translating code in Java to C# and vice versa.

### Languages

- Java **programming** language
- C# **programming** language

## Dataset Structure

### Data Instances

An example of 'validation' looks as follows.
```
{
"cs": "public DVRecord(RecordInputStream in1){_option_flags = in1.ReadInt();_promptTitle = ReadUnicodeString(in1);_errorTitle = ReadUnicodeString(in1);_promptText = ReadUnicodeString(in1);_errorText = ReadUnicodeString(in1);int field_size_first_formula = in1.ReadUShort();_not_used_1 = in1.ReadShort();_formula1 = NPOI.SS.Formula.Formula.Read(field_size_first_formula, in1);int field_size_sec_formula = in1.ReadUShort();_not_used_2 = in1.ReadShort();_formula2 = NPOI.SS.Formula.Formula.Read(field_size_sec_formula, in1);_regions = new CellRangeAddressList(in1);}\n",
"id": 0,
"java": "public DVRecord(RecordInputStream in) {_option_flags = in.readInt();_promptTitle = readUnicodeString(in);_errorTitle = readUnicodeString(in);_promptText = readUnicodeString(in);_errorText = readUnicodeString(in);int field_size_first_formula = in.readUShort();_not_used_1 = in.readShort();_formula1 = Formula.read(field_size_first_formula, in);int field_size_sec_formula = in.readUShort();_not_used_2 = in.readShort();_formula2 = Formula.read(field_size_sec_formula, in);_regions = new CellRangeAddressList(in);}\n"
}
```

### Data Fields

In the following each data field in go is explained for each config. The data fields are the same among all splits.

#### default

|field name| type | description |
|----------|------|-----------------------------|
|id |int32 | Index of the sample |
|java |string| The java version of the code|
|cs |string| The C# version of the code |

### Data Splits

| name |train|validation|test|
|-------|----:|---------:|---:|
|default|10300| 500|1000|

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

https://github.com/microsoft, https://github.com/madlag

### Licensing Information

Computational Use of Data Agreement (C-UDA) License.

### Citation Information

```
@article{DBLP:journals/corr/abs-2102-04664,
author = {Shuai Lu and
Daya Guo and
Shuo Ren and
Junjie Huang and
Alexey Svyatkovskiy and
Ambrosio Blanco and
Colin B. Clement and
Dawn Drain and
Daxin Jiang and
Duyu Tang and
Ge Li and
Lidong Zhou and
Linjun Shou and
Long Zhou and
Michele Tufano and
Ming Gong and
Ming Zhou and
Nan Duan and
Neel Sundaresan and
Shao Kun Deng and
Shengyu Fu and
Shujie Liu},
title = {CodeXGLUE: {A} Machine Learning Benchmark Dataset for Code Understanding
and Generation},
journal = {CoRR},
volume = {abs/2102.04664},
year = {2021}
}
```

### Contributions

Thanks to @madlag (and partly also @ncoop57) for adding this dataset.

# 「code_x_glue_cc_code_to_code_trans」数据集卡片

## 目录
- [数据集描述](#dataset-description)
- [数据集摘要](#dataset-summary)
- [支持任务与评测基准](#supported-tasks)
- [编程语言](#languages)
- [数据集结构](#dataset-structure)
- [数据实例](#data-instances)
- [数据字段](#data-fields)
- [数据划分](#data-splits-sample-size)
- [数据集构建](#dataset-creation)
- [构建初衷](#curation-rationale)
- [源数据](#source-data)
- [标注信息](#annotations)
- [个人与敏感信息](#personal-and-sensitive-information)
- [数据集使用注意事项](#considerations-for-using-the-data)
- [数据集的社会影响](#social-impact-of-dataset)
- [偏差讨论](#discussion-of-biases)
- [其他已知局限性](#other-known-limitations)
- [附加信息](#additional-information)
- [数据集维护者](#dataset-curators)
- [许可协议](#licensing-information)
- [引用信息](#citation-information)
- [贡献致谢](#contributions)

## 数据集描述

- **主页：** https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-to-code-trans
- **论文：** https://arxiv.org/abs/2102.04664

### 数据集摘要

CodeXGLUE代码到代码翻译数据集，获取地址为https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-to-code-trans。

该数据集采集自多个公开代码仓库，包括Lucene(http://lucene.apache.org/)、POI(http://poi.apache.org/)、JGit(https://github.com/eclipse/jgit/)以及Antlr(https://github.com/antlr/)。

我们采集了代码的Java与C#两个版本，并匹配得到并行等价函数。在去除重复样本与空体函数后，我们将完整数据集划分为训练集、验证集与测试集。

### 支持任务与评测基准

- `机器翻译`：该数据集可用于训练模型，实现Java代码与C#代码之间的互相转换。

### 编程语言

- Java编程语言
- C#编程语言

## 数据集结构

### 数据实例

「验证集」的一个示例如下所示。
json
{
"cs": "public DVRecord(RecordInputStream in1){_option_flags = in1.ReadInt();_promptTitle = ReadUnicodeString(in1);_errorTitle = ReadUnicodeString(in1);_promptText = ReadUnicodeString(in1);_errorText = ReadUnicodeString(in1);int field_size_first_formula = in1.ReadUShort();_not_used_1 = in1.ReadShort();_formula1 = NPOI.SS.Formula.Formula.Read(field_size_first_formula, in1);int field_size_sec_formula = in1.ReadUShort();_not_used_2 = in1.ReadShort();_formula2 = NPOI.SS.Formula.Formula.Read(field_size_sec_formula, in1);_regions = new CellRangeAddressList(in1);}
",
"id": 0,
"java": "public DVRecord(RecordInputStream in) {_option_flags = in.readInt();_promptTitle = readUnicodeString(in);_errorTitle = readUnicodeString(in);_promptText = readUnicodeString(in);_errorText = readUnicodeString(in);int field_size_first_formula = in.readUShort();_not_used_1 = in.readShort();_formula1 = Formula.read(field_size_first_formula, in);int field_size_sec_formula = in.readUShort();_not_used_2 = in.readShort();_formula2 = Formula.read(field_size_sec_formula, in);_regions = new CellRangeAddressList(in);}
"
}

### 数据字段

下文将针对各配置项逐一解释数据字段。所有数据划分下的字段均保持一致。

#### 默认配置

|字段名|类型|描述|
|----------|------|-----------------------------|
|id |int32 | 样本索引 |
|java |string| 代码的Java版本|
|cs |string| 代码的C#版本 |

### 数据划分

|划分名称|训练集|验证集|测试集|
|-------|----:|---------:|---:|
|default|10300| 500|1000|

## 数据集构建

### 构建初衷

[需补充更多信息]

### 源数据

#### 初始数据采集与标准化

[需补充更多信息]

#### 源数据生产者

[需补充更多信息]

### 标注信息

#### 标注流程

[需补充更多信息]

#### 标注者

[需补充更多信息]

### 个人与敏感信息

[需补充更多信息]

## 数据集使用注意事项

### 数据集的社会影响

[需补充更多信息]

### 偏差讨论

[需补充更多信息]

### 其他已知局限性

[需补充更多信息]

## 附加信息

### 数据集维护者

https://github.com/microsoft, https://github.com/madlag

### 许可协议

计算数据使用协议（Computational Use of Data Agreement，C-UDA）许可。

### 引用信息

bibtex
@article{DBLP:journals/corr/abs-2102.04664,
author = {Shuai Lu and
Daya Guo and
Shuo Ren and
Junjie Huang and
Alexey Svyatkovskiy and
Ambrosio Blanco and
Colin B. Clement and
Dawn Drain and
Daxin Jiang and
Duyu Tang and
Ge Li and
Lidong Zhou and
Linjun Shou and
Long Zhou and
Michele Tufano and
Ming Gong and
Ming Zhou and
Nan Duan and
Neel Sundaresan and
Shao Kun Deng and
Shengyu Fu and
Shujie Liu},
title = {CodeXGLUE: {A} Machine Learning Benchmark Dataset for Code Understanding and Generation},
journal = {CoRR},
volume = {abs/2102.04664},
year = {2021}
}

### 贡献致谢

感谢@madlag（部分感谢@ncoop57）贡献本数据集的添加工作。