google/code_x_glue_cc_cloze_testing_maxmin

---
annotations_creators:
- found
language_creators:
- found
language:
- code
license:
- c-uda
multilinguality:
- monolingual
size_categories:
- 10K<n<100K
- 1K<n<10K
source_datasets:
- original
task_categories:
- text-generation
- fill-mask
task_ids:
- slot-filling
pretty_name: CodeXGlueCcClozeTestingMaxmin
config_names:
- go
- java
- javascript
- php
- python
- ruby
dataset_info:
- config_name: go
features:
- name: id
dtype: int32
- name: idx
dtype: string
- name: nl_tokens
sequence: string
- name: pl_tokens
sequence: string
splits:
- name: train
num_bytes: 204977
num_examples: 152
download_size: 68965
dataset_size: 204977
- config_name: java
features:
- name: id
dtype: int32
- name: idx
dtype: string
- name: nl_tokens
sequence: string
- name: pl_tokens
sequence: string
splits:
- name: train
num_bytes: 785734
num_examples: 482
download_size: 250672
dataset_size: 785734
- config_name: javascript
features:
- name: id
dtype: int32
- name: idx
dtype: string
- name: nl_tokens
sequence: string
- name: pl_tokens
sequence: string
splits:
- name: train
num_bytes: 594327
num_examples: 272
download_size: 188271
dataset_size: 594327
- config_name: php
features:
- name: id
dtype: int32
- name: idx
dtype: string
- name: nl_tokens
sequence: string
- name: pl_tokens
sequence: string
splits:
- name: train
num_bytes: 705457
num_examples: 407
download_size: 211107
dataset_size: 705457
- config_name: python
features:
- name: id
dtype: int32
- name: idx
dtype: string
- name: nl_tokens
sequence: string
- name: pl_tokens
sequence: string
splits:
- name: train
num_bytes: 2566333
num_examples: 1264
download_size: 885488
dataset_size: 2566333
- config_name: ruby
features:
- name: id
dtype: int32
- name: idx
dtype: string
- name: nl_tokens
sequence: string
- name: pl_tokens
sequence: string
splits:
- name: train
num_bytes: 48926
num_examples: 38
download_size: 22528
dataset_size: 48926
configs:
- config_name: go
data_files:
- split: train
path: go/train-*
- config_name: java
data_files:
- split: train
path: java/train-*
- config_name: javascript
data_files:
- split: train
path: javascript/train-*
- config_name: php
data_files:
- split: train
path: php/train-*
- config_name: python
data_files:
- split: train
path: python/train-*
- config_name: ruby
data_files:
- split: train
path: ruby/train-*
---
# Dataset Card for "code_x_glue_cc_cloze_testing_maxmin"

## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks and Leaderboards](#supported-tasks)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-fields)
- [Data Splits](#data-splits-sample-size)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Contributions](#contributions)

## Dataset Description

- **Homepage:** https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/ClozeTesting-maxmin

### Dataset Summary

CodeXGLUE ClozeTesting-maxmin dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/ClozeTesting-maxmin

Cloze tests are widely adopted in Natural Languages Processing to evaluate the performance of the trained language models. The task is aimed to predict the answers for the blank with the context of the blank, which can be formulated as a multi-choice classification problem.
Here we present the two cloze testing datasets in code domain with six different programming languages: ClozeTest-maxmin and ClozeTest-all. Each instance in the dataset contains a masked code function, its docstring and the target word.
The only difference between ClozeTest-maxmin and ClozeTest-all is their selected words sets, where ClozeTest-maxmin only contains two words while ClozeTest-all contains 930 words.

### Supported Tasks and Leaderboards

- `slot-filling`: The dataset can be used to train a model for predicting the missing token from a piece of code, similar to the Cloze test.

### Languages

- Go **programming** language
- Java **programming** language
- Javascript **programming** language
- PHP **programming** language
- Python **programming** language
- Ruby **programming** language

## Dataset Structure

### Data Instances

#### go

An example of 'train' looks as follows.
```
{
"id": 0,
"idx": "maxmin-1",
"nl_tokens": ["SetMaxStructPoolSize", "sets", "the", "struct", "pools", "max", "size", ".", "this", "may", "be", "usefull", "for", "fine", "grained", "performance", "tuning", "towards", "your", "application", "however", "the", "default", "should", "be", "fine", "for", "nearly", "all", "cases", ".", "only", "increase", "if", "you", "have", "a", "deeply", "nested", "struct", "structure", ".", "NOTE", ":", "this", "method", "is", "not", "thread", "-", "safe", "NOTE", ":", "this", "is", "only", "here", "to", "keep", "compatibility", "with", "v5", "in", "v6", "the", "method", "will", "be", "removed"],
"pl_tokens": ["func", "(", "v", "*", "Validate", ")", "SetMaxStructPoolSize", "(", "<mask>", "int", ")", "{", "structPool", "=", "&", "sync", ".", "Pool", "{", "New", ":", "newStructErrors", "}", "\n", "}"]
}
```

#### java

An example of 'train' looks as follows.
```
{
"id": 0,
"idx": "maxmin-1",
"nl_tokens": ["Test", "whether", "find", "can", "be", "found", "at", "position", "startPos", "in", "the", "string", "src", "."],
"pl_tokens": ["public", "static", "boolean", "startsWith", "(", "char", "[", "]", "src", ",", "char", "[", "]", "find", ",", "int", "startAt", ")", "{", "int", "startPos", "=", "startAt", ";", "boolean", "result", "=", "true", ";", "// Check ranges", "if", "(", "src", ".", "length", "<", "startPos", "+", "find", ".", "length", ")", "{", "result", "=", "false", ";", "}", "else", "{", "final", "int", "<mask>", "=", "find", ".", "length", ";", "for", "(", "int", "a", "=", "0", ";", "a", "<", "max", "&&", "result", ";", "a", "++", ")", "{", "if", "(", "src", "[", "startPos", "]", "!=", "find", "[", "a", "]", ")", "{", "result", "=", "false", ";", "}", "startPos", "++", ";", "}", "}", "return", "result", ";", "}"]
}
```

#### javascript

An example of 'train' looks as follows.
```
{
"id": 0,
"idx": "maxmin-1",
"nl_tokens": ["string", ".", "max", "Maximum", "length", "of", "the", "string"],
"pl_tokens": ["function", "(", "string", ")", "{", "// string.check check sting type and size", "return", "(", "(", "typeof", "string", "===", "'string'", "||", "string", "instanceof", "String", ")", "&&", "string", ".", "length", ">=", "this", ".", "<mask>", "&&", "string", ".", "length", "<=", "this", ".", "max", "&&", "(", "!", "this", ".", "match", "||", "string", ".", "match", "(", "this", ".", "match", ")", ")", ")", ";", "}"]
}
```

#### php

An example of 'train' looks as follows.
```
{
"id": 0,
"idx": "maxmin-1",
"nl_tokens": ["Read", "the", "next", "character", "from", "the", "supplied", "string", ".", "Return", "null", "when", "we", "have", "run", "out", "of", "characters", "."],
"pl_tokens": ["public", "function", "readOne", "(", ")", "{", "if", "(", "$", "this", "->", "pos", "<=", "$", "this", "->", "<mask>", ")", "{", "$", "value", "=", "$", "this", "->", "string", "[", "$", "this", "->", "pos", "]", ";", "$", "this", "->", "pos", "+=", "1", ";", "}", "else", "{", "$", "value", "=", "null", ";", "}", "return", "$", "value", ";", "}"]
}
```

#### python

An example of 'train' looks as follows.
```
{
"id": 0,
"idx": "maxmin-1",
"nl_tokens": ["Returns", "intermediary", "colors", "for", "given", "list", "of", "colors", "."],
"pl_tokens": ["def", "_interpolate", "(", "self", ",", "colors", ",", "n", "=", "100", ")", ":", "gradient", "=", "[", "]", "for", "i", "in", "_range", "(", "n", ")", ":", "l", "=", "len", "(", "colors", ")", "-", "1", "x", "=", "int", "(", "1.0", "*", "i", "/", "n", "*", "l", ")", "x", "=", "<mask>", "(", "x", "+", "0", ",", "l", ")", "y", "=", "min", "(", "x", "+", "1", ",", "l", ")", "base", "=", "1.0", "*", "n", "/", "l", "*", "x", "d", "=", "(", "i", "-", "base", ")", "/", "(", "1.0", "*", "n", "/", "l", ")", "r", "=", "colors", "[", "x", "]", ".", "r", "*", "(", "1", "-", "d", ")", "+", "colors", "[", "y", "]", ".", "r", "*", "d", "g", "=", "colors", "[", "x", "]", ".", "g", "*", "(", "1", "-", "d", ")", "+", "colors", "[", "y", "]", ".", "g", "*", "d", "b", "=", "colors", "[", "x", "]", ".", "b", "*", "(", "1", "-", "d", ")", "+", "colors", "[", "y", "]", ".", "b", "*", "d", "a", "=", "colors", "[", "x", "]", ".", "a", "*", "(", "1", "-", "d", ")", "+", "colors", "[", "y", "]", ".", "a", "*", "d", "gradient", ".", "append", "(", "color", "(", "r", ",", "g", ",", "b", ",", "a", ",", "mode", "=", "\"rgb\"", ")", ")", "gradient", ".", "append", "(", "colors", "[", "-", "1", "]", ")", "return", "gradient"]
}
```

#### ruby

An example of 'train' looks as follows.
```
{
"id": 0,
"idx": "maxmin-1",
"nl_tokens": ["Delete", "all", "copies", "that", "are", "older", "than", "the", "max", "age", "provided", "in", "seconds", "."],
"pl_tokens": ["def", "clean", "(", "<mask>", ":", "24", "*", "60", "*", "60", ")", "Futex", ".", "new", "(", "file", ",", "log", ":", "@log", ")", ".", "open", "do", "list", "=", "load", "list", ".", "reject!", "do", "|", "s", "|", "if", "s", "[", ":time", "]", ">=", "Time", ".", "now", "-", "max", "false", "else", "@log", ".", "debug", "(", "\"Copy ##{s[:name]}/#{s[:host]}:#{s[:port]} is too old, over #{Age.new(s[:time])}\"", ")", "true", "end", "end", "save", "(", "list", ")", "deleted", "=", "0", "files", ".", "each", "do", "|", "f", "|", "next", "unless", "list", ".", "find", "{", "|", "s", "|", "s", "[", ":name", "]", "==", "File", ".", "basename", "(", "f", ",", "Copies", "::", "EXT", ")", "}", ".", "nil?", "file", "=", "File", ".", "join", "(", "@dir", ",", "f", ")", "size", "=", "File", ".", "size", "(", "file", ")", "File", ".", "delete", "(", "file", ")", "@log", ".", "debug", "(", "\"Copy at #{f} deleted: #{Size.new(size)}\"", ")", "deleted", "+=", "1", "end", "list", ".", "select!", "do", "|", "s", "|", "cp", "=", "File", ".", "join", "(", "@dir", ",", "\"#{s[:name]}#{Copies::EXT}\"", ")", "wallet", "=", "Wallet", ".", "new", "(", "cp", ")", "begin", "wallet", ".", "refurbish", "raise", "\"Invalid protocol #{wallet.protocol} in #{cp}\"", "unless", "wallet", ".", "protocol", "==", "Zold", "::", "PROTOCOL", "true", "rescue", "StandardError", "=>", "e", "FileUtils", ".", "rm_rf", "(", "cp", ")", "@log", ".", "debug", "(", "\"Copy at #{cp} deleted: #{Backtrace.new(e)}\"", ")", "deleted", "+=", "1", "false", "end", "end", "save", "(", "list", ")", "deleted", "end", "end"]
}
```

### Data Fields

In the following each data field in go is explained for each config. The data fields are the same among all splits.

#### go, java, javascript, php, python, ruby

|field name| type | description |
|----------|----------------|------------------------------|
|id |int32 | Index of the sample |
|idx |string | Original index in the dataset|
|nl_tokens |Sequence[string]| Natural language tokens |
|pl_tokens |Sequence[string]| Programming language tokens |

### Data Splits

| name |train|
|----------|----:|
|go | 152|
|java | 482|
|javascript| 272|
|php | 407|
|python | 1264|
|ruby | 38|

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

Data from CodeSearchNet Challenge dataset.
[More Information Needed]

#### Who are the source language producers?

Software Engineering developers.

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

https://github.com/microsoft, https://github.com/madlag

### Licensing Information

Computational Use of Data Agreement (C-UDA) License.

### Citation Information

```
@article{CodeXGLUE,
title={CodeXGLUE: An Open Challenge for Code Intelligence},
journal={arXiv},
year={2020},
}
@article{feng2020codebert,
title={CodeBERT: A Pre-Trained Model for Programming and Natural Languages},
author={Feng, Zhangyin and Guo, Daya and Tang, Duyu and Duan, Nan and Feng, Xiaocheng and Gong, Ming and Shou, Linjun and Qin, Bing and Liu, Ting and Jiang, Daxin and others},
journal={arXiv preprint arXiv:2002.08155},
year={2020}
}
@article{husain2019codesearchnet,
title={CodeSearchNet Challenge: Evaluating the State of Semantic Code Search},
author={Husain, Hamel and Wu, Ho-Hsiang and Gazit, Tiferet and Allamanis, Miltiadis and Brockschmidt, Marc},
journal={arXiv preprint arXiv:1909.09436},
year={2019}
}
```

### Contributions

Thanks to @madlag (and partly also @ncoop57) for adding this dataset.

原始信息汇总

数据集概述

数据集名称

• 名称: CodeXGLUECcClozeTestingMaxmin
• 别名: CodeXGLUE ClozeTesting-maxmin

数据集基本信息

• 语言: 单语种，涵盖Go、Java、JavaScript、PHP、Python、Ruby编程语言。
• 许可证: C-UDA许可证。
• 数据集大小: 分为两个类别，1K<n<10K和10K<n<100K。
• 数据来源: 原始数据。

任务类型

• 任务类别: 文本生成、填空。
• 具体任务: 槽填充（slot-filling）。

数据集结构

• 数据实例: 每个实例包含一个掩码代码函数、其文档字符串和目标词。
• 数据字段:
  • id: 样本索引，类型为int32。
  • idx: 数据集中的原始索引，类型为string。
  • nl_tokens: 自然语言令牌，类型为string序列。
  • pl_tokens: 编程语言令牌，类型为string序列。
• 数据分割:
  • 训练集:
    • Go: 152个样本，数据大小204977字节。
    • Java: 482个样本，数据大小785734字节。
    • JavaScript: 272个样本，数据大小594327字节。
    • PHP: 407个样本，数据大小705457字节。
    • Python: 1264个样本，数据大小2566333字节。
    • Ruby: 38个样本，数据大小48926字节。

数据集创建

• 数据来源: 来自CodeSearchNet Challenge数据集。
• 语言生产者: 软件工程开发者。

许可证信息

• 许可证: 计算数据使用协议（C-UDA）许可证。

引用信息

@article{CodeXGLUE, title={CodeXGLUE: An Open Challenge for Code Intelligence}, journal={arXiv}, year={2020}, } @article{feng2020codebert, title={CodeBERT: A Pre-Trained Model for Programming and Natural Languages}, author={Feng, Zhangyin and Guo, Daya and Tang, Duyu and Duan, Nan and Feng, Xiaocheng and Gong, Ming and Shou, Linjun and Qin, Bing and Liu, Ting and Jiang, Daxin and others}, journal={arXiv preprint arXiv:2002.08155}, year={2020} } @article{husain2019codesearchnet, title={CodeSearchNet Challenge: Evaluating the State of Semantic Code Search}, author={Husain, Hamel and Wu, Ho-Hsiang and Gazit, Tiferet and Allamanis, Miltiadis and Brockschmidt, Marc}, journal={arXiv preprint arXiv:1909.09436}, year={2019} }