Readme Files In 16,000,000 Public Github Repositories (October 2016)
收藏资源简介：
Format index.csv.gz - CSV comma separated file with 3 columns: <repository name>, <flag>,<readme file name> For example: src-d/go-git,s,README.md The flag is either "s" (readme found) or "r" (readme does not exist on the root directory level). Readme file name may be any from the list: "README.md", "readme.md", "Readme.md", "README.MD", "README.txt", "readme.txt", "Readme.txt", "README.TXT", "README", "readme", "Readme", "README.rst", "readme.rst", "Readme.rst", "README.RST" 100 part-r-00xxx files are in "new" Hadoop API format with the following settings: inputFormatClass is org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat keyClass is org.apache.hadoop.io.Text - repository name valueClass is org.apache.hadoop.io.BytesWritable - gzipped readme file
### 数据集格式说明 1. **index.csv.gz**：为逗号分隔值（CSV）文件，包含三列，依次为`<仓库名称>`、`<标记>`与`<README文件名>`。示例：`src-d/go-git,s,README.md`。 其中标记仅可取值为`s`（根目录下存在README文件）或`r`（根目录无README文件）。允许的README文件名集合如下： `"README.md", "readme.md", "Readme.md", "README.MD", "README.txt", "readme.txt", "Readme.txt", "README.TXT", "README", "readme", "Readme", "README.rst", "readme.rst", "Readme.rst", "README.RST"` 2. 数据集包含100个`part-r-00xxx`格式文件，采用新版Hadoop API格式，具体配置参数如下： - 输入格式类（inputFormatClass）：`org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat` - 键类（keyClass）：`org.apache.hadoop.io.Text`，对应存储内容为仓库名称 - 值类（valueClass）：`org.apache.hadoop.io.BytesWritable`，对应存储内容为经过gzip压缩的README文件
