dfalbel/github-r-repos

---
license: other
task_categories:
- text-generation
language:
- code
pretty_name: github-r-repos
size_categories:
- 100K<n<1M
---

## GitHub R repositories dataset

R source files from GitHub.

This dataset has been created using the public GitHub datasets from Google BigQuery.
This is the actual query that has been used to export the data:

```
EXPORT DATA
OPTIONS (
uri = 'gs://your-bucket/gh-r/*.parquet',
format = 'PARQUET') as
(
select
f.id, f.repo_name, f.path,
c.content, c.size
from (
SELECT distinct
id, repo_name, path
FROM `bigquery-public-data.github_repos.files`
where ends_with(path, ".R")
) as f
left join `bigquery-public-data.github_repos.contents` as c on f.id = c.id
)

EXPORT_DATA
OPTIONS (
uri = 'gs://your-bucket/licenses.parquet',
format = 'PARQUET') as
(select * from `bigquery-public-data.github_repos.licenses`)
```

Files were then exported and processed locally with files in the root of this repository.
Datasets in this repository contain data from reositories with different licenses.

The data schema is:

```
id: string
repo_name: string
path: string
content: string
size: int32
license: string
```

Last updated: Jun 6th 2023

原始信息汇总

GitHub R repositories dataset 概述

数据集基本信息

• 许可证: other
• 任务类别: text-generation
• 语言: code
• 数据集大小: 100K<n<1M

数据来源与处理

• 数据集由 Google BigQuery 的公共 GitHub 数据集创建。

• 使用以下 SQL 查询导出数据： sql EXPORT DATA OPTIONS ( uri = gs://your-bucket/gh-r/*.parquet, format = PARQUET) as ( select f.id, f.repo_name, f.path, c.content, c.size from ( SELECT distinct id, repo_name, path FROM bigquery-public-data.github_repos.files where ends_with(path, ".R") ) as f left join bigquery-public-data.github_repos.contents as c on f.id = c.id )

• 数据集包含来自具有不同许可证的存储库的数据。

数据结构

• id: string
• repo_name: string
• path: string
• content: string
• size: int32
• license: string

更新日期

• 最后更新: Jun 6th 2023