(No) Influence Of Continuous Integration On The Development Activity In Github Projects — Dataset
收藏资源简介：
This dataset is based on the TravisTorrent dataset released 2017-01-11 (https://travistorrent.testroots.org), the Google BigQuery GHTorrent dataset accessed 2017-07-03, and the Git log history of all projects in the dataset, retrieved 2017-07-16 and 2017-07-17. We selected projects hosted on GitHub that employ the Continuous Integration (CI) system Travis CI. We identified the projects using the TravisTorrent data set and considered projects that: used GitHub from the beginning (first commit not more than seven days before project creation date according to GHTorrent), were active for at least one year (365 days) before the first build with Travis CI (before_ci), used Travis CI at least for one year (during_ci), had commit or merge activity on the default branch in both of these phases, and used the default branch to trigger builds. To derive the time frames, we employed the GHTorrent Big Query data set. The resulting sample contains 113 projects. Of these projects, 89 are Ruby projects and 24 are Java projects. For our analysis, we only consider the activity one year before and after the first build. We cloned the selected project repositories and extracted the version history for all branches (see https://github.com/sbaltes/git-log-parser). For each repo and branch, we created one log file with all regular commits and one log file with all merges. We only considered commits changing non-binary files and applied a file extension filter to only consider changes to Java or Ruby source code files. From the log files, we then extracted metadata about the commits and stored this data in CSV files (see https://github.com/sbaltes/git-log-parser). We also retrieved a random sample of GitHub project to validate the effects we observed in the CI project sample. We only considered projects that: have Java or Ruby as their project language used GitHub from the beginning (first commit not more than seven days before project creation date according to GHTorrent) have commit activity for at least two years (730 days) are engineered software projects (at least 10 watchers) were not in the TravisTorrent dataset In total, 8,046 projects satisfied those constraints. We drew a random sample of 100 projects from this sampling frame and retrieve the commit and merge data ni the same way as for the CI sample. This dataset contains the following files: <strong>tr_projects_sample_filtered_2.csv</strong><br> A CSV file with information about the 113 selected projects. <strong>tr_sample_commits_default_branch_before_ci.csv<br> tr_sample_commits_default_branch_during_ci.csv</strong><br> One CSV file with information about all commits to the default branch before and after the first CI build. Only commits modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the following columns: project: GitHub project name ("/" replaced by "_").<br> branch: The branch to which the commit was made.<br> hash_value: The SHA1 hash value of the commit.<br> author_name: The author name.<br> author_email: The author email address.<br> author_date: The authoring timestamp.<br> commit_name: The committer name.<br> commit_email: The committer email address.<br> commit_date: The commit timestamp.<br> log_message_length: The length of the git commit messages (in characters).<br> file_count: Files changed with this commit.<br> lines_added: Lines added to all files changed with this commit.<br> lines_deleted: Lines deleted in all files changed with this commit.<br> file_extensions: Distinct file extensions of files changed with this commit. <strong>tr_sample_merges_default_branch_before_ci.csv<br> tr_sample_merges_default_branch_during_ci.csv</strong><br> One CSV file with information about all merges into the default branch before and after the first CI build. Only merges modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the following columns: project: GitHub project name ("/" replaced by "_").<br> branch: The destination branch of the merge.<br> hash_value: The SHA1 hash value of the merge commit.<br> merged_commits: Unique hash value prefixes of the commits merged with this commit.<br> author_name: The author name.<br> author_email: The author email address.<br> author_date: The authoring timestamp.<br> commit_name: The committer name.<br> commit_email: The committer email address.<br> commit_date: The commit timestamp.<br> log_message_length: The length of the git commit messages (in characters).<br> file_count: Files changed with this commit.<br> lines_added: Lines added to all files changed with this commit.<br> lines_deleted: Lines deleted in all files changed with this commit.<br> file_extensions: Distinct file extensions of files changed with this commit.<br> pull_request_id: ID of the GitHub pull request that has been merged with this commit (extracted from log message).<br> source_user: GitHub login name of the user who initiated the pull request (extracted from log message).<br> source_branch : Source branch of the pull request (extracted from log message). <strong>comparison_project_sample_100.csv</strong><br> A CSV file with information about the 100 projects in the comparison sample. <strong>commits_default_branch_before_mid.csv<br> commits_default_branch_after_mid.csv</strong><br> One CSV file with information about all commits to the default branch before and after the medium date of the commit history. Only commits modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the same columns as the commits tables described above. <strong>merges_default_branch_before_mid.csv<br> merges_default_branch_after_mid.csv</strong><br> One CSV file with information about all merges into the default branch before and after the medium date of the commit history. Only merges modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the same columns as the merge tables described above.
本数据集基于2017年1月11日发布的TravisTorrent数据集（https://travistorrent.testroots.org）、2017年7月3日获取的Google BigQuery GHTorrent数据集，以及2017年7月16日、2017年7月17日获取的数据集中所有项目的Git日志历史。 我们选取了托管于GitHub且采用持续集成（Continuous Integration，CI）系统Travis CI的项目。我们通过TravisTorrent数据集识别目标项目，并筛选满足以下条件的项目： 1. 自项目创建起便使用GitHub（根据GHTorrent数据，首次提交距项目创建日期不超过7天）； 2. 在首次使用Travis CI构建前（before_ci阶段）至少活跃1年（365天）； 3. 使用Travis CI至少1年（during_ci阶段）； 4. 在上述两个阶段中，默认分支均存在提交或合并活动； 5. 通过默认分支触发构建。 我们借助GHTorrent BigQuery数据集确定时间区间。最终筛选得到113个项目，其中89个为Ruby项目，24个为Java项目。本次分析仅关注首次构建前后各1年的活动数据。 我们克隆了筛选出的项目仓库，并提取所有分支的版本历史（详见https://github.com/sbaltes/git-log-parser）。针对每个仓库及其分支，我们分别生成包含所有常规提交的日志文件，以及包含所有合并操作的日志文件。我们仅考虑修改非二进制文件的提交，并应用文件扩展名过滤器，仅保留Java或Ruby源代码文件的变更。随后，我们从日志文件中提取提交元数据，并将其存储为CSV文件（详见https://github.com/sbaltes/git-log-parser）。 我们还选取了GitHub项目的随机样本，用于验证CI项目样本中观测到的效应。我们筛选满足以下条件的项目： 1. 项目语言为Java或Ruby； 2. 自项目创建起便使用GitHub（根据GHTorrent数据，首次提交距项目创建日期不超过7天）； 3. 提交活动至少持续2年（730天）； 4. 为工程化软件项目（至少拥有10个关注者）； 5. 未出现在TravisTorrent数据集中。 总计有8046个项目符合上述约束条件。我们从该抽样框架中随机抽取100个项目，并以与CI样本相同的方式获取提交与合并数据。 本数据集包含以下文件： **tr_projects_sample_filtered_2.csv**：包含113个筛选项目信息的CSV文件。 **tr_sample_commits_default_branch_before_ci.csv** **tr_sample_commits_default_branch_during_ci.csv**：包含首次CI构建前后默认分支所有提交信息的CSV文件。仅考虑修改、添加或删除Java或Ruby源代码文件的提交。这些CSV文件包含以下字段： - project：GitHub项目名称（"/"已替换为"_"） - branch：提交所属分支 - hash_value：提交的SHA1哈希值 - author_name：提交作者姓名 - author_email：提交作者电子邮箱 - author_date：提交创作时间戳 - commit_name：提交者姓名 - commit_email：提交者电子邮箱 - commit_date：提交时间戳 - log_message_length：Git提交消息的长度（单位：字符） - file_count：本次提交修改的文件数量 - lines_added：本次提交修改的所有文件新增的代码行数 - lines_deleted：本次提交修改的所有文件删除的代码行数 - file_extensions：本次提交修改文件的不同扩展名 **tr_sample_merges_default_branch_before_ci.csv** **tr_sample_merges_default_branch_during_ci.csv**：包含首次CI构建前后合并至默认分支的所有合并操作信息的CSV文件。仅考虑修改、添加或删除Java或Ruby源代码文件的合并操作。这些CSV文件包含以下字段： - project：GitHub项目名称（"/"已替换为"_"） - branch：合并操作的目标分支 - hash_value：合并提交的SHA1哈希值 - merged_commits：本次合并所包含提交的唯一哈希前缀 - author_name：合并提交的作者姓名 - author_email：合并提交的作者电子邮箱 - author_date：合并提交创作时间戳 - commit_name：合并提交者姓名 - commit_email：合并提交者电子邮箱 - commit_date：合并提交时间戳 - log_message_length：Git提交消息的长度（单位：字符） - file_count：本次合并提交修改的文件数量 - lines_added：本次合并提交修改的所有文件新增的代码行数 - lines_deleted：本次合并提交修改的所有文件删除的代码行数 - file_extensions：本次合并提交修改文件的不同扩展名 - pull_request_id：从提交消息中提取的本次合并所对应的GitHub拉取请求（Pull Request）ID - source_user：从提交消息中提取的发起拉取请求的GitHub用户名 - source_branch：从提交消息中提取的拉取请求源分支 **comparison_project_sample_100.csv**：包含100个对照样本项目信息的CSV文件。 **commits_default_branch_before_mid.csv** **commits_default_branch_after_mid.csv**：包含提交历史中值日期前后默认分支所有提交信息的CSV文件。仅考虑修改、添加或删除Java或Ruby源代码文件的提交。这些CSV文件的字段与上述提交表一致。 **merges_default_branch_before_mid.csv** **merges_default_branch_after_mid.csv**：包含提交历史中值日期前后合并至默认分支的所有合并操作信息的CSV文件。仅考虑修改、添加或删除Java或Ruby源代码文件的合并操作。这些CSV文件的字段与上述合并表一致。
