Hadoop Dataset: Fine-Grained Code Changes (Diff-Level) Linked to Issue Descriptions for Software Traceability Analysis
收藏资源简介：
Dataset: Apache Hadoop Commit–Task TraceabilityProject: apache/hadoop | Organization: Apache Software Foundation | Language: Java (.java .xml .sh) 1. CONTEXT AND MOTIVATIONThis dataset supports research into traceability between task descriptions and code changes indistributed filesystem and resource-management infrastructure. Apache Hadoop is the foundationalJava framework for large-scale distributed storage (HDFS) and processing (YARN/MapReduce), witha development history spanning over 16 years. Its Apache Jira tracker uses structured identifiers(HADOOP-NNNN) that are consistently embedded in commit messages, yielding one of the highesttask-linking rates in this collection (75 %). The mix of Java, XML configuration, and shellscripts provides a diverse file-type profile for file-level analysis. 2. COLLECTION METHODOLOGYSource: GitHub repository — https://github.com/apache/hadoopCommits: Extracted via the GitHub API (all branches, full history)Task linking: Jira ticket references extracted from commit messages using the pattern HADOOP-[0-9]+ ; each reference was resolved via the Apache Jira APIIssue content: Title, description, and comment thread fetched via Apache Jira REST API (https://issues.apache.org/jira/projects/HADOOP) for each referenced ticketTime range: 2009-05-19 to 2025-10-16Anonymization: Author names and e-mail addresses replaced with sequential pseudonyms (User1, User2, …) prior to publication 3. DATASET STRUCTURETable COMMITS — one row per commit ID INTEGER Primary key SHA TEXT Full commit hash AUTHOR_NAME TEXT Anonymized author pseudonym (e.g. User42) AUTHOR_EMAIL TEXT Anonymized e-mail (e.g. user42@anon.local) CMT_DATE TEXT Commit timestamp (ISO-8601 with timezone) MESSAGE BLOB Full commit message text PATH BLOB List of file paths changed in this commit DIFF BLOB Unified diff of the commit TASK_NAME TEXT Linked Jira ticket ID, e.g. HADOOP-12345 (NULL if no link detected) Table TASK — one row per unique linked Jira ticket ID INTEGER Primary key (autoincrement) NAME TEXT Jira ticket ID (matches TASK_NAME in COMMITS) TITLE TEXT Issue summary as fetched from Jira DESCRIPTION TEXT Issue description body COMMENTS TEXT Serialized comment thread 4. BASIC STATISTICSTotal commits: 259,081Commits with linked task: 194,330 (75.0 %)Commits without linked task: 64,751 (25.0 %)Unique linked issues: 24,719Issues with description text: 23,495 (95.0 %)Unique authors (anonymized): 1,035Date range: 2009-05-19 — 2025-10-16
