GitHub Workflows in Open Source Repositories

Overview This dataset provides a large collection of GitHub Actions workflows, their execution history, and associated metadata. The database contains information from more than 5000 unique repositories, capturing both workflow definitions and their runtime behavior across nearly one million workflow runs. Important: This is a living dataset that is continuously growing. We publish monthly snapshots to provide stable reference points for research and analysis. The current snapshot reflects data collected through September 2025. Check back regularly for updated monthly releases containing new repositories, workflows, and execution data. Dataset updated on 13th of November 2025 Collections The dataset is organized into five interconnected collections: 1. workflow_files The primary collection containing workflow definitions and their metadata. Each document represents a single GitHub Actions workflow file from a repository. Key Fields: Repository identification (owner, repo, workflow_id) Workflow metadata (name, path, state, creation/update timestamps) Parsed workflow structure including jobs, steps, and configurations Actions usage tracking (which GitHub Actions are used) Trigger configurations (push, pull_request, schedule, etc.) Programming language detection Repository metadata (stars, forks, topics, license) AI classification (whether the workflow involves AI-powered CI/CD) Run statistics (total runs, success/failure counts, timing information) Sample Use Cases: Analyze workflow patterns and best practices Study the adoption of specific GitHub Actions Identify correlations between repository characteristics and workflow design Track AI integration in CI/CD pipelines 2. workflow_runs Contains execution records for workflow runs, tracking when workflows were triggered and their outcomes. Key Fields: Unique run identifiers and associations to workflows Execution metadata (status, conclusion, timestamps) Trigger information (event type, branch, commit SHA) Actor information (who triggered the run) Pull request associations 3. jobs_of_runs Provides detailed information about individual jobs within workflow runs, including step-by-step execution data. Key Fields: Job identification and association to workflow runs Execution status and timing (created, started, completed timestamps) Runner information (runner ID, name, group) Individual step details with status and timing Conclusion for each step Key Insights: Average of 1.89 jobs per workflow run Detailed step-level execution tracking including setup and teardown steps 4. workflow_files_commits Tracks the evolution of workflow files through commit history. Key Fields: Commit metadata (SHA, date, message, author) File state at each commit (file SHA, size) Parsed workflow configuration at that point in time Actions and triggers used in each version Sample Use Cases: Study workflow evolution patterns Analyze how teams iterate on CI/CD configurations Identify common workflow improvements over time Data Collection and Updates Monthly Snapshots. The dataset is continuously growing as we monitor and collect new workflow data from GitHub repositories. To support reproducible research while providing access to the latest data, we publish monthly snapshots. Each snapshot is: Versioned by month and year (e.g., September 2025, October 2025) Cumulative, potentially including previously collected repositories plus new additions Independently restorable, allowing researchers to work with specific time periods Documented with statistics reflecting the data contained in that particular snapshot Researchers can choose to work with a single snapshot for consistency or compare across multiple snapshots to study temporal trends in CI/CD practices. Database Setup and Restoration Prerequisites MongoDB installed on your system Sufficient disk space (the dataset requires approximately 5-10 GB) MongoDB tools including `mongorestore` Connection String mongosh --host localhost --port 27017 --db github-workflows-september-2025 Restoring from Backup If you have received this dataset as a MongoDB dump, you can restore it using the following command: mongorestore --host localhost --port 27017 --db github-workflows-september-2025 /path/to/dump/directory For a compressed archive: Extract the archive first tar -xzvf github-workflows-september-2025.tar.gz Then, restore the database mongorestore --host localhost --port 27017 --db github-workflows-september-2025 ./github-workflows-september-2025 Note: When working with multiple monthly snapshots, you can restore them to different database names to allow side-by-side comparison: mongorestore --host localhost --port 27017 --db github-workflows-october-2025 ./github-workflows-october-2025 mongorestore --host localhost --port 27017 --db github-workflows-november-2025 ./github-workflows-november-2025