C4Audit: A Dataset for Empirical Analysis of Code Quality and Automated Security Detection in Smart Contracts

C4Audit - Dataset Description C4Audit is a curated, structured dataset that bridges real-world smart contract audit reports with their corresponding source code and quantitative code metrics.It was built from Code4rena audit contests ( a large community-driven security auditing platform) and is organized in three sub datasets, where: C4Audit-Reports: Contains a folder named reports_parser. For each audit, there is a report (e.g., reports_parser/2025-03-silo-finance/ contains the report for this audit). The names of the directories were based on the report identifier: https://code4rena.com/reports/2025-03-silo-finance C4Audit_Repos: contains a folder named repositories. Similarly, we followed the same structure for the repositories to allow identify the code for the respective report. C4Audit_Metrics: contains a folder named C4Audit_metrics. This folder consists of a subfolder with code metrics for each audit in CSV format and also, additional CSV files with classifications for the type of code. The content of each sub dataset is detailed in the next sections. 1. C4Audit-Reports - Parsed Audit Reports Structured JSON files derived from the original Code4rena audit reports. Each parsed report includes: Metadata — audit name, date, protocol name, repository links, scope information (number of contracts, LOC). Vulnerability Findings — all issues identified by wardens, categorized by severity (High, Medium, Low). Issue Attributes: issue_id: unique identifier title: short description of the finding severity: severity classification description: natural language explanation of the bug vulnerable_code_links: URLs pointing to affected files and line numbers in the audited repository These structured fields enable linking each vulnerability directly to its corresponding source file and function in the associated GitHub repository. For example, the contest of the audit 2024-02-wise-lending: { "report_title": "Wise Lending Findings & Analysis Report", "scope": { "contracts": 44, "lines_solidity": 6326, "repositories": [ "https://github.com/code-423n4/2024-02-wise-lending" ] }, "issues": [ { "issue_id": "H-01", "title": "Exploitation of the receive Function to Steal Funds", "severity": "High", "description": "...The WiseLending contract incorporates a reentrancy guard through its syncPool modifier, specifically within the _syncPoolBeforeCodeExecution function. ...", "vulnerable_code_links": [ "https://github.com/code-423n4/2024-02-wise-lending/blob/79186b243d85 53e66358c05497e5ccfd9488b5e2/contracts/WiseLending.sol#L49", ..., "https://github.com/code-423n4/2024-02-wise-lending/blob/79186b243d85 53e66358c05497e5ccfd9488b5e2/contracts/WiseLending.sol#L636", "https://github.com/code-423n4/2024-02-wise-lending/blob/79186b243d85 53e66358c05497e5ccfd9488b5e2/contracts/TransferHub/SendValueHelper .sol#L12" ] }, { "issue_id": "H-02", "title": "User can erase their position debt for free", "severity": "High", "description": "...In the case of function FeeManager.paybackBadDebtNoReward() , which indirectly calls _removePositionData , insufficient validation doesn’t check if repay token is in user array, which results in zeroing out information about user debt. ...", "vulnerable_code_links": [ ... ] } The issue H-01 includes a title, a detailed description of the problem, and several links pointing to the affected code paths, which span multiple functions on the execution flow. 2. C4Audit-Repos - Linked Repositories Each audit links to one or more GitHub repositories that were within scope during the audit. For reproducibility, the dataset stores the repository URLs (in the report), commit hashes if needed (in the repo README file for each repository), and a local copy of the source code. All the reports point to https://github.com/code-423n4 + the audit name (e.g., 2022-11-debtdao). And several times, the actual code to be cloned was in its README file. The cloning process required manual clone for 34 repositories because they not followed different structure to clone and it became complicated to automatize all of them. For example: The 2022-11-debtdao audit, the actual smart contracts were hosted in an external repository. This information was found in the README file in the https://github.com/code-423n4/2022-11-debtdao. One example of the standard Code4rena structure is for https://github.com/code-423n4/2025-07-lido-finance. Use the command: cat C4Audit_Repos.tar.xz.part-* | tar -xzvf - to uncompress 4. C4Audit-Metrics 4.1. Code Metrics (Lizard Output) Quantitative metrics are extracted for every file and function using the Lizard static analysis tool. Each audit directory contains CSV files with raw and aggregated metrics: Function-level metrics (functions.csv): function_name nloc — lines of code (excluding comments) ccn — cyclomatic complexity token_count and parameter_count start_line and end_line of the function 4.2. Dataset Organization We also classified the code files for each repository from Lizard CSV files to distinguish between key code types: Code/ — business logic smart contracts (Solidity .sol files within src/ or contracts/) Test/ — testing and validation files (test/, tests/, .t.sol, or mock harnesses) DeployScript/ — deployment and automation scripts (scripts/, deploy*/, and tasks/ folders) ExternalDependency/ — imported libraries and third-party packages (lib/ (that is outside the Code folders), node_modules/, artifacts/, etc.) This hierarchical classification helps to conduct a fine-grained analyses of modularity, dependency use, and testing practices in DeFi protocols. This information can be found in merged_lizard_classified.csv. C4Audit_metrics/ ├── Lizard_metrics_csv/ │ ├── 2024-12-lido-oracle/ │ └── 2025-03-silo-finance/ │ └── ... ├── Code_metrics.csv ├── Test_metrics.csv ├── DeployScript_metrics.csv ├── ExternalDependency_metrics.csv └── merged_lizard_classified.csvThe remaining file is a summary of how many contracts by repository we found. For that, we collected the information frommerged_lizard_classified.csv. 5. Dataset Features and Applications Scale: 342 audits, ≈10k smart contracts as code (there are smart contracts implemented as tests), 15k+ test files (.sol, .py, .ts, .js), ≈6k deployment scripts. Coverage: All vulnerability classes reported by the Code4rena community to date. Linkage: Ground-truth vulnerabilities are mapped to actual source code and metrics. Use Cases: Empirical studies on code complexity, modularity, and testing practices Machine learning models for automated vulnerability detection Large-scale benchmarking of static analysis tools Mining studies on DeFi ecosystem architecture and security patterns