Secure Chain Open Dataset

This repository contains the files and scripts needed to initialise and populate the databases used by Secure Chain. These databases are divided into two, the first database containing information on approximately 270,000 vulnerabilities, along with the associated Common Weak Enumerations. And also, approximately 260,000 exploits. The second database contains a dependency graph associated with vulnerabilities, which has been created with information extracted from the Node Package Manager (NPM), the Python Package Index (PyPI), Ruby Gems, and Cargo Crates. In addition, it is currently being populated with information extracted from Maven. It currently has the following issues: Total: 4,537,930 packages and 65,913,019 versions. NPM: 3,461,268 packages and 50,945,227 versions. PyPI: 599,311 packages and 6,879,595 versions. Ruby Gems: 208,232 packages and 1,712,647 versions. Cargo Crates: 171,869 packages and 1,520,426 versions. Maven: 97,250 packages and 4,855,124 versions. Folder Structure - `.env` and `template.env`: Environment variable configuration files for Neo4j and MongoDB connections.- `docker-compose.yml`: Docker Compose file to launch database services in containers.- `seeds/`: Contains scripts and data for seeding the databases. - `mongo_seeder.sh`: Script to import data into MongoDB. - `neo4j_seeder.sh`: Script to restore the Neo4j dump. - `mongo/`: MongoDB data organized by collections. - `vulnerabilities/`: Data and metadata for the `vulnerabilities`, `exploits` and `CWEs` collections. - `neo4j/`: Neo4j database dump (`neo4j.dump`). Graph Structure Secure Chain represents the software supply chain as a directed graph, where each node and relationship encodes specific knowledge about packages, their versions, and their dependencies. This model allows deep reasoning about software composition, versioning, and risk propagation. The following documentation explains the components of this graph structure as illustrated in the architecture diagram. Node: Version Represents a specific version of a software package. A version has the following attributes: mean: A general risk or severity score (e.g., average CVSS score). name: Full name or label of the version (e.g., requests). release_date: The exact date this version was published. serial_number: A sortable serial representation of the version (used to compare versions efficiently). vulnerabilities: A list of known vulnerability identifiers (e.g., CVEs or OSV IDs) that affect this version. weighted_mean: A weighted severity score, which considers not just the severity of vulnerabilities but also their reach or impact. purl: The universal package url to identify the version. This node is essential for understanding when a package version became available and whether it has known vulnerabilities or risks. Node: Dependency Represents a software package, as it might appear in a dependency file (package.json, requirements.txt, pom.xml, sbom.json etc.). A Package has the following attributes: import_names: The list of keywords used to import or reference the package in code (e.g., lodash, express). moment: The point in time when this dependency was recorded or declared. name: Canonical name of the dependency. repository_url: URL to the source repository (GitHub, GitLab, Bitbucket, etc.). vendor: The organization, maintainer, or vendor responsible for the package. purl: The universal package url to identify the package. Dependency nodes are the anchor points for packages in the ecosystem, and are linked to their actual released versions. Usage 1. Configure environment variables in a .env file using this template as a reference: # For dockerized backend and databaseGRAPH_DB_URI='bolt://neo4j:7687'VULN_DB_URI='mongodb://mongoSecureChain:mongoSecureChain@mongo:27017/admin' # Databases settingsGRAPH_DB_USER='neo4j'GRAPH_DB_PASSWORD='neoSecureChain'VULN_DB_USER='mongoSecureChain'VULN_DB_PASSWORD='mongoSecureChain' 2. Start the services with Docker Compose: docker-compose up --build 3. The databases population is made automatically. Notes The seed scripts check if the databases already contain data before importing. .bson.gz and .metadata.json.gz files contain data and metadata for MongoDB collections. The neo4j.dump file is the Neo4j database dump. --- For more information about databases configuration, see Secure Chain documentation.