Replication Package: Analyzing Misconfiguration Vulnerabilities with Large-Language Models

Welcome to the public repository for the additional content related to the analysis of misconfiguration vulnerabilities with Large-Language Models (LLMs). This repository provides additional information to the empirical analysis of misconfiguration vulnerabilities fetched from the National Vulnerability Database, including the following files: PROMPTS.py: full prompting setup for ground-truth dataset and full dataset GROUND_TRUTH_ORIGINAL.jsonl: 60 CVEs with labeled ground truth data (15 CVEs per misconfiguration vulnerability type) GROUND_TRUTH_CLASSIFICATION_CSMODELS.jsonl: 60 CVEs of the ground truth, classified by four closed-source (CS) models GROUND_TRUTH_CLASSIFICATIONPROCESS_OSMODELS.csv: Classification process of the 60 CVEs of the ground truth, classified by 13 open-source (OS) models FULL_DATASET_CLASSIFICATION_6MODELS.jsonl: 4,231 CVEs, classified by 2 open-source and 4 closed-source models SELECTED_CVE_ANALYSIS.json: 2,491 selected CVEs of the full dataset, used for analyzing meta data, CWE, CVSS base metrics, and CVSS scores. README.txt: readMe file Keywords used for filtering (“*config*”) AND (“*misconfig*” OR “*wrong*” OR “*bad*” OR “*insecure*” OR “*default*” OR “*incorrect*” OR “*faulty*” OR “*unsafe*” OR “*poor*” OR “*flawed*” OR “*suboptimal*” OR “*missing*” OR “*deprecated*” OR “*noncompliant*” OR “*malicious*” OR “*insecure interaction*” OR “*improper*” OR “*weak default*” OR “*incorrect handling*” OR “*unprotected value*” OR “*invalid*” OR “*unauth*” OR “*insufficient*” OR “*uninitialized*” OR “*outdated*” OR “*hardcoded*” OR “*exposed*” OR “*debug*” OR “*unrestricted*”) Requirements for using the data Python >= 3.9; recommended Python packages for analysis: pandas, numpy, scikit-learn, krippendorff, matplotlib No further requirements License for using the data Creative Commons Attribution 4.0 International The Creative Commons Attribution license allows re-distribution and re-use of a licensed work on the condition that the creator is appropriately credited. Further information: https://creativecommons.org/licenses/by/4.0/legalcode