VulnSage.zip

Automating software vulnerability detection (SVD) remainsa critical challenge in an era of increasingly complex and in-terdependent software systems. Despite significant advancesin Large Language Models (LLMs) for code analysis, prevail-ing evaluation methodologies often lack the context-awarerobustness necessary to capture real-world intricacies andcross-component interactions. To address these limitations,we present VulnSage, a comprehensive evaluation frameworkand a dataset curated from diverse, large-scale open-sourcesystem software projects developed in C/C++. Unlike priordatasets, VulnSage leverages a heuristic noise pre-filteringapproach combined with LLM-based reasoning to ensure arepresentative and minimally noisy spectrum of vulnerabili-ties. The framework can be used to rigorously assess LLMsby supporting a multi-granular analysis across function, file,and inter-function levels and employing four diverse zero-shot prompt strategies: Baseline, Chain-of-Thought, Think,and Think & Verify.