DetectVul/devign

---
dataset_info:
features:
- name: id
dtype: int32
- name: func
dtype: string
- name: target
dtype: bool
- name: project
dtype: string
- name: commit_id
dtype: string
- name: func_clean
dtype: string
- name: vul_lines
struct:
- name: code
sequence: string
- name: line_no
sequence: int64
- name: normalized_func
dtype: string
- name: lines
sequence: string
- name: label
sequence: int64
- name: line_no
sequence:
sequence: int64
splits:
- name: test
num_bytes: 22801956
num_examples: 2732
- name: train
num_bytes: 183794878
num_examples: 21854
- name: validation
num_bytes: 22451009
num_examples: 2732
download_size: 72100845
dataset_size: 229047843
---
# Dataset Card for "devign_with_norm_vul_lines"

[More Information needed](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

Original Paper: https://www.sciencedirect.com/science/article/abs/pii/S0167739X24004680

bibtex

```
@article{TRAN2024107504,
title = {DetectVul: A statement-level code vulnerability detection for Python},
journal = {Future Generation Computer Systems},
pages = {107504},
year = {2024},
issn = {0167-739X},
doi = {https://doi.org/10.1016/j.future.2024.107504},
url = {https://www.sciencedirect.com/science/article/pii/S0167739X24004680},
author = {Hoai-Chau Tran and Anh-Duy Tran and Kim-Hung Le},
keywords = {Source code vulnerability detection, Deep learning, Natural language processing},
abstract = {Detecting vulnerabilities in source code using graph neural networks (GNN) has gained significant attention in recent years. However, the detection performance of these approaches relies highly on the graph structure, and constructing meaningful graphs is expensive. Moreover, they often operate at a coarse level of granularity (such as function-level), which limits their applicability to other scripting languages like Python and their effectiveness in identifying vulnerabilities. To address these limitations, we propose DetectVul, a new approach that accurately detects vulnerable patterns in Python source code at the statement level. DetectVul applies self-attention to directly learn patterns and interactions between statements in a raw Python function; thus, it eliminates the complicated graph extraction process without sacrificing model performance. In addition, the information about each type of statement is also leveraged to enhance the model’s detection accuracy. In our experiments, we used two datasets, CVEFixes and Vudenc, with 211,317 Python statements in 21,571 functions from real-world projects on GitHub, covering seven vulnerability types. Our experiments show that DetectVul outperforms GNN-based models using control flow graphs, achieving the best F1 score of 74.47%, which is 25.45% and 18.05% higher than the best GCN and GAT models, respectively.}
}
```

数据集信息：
特征字段：
- 字段名：id，数据类型：int32（32位整数）
- 字段名：func，数据类型：字符串
- 字段名：target，数据类型：布尔值
- 字段名：project，数据类型：字符串
- 字段名：commit_id，数据类型：字符串
- 字段名：func_clean，数据类型：字符串
- 字段名：vul_lines，结构体类型：
- 子字段：code，序列类型：字符串序列
- 子字段：line_no，序列类型：int64（64位整数）序列
- 字段名：normalized_func，数据类型：字符串
- 字段名：lines，序列类型：字符串序列
- 字段名：label，序列类型：int64（64位整数）序列
- 字段名：line_no，序列类型：嵌套int64（64位整数）序列
数据划分：
- 划分名称：测试集（test），字节数：22801956，样本数：2732
- 划分名称：训练集（train），字节数：183794878，样本数：21854
- 划分名称：验证集（validation），字节数：22451009，样本数：2732
下载大小：72100845字节
数据集总大小：229047843字节

---
# 数据集卡片："devign_with_norm_vul_lines"

[需补充更多信息](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

## 原始论文
论文链接：https://www.sciencedirect.com/science/article/abs/pii/S0167739X24004680

### BibTeX引用格式

@article{TRAN2024107504,
title = {DetectVul：面向Python的语句级代码漏洞检测},
journal = {Future Generation Computer Systems},
pages = {107504},
year = {2024},
issn = {0167-739X},
doi = {https://doi.org/10.1016/j.future.2024.107504},
url = {https://www.sciencedirect.com/science/article/pii/S0167739X24004680},
author = {Hoai-Chau Tran、Anh-Duy Tran、Kim-Hung Le},
keywords = {源代码漏洞检测、深度学习、自然语言处理},
abstract = {近年来，使用图神经网络（Graph Neural Network, GNN）检测源代码漏洞的研究受到广泛关注。然而，这类方法的检测性能高度依赖图结构，且构建有意义的图结构成本高昂。此外，它们通常以较粗的粒度（如函数级）进行检测，这限制了其在Python等脚本语言中的适用性，以及在识别漏洞时的有效性。为解决这些局限，我们提出DetectVul——一种可在语句级精准检测Python源代码中脆弱模式的新方法。DetectVul应用自注意力机制直接学习原始Python函数内语句间的模式与交互关系，因此无需复杂的图提取流程，同时未牺牲模型性能。此外，我们还利用各类语句的信息来提升模型的检测准确率。在实验中，我们使用了CVEFixes与Vudenc两个数据集，其包含来自GitHub真实项目的21571个函数中的211317条Python语句，覆盖7种漏洞类型。实验结果表明，DetectVul优于基于控制流图的图神经网络模型，取得了74.47%的最优F1分数，分别比最优的图卷积网络（Graph Convolutional Network, GCN）和图注意力网络（Graph Attention Network, GAT）模型高出25.45%与18.05%。}
}

搜集汇总

背景与挑战

背景概述

该数据集是DetectVul项目的一部分，专门用于Python源代码的漏洞检测，提供函数级和语句级的标注数据，包括易受攻击行信息和多种特征字段。数据集包含训练、验证和测试分割，适用于基于深度学习的漏洞检测模型开发，并支持语句级分析以提升检测精度。