Open source projects for software vulnerabilities

Here are some real-world data sets for software vulnerabilities.The original real-world data sets, collected by Lin et al. (https://github.com/DanielLin1986/TransferRepresentationLearning), which contain the source codes of vulnerable and non-vulnerable functions obtained from six real-world software projects, namely FFmpeg, LibTIFF, LibPNG, VLC and Pidgin. These datasets cover both multimedia and image application categories.To obtain our used data sets, we preprocess these data sets before inputting into the deep neural networks. Firstly, we standardize the source codes by removing comments, blank lines and non-ASCII characters. Secondly, we map user-defined variables to symbolic names (e.g., “var1”, “var2”) and user-defined functions to symbolic names (e.g., “func1”, “func2”). We also replace integers, real and hexadecimal numbers with a generic "num" token and strings with a generic "str" token. We usehttps://joern.readthedocs.io/en/latest/to analyze the source codes to get user-defined variables and functions.

以下为若干用于软件漏洞分析的真实世界数据集。此类原始真实世界数据集由Lin等人（https://github.com/DanielLin1986/TransferRepresentationLearning）收集，包含从FFmpeg、LibTIFF、LibPNG、VLC及Pidgin这6个真实软件项目中提取的易受攻击函数与无漏洞函数的源代码。此类数据集覆盖多媒体与图像应用两大类别。
为获取本研究使用的数据集，我们在将其输入深度神经网络前进行了预处理操作。首先，我们对源代码开展标准化处理：移除注释、空行与非ASCII字符。其次，将用户自定义变量映射为统一符号化名称（例如"var1", "var2"），并将用户自定义函数映射为统一符号化名称（例如"func1", "func2"）。此外，将整数、实数与十六进制数替换为通用`num` Token（token），将字符串替换为通用`str` Token（token）。我们使用https://joern.readthedocs.io/en/latest/ 提供的工具对源代码进行分析，以提取用户自定义变量与函数。