The FormAI Dataset : 112,000 AI generated C Programs Classified by Vulnerabilities

The FormAI dataset is the first AI-generated repository comprising 112,000 compilable C programs. The length of each generated program varies between 6 and 616 lines, covering a diverse set of tasks ranging from simple to complex ones, such as network management, table games, or string manipulation. Each program is labelled based on vulnerabilities present in the code using a formal verification method based on the Efficient SMT-based Bounded Model Checker (ESBMC). This strategy conclusively identifies vulnerabilities without reporting false positives. We built this dataset with a methodology utilizing zero-shot prompts, and created a diverse datasets facilitating a wide range of coding errors. All the programs in the dataset were generated by GPT-3.5-turbo. Additionally, these labeled samples can be utilized to train Large Language Models (LLMs) since they contain the exact program location of the software vulnerability.