"CForge"

"C programming remains a foundational language inboth undergraduate computer science education and the defenseindustry. As Large Language Models (LLMs) are increasinglyadopted for AI-assisted programming, evaluating their reliabilityin these critical domains is essential. However, current benchmarksfor assessing LLM code generation capabilities predominantlyfocus on higher-level languages like Python and Java, leaving acritical void in evaluating C language proficiency. To address this,we introduce CForge, a comprehensive benchmark specificallydesigned to evaluate the C code generation capabilities of LLMs,encompassing over 6000 diverse programming tasks. Moreover,the manual memory management paradigm inherent in C\u2019ssyntax renders AI-generated code particularly vulnerable tosystem-level security flaws, most notably memory leaks andbuffer overflows. To tackle this, we propose Memory Safetyof C (MSC), a novel evaluation metric designed to rigorouslyassess the memory safety of the generated C code. Based on thevulnerability of current LLMs, we proposed a RL-based trainingstrategy to enhance its performance on our benchmark. Extensiveexperiments demonstrate that current mainstream LLMs stillhave significant room for improvement in both overall C codegeneration proficiency and strict memory safety."