The Artifacts of OOPSLA-2023 Submission #197

This is the online repository of the OOPSLA-2023 Submission #197. We have released the source code and dataset. <strong>Dataset</strong>: Our evaluation is based on the large-scale CodeSearchNet dataset. Use the following command to download and preprocess the data: <pre><code class="language-bash">cd dataset bash run.sh cd ..</code></pre> <strong>Dependencies</strong> <pre><code class="language-bash">pip install -r requirements.txt</code></pre> [Optional] We have built the tree-sitter parser stored at `evaluator/CodeBLEU/parser/languages.so`. If it doesn't work for you, it can be rebuilt with the following command: <pre><code class="language-bash">cd evaluator/CodeBLEU/parser bash build.sh</code></pre> <strong>Training</strong> <pre><code class="language-bash">bash sh/train.sh [python/java] [CodeT5/Natgen] [CodeBERT/GraphCodeBERT]</code></pre> <strong>Evaluation</strong> Evaluate generator: <pre><code class="language-bash">bash sh/evaluate.sh [python/java] [CodeT5/Natgen]</code></pre> Evaluate discriminator: We evaluate the discriminator by reusing the code from CodeBERT and GraphCodeBERT. According to the Evaluate section in the corresponding model's Readme, replace `model_name_or_path` with the discriminator that you want to evaluate.