SenseLLM/ReflectionSeq-DS

---
license: apache-2.0
task_categories:
- text-generation
language:
- en
tags:
- code
pretty_name: reflection_seq_for_code_generation
size_categories:
- 10K<n<100K
---

## ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generation

<p align="center">
<a href="https://arxiv.org/abs/2405.17057">📄 Paper</a> •
<a href="https://github.com/SenseLLM/ReflectionCoder">🏠 Repo</a> •
<a href="https://huggingface.co/SenseLLM/ReflectionCoder-DS-33B">🤖 Models</a> •
<a href="https://huggingface.co/datasets/SenseLLM/ReflectionSeq-GPT">📚 Datasets </a>
</p>

## Introduction
ReflectionCoder is a novel approach that effectively leverages reflection sequences constructed by integrating compiler feedback to improve one-off code generation performance. Please refer to our paper and repo for more details!


<hr>

## Models

| Model | Checkpoint | Size | HumanEval (+) | MBPP (+) | License|
|:-------|:------------|:------|:---------------|:----------|:--------|
| ReflectionCoder-CL-7B | 🤗 [HF Link](https://huggingface.co/SenseLM/ReflectionCoder-CL-7B) | 7B | 75.0 (68.9) | 72.2 (61.4) | [Llama2](https://ai.meta.com/llama/license/) |
| ReflectionCoder-CL-34B | 🤗 [HF Link](https://huggingface.co/SenseLM/ReflectionCoder-CL-34B) | 34B | 70.7 (66.5) | 68.4 (56.6) | [Llama2](https://ai.meta.com/llama/license/) |
| ReflectionCoder-DS-6.7B | 🤗 [HF Link](https://huggingface.co/SenseLM/ReflectionCoder-DS-6.7B) | 6.7B | 80.5 (74.4) | 81.5 (69.6) | [DeepSeek](https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/LICENSE-MODEL) |
| ReflectionCoder-DS-33B | 🤗 [HF Link](https://huggingface.co/SenseLM/ReflectionCoder-DS-33B) | 33B | 82.9 (76.8) | 84.1 (72.0) | [DeepSeek](https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/LICENSE-MODEL) |

## Datasets

| Dataset | Link | License |
|:-------------------|:----------------|:----------------------------------------------|
| ReflectionSeq-GPT | 🤗 [HF Link](https://huggingface.co/datasets/SenseLM/ReflectionSeq-GPT) | [License](LICENSE) |
| ReflectionSeq-DS | 🤗 [HF Link](https://huggingface.co/datasets/SenseLM/ReflectionSeq-DS) | [License](LICENSE) |

## How to Use

#### Chat Format
Following chat templates of most models, we use two special tokens to wrap the message of user and assistant, *i.e.*, ``<|user|>``, ``<|assistant|>``, and ``<|endofmessage|>``. Furthermore, we use two special tokens to wrap the content of different blocks, *i.e.*, ``<|text|>`` and ``<|endofblock|>``. You can use the following template to prompt our ReflectionCoder.

```python
<|user|><|text|>
Your Instruction
<|endofblock|><|endofmessage|><|assistant|>
```

#### Inference Code
Please refer to our GitHub Repo [need update](https://ai.meta.com/llama/license/) for more technical details.

## Citation

If you find this repo useful for your research, please kindly cite our paper:
```
@misc{ren2024reflectioncoder,
title={ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generation},
author={Houxing Ren and Mingjie Zhan and Zhongyuan Wu and Aojun Zhou and Junting Pan and Hongsheng Li},
year={2024},
eprint={2405.17057},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```

## Acknowledgments

We thank the following amazing projects that truly inspired us:

- [CodeLlama](https://ai.meta.com/research/publications/code-llama-open-foundation-models-for-code/)
- [DeepSeek-Coder](https://github.com/deepseek-ai/DeepSeek-Coder)
- [WizardCoder](https://github.com/nlpxucan/WizardLM/tree/main/WizardCoder)
- [Evol-CodeAlpaca-v1](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1)
- [MagiCoder](https://github.com/ise-uiuc/magicoder/tree/main)
- [EvalPlus](https://github.com/evalplus/evalplus)
- [OpenCoderInterpreter](https://github.com/OpenCodeInterpreter/OpenCodeInterpreter/tree/main)

原始信息汇总

数据集概述

基本信息

• 许可证: Apache-2.0
• 任务类别: 文本生成
• 语言: 英语
• 标签: 代码
• 美观名称: reflection_seq_for_code_generation
• 大小类别: 10K<n<100K

数据集详情

• 名称: ReflectionSeq-GPT

• 链接: Hugging Face Link

• 许可证: 详见LICENSE

• 名称: ReflectionSeq-DS

• 链接: Hugging Face Link

• 许可证: 详见LICENSE

使用方法

• 聊天格式: 使用特殊标记如 <|user|>, <|assistant|>, <|text|>, 和 <|endofblock|> 来组织用户和助手的消息内容。

python <|user|><|text|> Your Instruction <|endofblock|><|endofmessage|><|assistant|>

引用信息

@misc{ren2024reflectioncoder, title={ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generation}, author={Houxing Ren and Mingjie Zhan and Zhongyuan Wu and Aojun Zhou and Junting Pan and Hongsheng Li}, year={2024}, eprint={2405.17057}, archivePrefix={arXiv}, primaryClass={cs.CL} }