five

zilu-peter-tang/MultiPL-C2C

收藏
Hugging Face2023-11-13 更新2024-03-04 收录
下载链接：
https://hf-mirror.com/datasets/zilu-peter-tang/MultiPL-C2C
下载链接
链接失效反馈
官方服务：
资源简介：
--- license: mit dataset_info: - config_name: humaneval-cpp-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 323975 num_examples: 148 download_size: 370379 dataset_size: 323975 - config_name: humaneval-cpp-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 369855 num_examples: 148 download_size: 417147 dataset_size: 369855 - config_name: humaneval-cpp-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 228556 num_examples: 146 download_size: 274393 dataset_size: 228556 - config_name: humaneval-cpp-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 273670 num_examples: 146 download_size: 320383 dataset_size: 273670 - config_name: humaneval-cpp-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 266725 num_examples: 145 download_size: 318988 dataset_size: 266725 - config_name: humaneval-cpp-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 311675 num_examples: 145 download_size: 364808 dataset_size: 311675 - config_name: humaneval-cpp-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 301674 num_examples: 148 download_size: 346400 dataset_size: 301674 - config_name: humaneval-cpp-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 347850 num_examples: 148 download_size: 393464 dataset_size: 347850 - config_name: humaneval-cpp-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 180272 num_examples: 148 download_size: 223520 dataset_size: 180272 - config_name: humaneval-cpp-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 226596 num_examples: 148 download_size: 270732 dataset_size: 226596 - config_name: humaneval-cpp-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 194639 num_examples: 149 download_size: 239420 dataset_size: 194639 - config_name: humaneval-cpp-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 242021 num_examples: 149 download_size: 287696 dataset_size: 242021 - config_name: humaneval-cpp-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202873 num_examples: 149 download_size: 244832 dataset_size: 202873 - config_name: humaneval-cpp-lua-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 208982 num_examples: 149 download_size: 251239 dataset_size: 208982 - config_name: humaneval-cpp-lua-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 208982 num_examples: 149 download_size: 251239 dataset_size: 208982 - config_name: humaneval-cpp-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 249212 num_examples: 149 download_size: 292065 dataset_size: 249212 - config_name: humaneval-cpp-lua-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 236398 num_examples: 149 download_size: 279251 dataset_size: 236398 - config_name: humaneval-cpp-lua-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 299368 num_examples: 149 download_size: 342412 dataset_size: 299368 - config_name: humaneval-cpp-lua-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260089 num_examples: 149 download_size: 302942 dataset_size: 260089 - config_name: humaneval-cpp-lua-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 498101 num_examples: 149 download_size: 543921 dataset_size: 498101 - config_name: humaneval-cpp-lua-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 238484 num_examples: 149 download_size: 281337 dataset_size: 238484 - config_name: humaneval-cpp-lua-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 402215 num_examples: 149 download_size: 447326 dataset_size: 402215 - config_name: humaneval-cpp-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 233624 num_examples: 149 download_size: 280652 dataset_size: 233624 - config_name: humaneval-cpp-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 279963 num_examples: 149 download_size: 327885 dataset_size: 279963 - config_name: humaneval-cpp-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 297359 num_examples: 149 download_size: 349118 dataset_size: 297359 - config_name: humaneval-cpp-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 343847 num_examples: 149 download_size: 396500 dataset_size: 343847 - config_name: humaneval-cpp-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 190944 num_examples: 149 download_size: 233686 dataset_size: 190944 - config_name: humaneval-cpp-py-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 197947 num_examples: 149 download_size: 240987 dataset_size: 197947 - config_name: humaneval-cpp-py-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 197947 num_examples: 149 download_size: 240987 dataset_size: 197947 - config_name: humaneval-cpp-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 237730 num_examples: 149 download_size: 281366 dataset_size: 237730 - config_name: humaneval-cpp-py-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 224916 num_examples: 149 download_size: 268552 dataset_size: 224916 - config_name: humaneval-cpp-py-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 301055 num_examples: 149 download_size: 344987 dataset_size: 301055 - config_name: humaneval-cpp-py-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 248607 num_examples: 149 download_size: 292243 dataset_size: 248607 - config_name: humaneval-cpp-py-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 454469 num_examples: 149 download_size: 500376 dataset_size: 454469 - config_name: humaneval-cpp-py-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 227002 num_examples: 149 download_size: 270638 dataset_size: 227002 - config_name: humaneval-cpp-py-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 378207 num_examples: 149 download_size: 423441 dataset_size: 378207 - config_name: humaneval-cpp-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 254398 num_examples: 149 download_size: 295016 dataset_size: 254398 - config_name: humaneval-cpp-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 300439 num_examples: 149 download_size: 341951 dataset_size: 300439 - config_name: humaneval-cpp-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199246 num_examples: 149 download_size: 243580 dataset_size: 199246 - config_name: humaneval-cpp-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 245734 num_examples: 149 download_size: 290962 dataset_size: 245734 - config_name: humaneval-cpp-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202983 num_examples: 149 download_size: 248062 dataset_size: 202983 - config_name: humaneval-cpp-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 249769 num_examples: 149 download_size: 295742 dataset_size: 249769 - config_name: humaneval-cpp-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 195653 num_examples: 146 download_size: 235797 dataset_size: 195653 - config_name: humaneval-cpp-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 241205 num_examples: 146 download_size: 282225 dataset_size: 241205 - config_name: humaneval-cpp-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 245856 num_examples: 149 download_size: 287955 dataset_size: 245856 - config_name: humaneval-cpp-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 292493 num_examples: 149 download_size: 335486 dataset_size: 292493 - config_name: humaneval-cpp-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 185420 num_examples: 146 download_size: 232965 dataset_size: 185420 - config_name: humaneval-cpp-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 230972 num_examples: 146 download_size: 279393 dataset_size: 230972 - config_name: humaneval-cpp-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 226396 num_examples: 149 download_size: 269393 dataset_size: 226396 - config_name: humaneval-cpp-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 273033 num_examples: 149 download_size: 316924 dataset_size: 273033 - config_name: humaneval-cpp-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202239 num_examples: 148 download_size: 246811 dataset_size: 202239 - config_name: humaneval-cpp-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 249303 num_examples: 148 download_size: 294763 dataset_size: 249303 - config_name: humaneval-cs-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 280216 num_examples: 144 download_size: 322436 dataset_size: 280216 - config_name: humaneval-cs-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 324856 num_examples: 144 download_size: 367940 dataset_size: 324856 - config_name: humaneval-cs-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 235080 num_examples: 142 download_size: 279666 dataset_size: 235080 - config_name: humaneval-cs-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 278816 num_examples: 142 download_size: 324254 dataset_size: 278816 - config_name: humaneval-cs-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 275866 num_examples: 142 download_size: 327192 dataset_size: 275866 - config_name: humaneval-cs-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 319744 num_examples: 142 download_size: 371922 dataset_size: 319744 - config_name: humaneval-cs-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 306842 num_examples: 144 download_size: 350502 dataset_size: 306842 - config_name: humaneval-cs-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 351626 num_examples: 144 download_size: 396150 dataset_size: 351626 - config_name: humaneval-cs-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 188340 num_examples: 143 download_size: 230208 dataset_size: 188340 - config_name: humaneval-cs-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 232956 num_examples: 143 download_size: 275682 dataset_size: 232956 - config_name: humaneval-cs-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202496 num_examples: 144 download_size: 245868 dataset_size: 202496 - config_name: humaneval-cs-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 248144 num_examples: 144 download_size: 292380 dataset_size: 248144 - config_name: humaneval-cs-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 210442 num_examples: 144 download_size: 251244 dataset_size: 210442 - config_name: humaneval-cs-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 255082 num_examples: 144 download_size: 296748 dataset_size: 255082 - config_name: humaneval-cs-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 239806 num_examples: 144 download_size: 285336 dataset_size: 239806 - config_name: humaneval-cs-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 284446 num_examples: 144 download_size: 330840 dataset_size: 284446 - config_name: humaneval-cs-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 300693 num_examples: 144 download_size: 350600 dataset_size: 300693 - config_name: humaneval-cs-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 345477 num_examples: 144 download_size: 396248 dataset_size: 345477 - config_name: humaneval-cs-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 198578 num_examples: 144 download_size: 240120 dataset_size: 198578 - config_name: humaneval-cs-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 243650 num_examples: 144 download_size: 286056 dataset_size: 243650 - config_name: humaneval-cs-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260114 num_examples: 144 download_size: 299620 dataset_size: 260114 - config_name: humaneval-cs-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 304466 num_examples: 144 download_size: 344836 dataset_size: 304466 - config_name: humaneval-cs-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 206956 num_examples: 144 download_size: 249896 dataset_size: 206956 - config_name: humaneval-cs-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 251740 num_examples: 144 download_size: 295544 dataset_size: 251740 - config_name: humaneval-cs-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 210439 num_examples: 144 download_size: 254099 dataset_size: 210439 - config_name: humaneval-cs-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 255511 num_examples: 144 download_size: 300035 dataset_size: 255511 - config_name: humaneval-cs-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 204182 num_examples: 142 download_size: 243362 dataset_size: 204182 - config_name: humaneval-cs-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 248344 num_examples: 142 download_size: 288376 dataset_size: 248344 - config_name: humaneval-cs-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 251450 num_examples: 144 download_size: 292230 dataset_size: 251450 - config_name: humaneval-cs-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 296378 num_examples: 144 download_size: 338022 dataset_size: 296378 - config_name: humaneval-cs-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 193763 num_examples: 141 download_size: 239923 dataset_size: 193763 - config_name: humaneval-cs-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 237614 num_examples: 141 download_size: 284620 dataset_size: 237614 - config_name: humaneval-cs-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 233036 num_examples: 144 download_size: 274680 dataset_size: 233036 - config_name: humaneval-cs-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 277964 num_examples: 144 download_size: 320472 dataset_size: 277964 - config_name: humaneval-cs-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 209691 num_examples: 143 download_size: 252845 dataset_size: 209691 - config_name: humaneval-cs-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 255022 num_examples: 143 download_size: 299034 dataset_size: 255022 - config_name: humaneval-d-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202851 num_examples: 116 download_size: 236152 dataset_size: 202851 - config_name: humaneval-d-cpp-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 207143 num_examples: 116 download_size: 240676 dataset_size: 207143 - config_name: humaneval-d-cpp-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 207143 num_examples: 116 download_size: 240676 dataset_size: 207143 - config_name: humaneval-d-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 238695 num_examples: 116 download_size: 272692 dataset_size: 238695 - config_name: humaneval-d-cpp-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 228719 num_examples: 116 download_size: 262716 dataset_size: 228719 - config_name: humaneval-d-cpp-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 305945 num_examples: 116 download_size: 340220 dataset_size: 305945 - config_name: humaneval-d-cpp-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 247163 num_examples: 116 download_size: 281160 dataset_size: 247163 - config_name: humaneval-d-cpp-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 370515 num_examples: 116 download_size: 405883 dataset_size: 370515 - config_name: humaneval-d-cpp-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 230343 num_examples: 116 download_size: 264340 dataset_size: 230343 - config_name: humaneval-d-cpp-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 333203 num_examples: 116 download_size: 368158 dataset_size: 333203 - config_name: humaneval-d-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 242100 num_examples: 116 download_size: 277887 dataset_size: 242100 - config_name: humaneval-d-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 277828 num_examples: 116 download_size: 314311 dataset_size: 277828 - config_name: humaneval-d-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199774 num_examples: 116 download_size: 240963 dataset_size: 199774 - config_name: humaneval-d-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 235502 num_examples: 116 download_size: 277387 dataset_size: 235502 - config_name: humaneval-d-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 224515 num_examples: 116 download_size: 258976 dataset_size: 224515 - config_name: humaneval-d-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260475 num_examples: 116 download_size: 295632 dataset_size: 260475 - config_name: humaneval-d-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 128375 num_examples: 116 download_size: 161649 dataset_size: 128375 - config_name: humaneval-d-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 164451 num_examples: 116 download_size: 198421 dataset_size: 164451 - config_name: humaneval-d-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 137957 num_examples: 116 download_size: 172186 dataset_size: 137957 - config_name: humaneval-d-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 174613 num_examples: 116 download_size: 209538 dataset_size: 174613 - config_name: humaneval-d-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 144363 num_examples: 116 download_size: 176836 dataset_size: 144363 - config_name: humaneval-d-lua-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 148655 num_examples: 116 download_size: 181360 dataset_size: 148655 - config_name: humaneval-d-lua-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 148655 num_examples: 116 download_size: 181360 dataset_size: 148655 - config_name: humaneval-d-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 180207 num_examples: 116 download_size: 213376 dataset_size: 180207 - config_name: humaneval-d-lua-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 170231 num_examples: 116 download_size: 203400 dataset_size: 170231 - config_name: humaneval-d-lua-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 231707 num_examples: 116 download_size: 265135 dataset_size: 231707 - config_name: humaneval-d-lua-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 188675 num_examples: 116 download_size: 221844 dataset_size: 188675 - config_name: humaneval-d-lua-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 321378 num_examples: 116 download_size: 355997 dataset_size: 321378 - config_name: humaneval-d-lua-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 171855 num_examples: 116 download_size: 205024 dataset_size: 171855 - config_name: humaneval-d-lua-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 264560 num_examples: 116 download_size: 298586 dataset_size: 264560 - config_name: humaneval-d-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 167393 num_examples: 116 download_size: 203308 dataset_size: 167393 - config_name: humaneval-d-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 203237 num_examples: 116 download_size: 239848 dataset_size: 203237 - config_name: humaneval-d-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 213993 num_examples: 116 download_size: 253057 dataset_size: 213993 - config_name: humaneval-d-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 249953 num_examples: 116 download_size: 289713 dataset_size: 249953 - config_name: humaneval-d-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 135146 num_examples: 116 download_size: 168227 dataset_size: 135146 - config_name: humaneval-d-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 171338 num_examples: 116 download_size: 205115 dataset_size: 171338 - config_name: humaneval-d-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 184062 num_examples: 116 download_size: 215491 dataset_size: 184062 - config_name: humaneval-d-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219674 num_examples: 116 download_size: 251799 dataset_size: 219674 - config_name: humaneval-d-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 141465 num_examples: 116 download_size: 175346 dataset_size: 141465 - config_name: humaneval-d-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 177425 num_examples: 116 download_size: 212002 dataset_size: 177425 - config_name: humaneval-d-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 144308 num_examples: 116 download_size: 178769 dataset_size: 144308 - config_name: humaneval-d-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 180500 num_examples: 116 download_size: 215657 dataset_size: 180500 - config_name: humaneval-d-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 140123 num_examples: 116 download_size: 171456 dataset_size: 140123 - config_name: humaneval-d-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 176083 num_examples: 116 download_size: 208112 dataset_size: 176083 - config_name: humaneval-d-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 178734 num_examples: 116 download_size: 210875 dataset_size: 178734 - config_name: humaneval-d-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 214810 num_examples: 116 download_size: 247647 dataset_size: 214810 - config_name: humaneval-d-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 130978 num_examples: 113 download_size: 167504 dataset_size: 130978 - config_name: humaneval-d-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 166008 num_examples: 113 download_size: 203212 dataset_size: 166008 - config_name: humaneval-d-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 162040 num_examples: 116 download_size: 194877 dataset_size: 162040 - config_name: humaneval-d-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 198116 num_examples: 116 download_size: 231649 dataset_size: 198116 - config_name: humaneval-d-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 145375 num_examples: 116 download_size: 179720 dataset_size: 145375 - config_name: humaneval-d-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 182031 num_examples: 116 download_size: 217072 dataset_size: 182031 - config_name: humaneval-go-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 178928 num_examples: 103 download_size: 210043 dataset_size: 178928 - config_name: humaneval-go-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 210858 num_examples: 103 download_size: 242591 dataset_size: 210858 - config_name: humaneval-go-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 213396 num_examples: 103 download_size: 246674 dataset_size: 213396 - config_name: humaneval-go-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 245223 num_examples: 103 download_size: 279119 dataset_size: 245223 - config_name: humaneval-go-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 148397 num_examples: 102 download_size: 181420 dataset_size: 148397 - config_name: humaneval-go-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 179813 num_examples: 102 download_size: 213448 dataset_size: 179813 - config_name: humaneval-go-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 198489 num_examples: 103 download_size: 230634 dataset_size: 198489 - config_name: humaneval-go-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 230522 num_examples: 103 download_size: 263285 dataset_size: 230522 - config_name: humaneval-go-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 113813 num_examples: 103 download_size: 144945 dataset_size: 113813 - config_name: humaneval-go-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 145949 num_examples: 103 download_size: 177699 dataset_size: 145949 - config_name: humaneval-go-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 122928 num_examples: 103 download_size: 154867 dataset_size: 122928 - config_name: humaneval-go-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 155579 num_examples: 103 download_size: 188136 dataset_size: 155579 - config_name: humaneval-go-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 128570 num_examples: 103 download_size: 158893 dataset_size: 128570 - config_name: humaneval-go-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 160500 num_examples: 103 download_size: 191441 dataset_size: 160500 - config_name: humaneval-go-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 150094 num_examples: 103 download_size: 183608 dataset_size: 150094 - config_name: humaneval-go-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 182024 num_examples: 103 download_size: 216156 dataset_size: 182024 - config_name: humaneval-go-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 193092 num_examples: 103 download_size: 229652 dataset_size: 193092 - config_name: humaneval-go-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 225125 num_examples: 103 download_size: 262303 dataset_size: 225125 - config_name: humaneval-go-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 119798 num_examples: 103 download_size: 150642 dataset_size: 119798 - config_name: humaneval-go-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 152037 num_examples: 103 download_size: 183499 dataset_size: 152037 - config_name: humaneval-go-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 164308 num_examples: 103 download_size: 193704 dataset_size: 164308 - config_name: humaneval-go-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 196032 num_examples: 103 download_size: 226046 dataset_size: 196032 - config_name: humaneval-go-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 126085 num_examples: 103 download_size: 157715 dataset_size: 126085 - config_name: humaneval-go-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 158118 num_examples: 103 download_size: 190366 dataset_size: 158118 - config_name: humaneval-go-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 128420 num_examples: 103 download_size: 160565 dataset_size: 128420 - config_name: humaneval-go-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 160659 num_examples: 103 download_size: 193422 dataset_size: 160659 - config_name: humaneval-go-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 123814 num_examples: 102 download_size: 152912 dataset_size: 123814 - config_name: humaneval-go-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 155536 num_examples: 102 download_size: 185246 dataset_size: 155536 - config_name: humaneval-go-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 159010 num_examples: 103 download_size: 189095 dataset_size: 159010 - config_name: humaneval-go-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 191146 num_examples: 103 download_size: 221849 dataset_size: 191146 - config_name: humaneval-go-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 118403 num_examples: 102 download_size: 152901 dataset_size: 118403 - config_name: humaneval-go-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 150125 num_examples: 102 download_size: 185235 dataset_size: 150125 - config_name: humaneval-go-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 144059 num_examples: 103 download_size: 174762 dataset_size: 144059 - config_name: humaneval-go-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 176195 num_examples: 103 download_size: 207516 dataset_size: 176195 - config_name: humaneval-go-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 129434 num_examples: 103 download_size: 161476 dataset_size: 129434 - config_name: humaneval-go-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 162085 num_examples: 103 download_size: 194745 dataset_size: 162085 - config_name: humaneval-java-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 301489 num_examples: 155 download_size: 346974 dataset_size: 301489 - config_name: humaneval-java-cpp-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 308154 num_examples: 155 download_size: 353949 dataset_size: 308154 - config_name: humaneval-java-cpp-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 308154 num_examples: 155 download_size: 353949 dataset_size: 308154 - config_name: humaneval-java-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 349849 num_examples: 155 download_size: 396264 dataset_size: 349849 - config_name: humaneval-java-cpp-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 336519 num_examples: 155 download_size: 382934 dataset_size: 336519 - config_name: humaneval-java-cpp-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 423727 num_examples: 155 download_size: 470497 dataset_size: 423727 - config_name: humaneval-java-cpp-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 361164 num_examples: 155 download_size: 407579 dataset_size: 361164 - config_name: humaneval-java-cpp-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 645005 num_examples: 155 download_size: 694525 dataset_size: 645005 - config_name: humaneval-java-cpp-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 338689 num_examples: 155 download_size: 385104 dataset_size: 338689 - config_name: humaneval-java-cpp-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 560777 num_examples: 155 download_size: 609918 dataset_size: 560777 - config_name: humaneval-java-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 353495 num_examples: 155 download_size: 402285 dataset_size: 353495 - config_name: humaneval-java-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 401700 num_examples: 155 download_size: 451420 dataset_size: 401700 - config_name: humaneval-java-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 252187 num_examples: 153 download_size: 300329 dataset_size: 252187 - config_name: humaneval-java-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 299617 num_examples: 153 download_size: 348677 dataset_size: 299617 - config_name: humaneval-java-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 291886 num_examples: 151 download_size: 346496 dataset_size: 291886 - config_name: humaneval-java-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 338847 num_examples: 151 download_size: 394363 dataset_size: 338847 - config_name: humaneval-java-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 201235 num_examples: 154 download_size: 246364 dataset_size: 201235 - config_name: humaneval-java-jl-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 208473 num_examples: 154 download_size: 253910 dataset_size: 208473 - config_name: humaneval-java-jl-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 208473 num_examples: 154 download_size: 253910 dataset_size: 208473 - config_name: humaneval-java-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 249591 num_examples: 154 download_size: 295644 dataset_size: 249591 - config_name: humaneval-java-jl-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 236347 num_examples: 154 download_size: 282400 dataset_size: 236347 - config_name: humaneval-java-jl-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 320453 num_examples: 154 download_size: 366967 dataset_size: 320453 - config_name: humaneval-java-jl-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260833 num_examples: 154 download_size: 306886 dataset_size: 260833 - config_name: humaneval-java-jl-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 560386 num_examples: 154 download_size: 609897 dataset_size: 560386 - config_name: humaneval-java-jl-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 238503 num_examples: 154 download_size: 284556 dataset_size: 238503 - config_name: humaneval-java-jl-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 481222 num_examples: 154 download_size: 531362 dataset_size: 481222 - config_name: humaneval-java-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 216015 num_examples: 155 download_size: 262740 dataset_size: 216015 - config_name: humaneval-java-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 265460 num_examples: 155 download_size: 313115 dataset_size: 265460 - config_name: humaneval-java-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 224601 num_examples: 155 download_size: 268408 dataset_size: 224601 - config_name: humaneval-java-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 272961 num_examples: 155 download_size: 317698 dataset_size: 272961 - config_name: humaneval-java-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 256108 num_examples: 155 download_size: 305144 dataset_size: 256108 - config_name: humaneval-java-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 304468 num_examples: 155 download_size: 354434 dataset_size: 304468 - config_name: humaneval-java-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 321795 num_examples: 155 download_size: 375673 dataset_size: 321795 - config_name: humaneval-java-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 370310 num_examples: 155 download_size: 425118 dataset_size: 370310 - config_name: humaneval-java-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212138 num_examples: 155 download_size: 256758 dataset_size: 212138 - config_name: humaneval-java-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260963 num_examples: 155 download_size: 306513 dataset_size: 260963 - config_name: humaneval-java-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 277995 num_examples: 155 download_size: 320407 dataset_size: 277995 - config_name: humaneval-java-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 326045 num_examples: 155 download_size: 369387 dataset_size: 326045 - config_name: humaneval-java-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 220814 num_examples: 155 download_size: 267074 dataset_size: 220814 - config_name: humaneval-java-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 269329 num_examples: 155 download_size: 316519 dataset_size: 269329 - config_name: humaneval-java-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 224654 num_examples: 155 download_size: 271689 dataset_size: 224654 - config_name: humaneval-java-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 273479 num_examples: 155 download_size: 321444 dataset_size: 273479 - config_name: humaneval-java-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219587 num_examples: 153 download_size: 261853 dataset_size: 219587 - config_name: humaneval-java-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 267476 num_examples: 153 download_size: 310660 dataset_size: 267476 - config_name: humaneval-java-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 269404 num_examples: 155 download_size: 313339 dataset_size: 269404 - config_name: humaneval-java-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 318074 num_examples: 155 download_size: 362939 dataset_size: 318074 - config_name: humaneval-java-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 206506 num_examples: 152 download_size: 256071 dataset_size: 206506 - config_name: humaneval-java-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 254082 num_examples: 152 download_size: 304559 dataset_size: 254082 - config_name: humaneval-java-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 248850 num_examples: 155 download_size: 293715 dataset_size: 248850 - config_name: humaneval-java-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 297520 num_examples: 155 download_size: 343315 dataset_size: 297520 - config_name: humaneval-java-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 223991 num_examples: 154 download_size: 270511 dataset_size: 223991 - config_name: humaneval-java-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 273117 num_examples: 154 download_size: 320561 dataset_size: 273117 - config_name: humaneval-jl-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 225765 num_examples: 138 download_size: 264732 dataset_size: 225765 - config_name: humaneval-jl-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 268959 num_examples: 138 download_size: 308754 dataset_size: 268959 - config_name: humaneval-jl-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 272014 num_examples: 138 download_size: 313929 dataset_size: 272014 - config_name: humaneval-jl-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 315070 num_examples: 138 download_size: 357813 dataset_size: 315070 - config_name: humaneval-jl-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 186573 num_examples: 137 download_size: 228304 dataset_size: 186573 - config_name: humaneval-jl-d-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 192190 num_examples: 137 download_size: 234195 dataset_size: 192190 - config_name: humaneval-jl-d-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 192190 num_examples: 137 download_size: 234195 dataset_size: 192190 - config_name: humaneval-jl-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 229180 num_examples: 137 download_size: 271733 dataset_size: 229180 - config_name: humaneval-jl-d-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 217398 num_examples: 137 download_size: 259951 dataset_size: 217398 - config_name: humaneval-jl-d-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 277543 num_examples: 137 download_size: 320346 dataset_size: 277543 - config_name: humaneval-jl-d-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 239181 num_examples: 137 download_size: 281734 dataset_size: 239181 - config_name: humaneval-jl-d-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 357853 num_examples: 137 download_size: 401646 dataset_size: 357853 - config_name: humaneval-jl-d-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219316 num_examples: 137 download_size: 261869 dataset_size: 219316 - config_name: humaneval-jl-d-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 284354 num_examples: 137 download_size: 327523 dataset_size: 284354 - config_name: humaneval-jl-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219929 num_examples: 135 download_size: 267260 dataset_size: 219929 - config_name: humaneval-jl-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 262049 num_examples: 135 download_size: 310190 dataset_size: 262049 - config_name: humaneval-jl-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 251261 num_examples: 138 download_size: 291608 dataset_size: 251261 - config_name: humaneval-jl-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 294593 num_examples: 138 download_size: 335768 dataset_size: 294593 - config_name: humaneval-jl-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 150698 num_examples: 138 download_size: 190769 dataset_size: 150698 - config_name: humaneval-jl-js-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 158840 num_examples: 138 download_size: 199187 dataset_size: 158840 - config_name: humaneval-jl-js-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 158840 num_examples: 138 download_size: 199187 dataset_size: 158840 - config_name: humaneval-jl-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 194858 num_examples: 138 download_size: 235757 dataset_size: 194858 - config_name: humaneval-jl-js-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 182990 num_examples: 138 download_size: 223889 dataset_size: 182990 - config_name: humaneval-jl-js-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 234038 num_examples: 138 download_size: 275161 dataset_size: 234038 - config_name: humaneval-jl-js-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 204932 num_examples: 138 download_size: 245831 dataset_size: 204932 - config_name: humaneval-jl-js-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 324346 num_examples: 138 download_size: 366628 dataset_size: 324346 - config_name: humaneval-jl-js-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 184922 num_examples: 138 download_size: 225821 dataset_size: 184922 - config_name: humaneval-jl-js-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 247327 num_examples: 138 download_size: 288804 dataset_size: 247327 - config_name: humaneval-jl-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 158269 num_examples: 138 download_size: 195854 dataset_size: 158269 - config_name: humaneval-jl-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 201463 num_examples: 138 download_size: 239876 dataset_size: 201463 - config_name: humaneval-jl-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 186290 num_examples: 138 download_size: 228421 dataset_size: 186290 - config_name: humaneval-jl-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 229484 num_examples: 138 download_size: 272443 dataset_size: 229484 - config_name: humaneval-jl-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 244111 num_examples: 138 download_size: 290487 dataset_size: 244111 - config_name: humaneval-jl-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 287443 num_examples: 138 download_size: 334647 dataset_size: 287443 - config_name: humaneval-jl-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 147244 num_examples: 138 download_size: 185551 dataset_size: 147244 - config_name: humaneval-jl-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 190852 num_examples: 138 download_size: 229987 dataset_size: 190852 - config_name: humaneval-jl-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 205816 num_examples: 138 download_size: 242159 dataset_size: 205816 - config_name: humaneval-jl-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 248734 num_examples: 138 download_size: 285905 dataset_size: 248734 - config_name: humaneval-jl-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 154919 num_examples: 138 download_size: 194576 dataset_size: 154919 - config_name: humaneval-jl-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 198251 num_examples: 138 download_size: 238736 dataset_size: 198251 - config_name: humaneval-jl-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 158215 num_examples: 138 download_size: 198562 dataset_size: 158215 - config_name: humaneval-jl-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 201823 num_examples: 138 download_size: 242998 dataset_size: 201823 - config_name: humaneval-jl-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 155083 num_examples: 137 download_size: 191459 dataset_size: 155083 - config_name: humaneval-jl-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 198101 num_examples: 137 download_size: 235299 dataset_size: 198101 - config_name: humaneval-jl-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 197634 num_examples: 138 download_size: 235221 dataset_size: 197634 - config_name: humaneval-jl-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 241104 num_examples: 138 download_size: 279519 dataset_size: 241104 - config_name: humaneval-jl-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 142849 num_examples: 135 download_size: 185491 dataset_size: 142849 - config_name: humaneval-jl-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 185239 num_examples: 135 download_size: 228691 dataset_size: 185239 - config_name: humaneval-jl-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 180002 num_examples: 138 download_size: 218417 dataset_size: 180002 - config_name: humaneval-jl-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 223472 num_examples: 138 download_size: 262715 dataset_size: 223472 - config_name: humaneval-jl-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 159584 num_examples: 138 download_size: 199793 dataset_size: 159584 - config_name: humaneval-jl-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 203744 num_examples: 138 download_size: 244781 dataset_size: 203744 - config_name: humaneval-js-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 255195 num_examples: 154 download_size: 298838 dataset_size: 255195 - config_name: humaneval-js-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 304167 num_examples: 154 download_size: 348734 dataset_size: 304167 - config_name: humaneval-js-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 300003 num_examples: 152 download_size: 346202 dataset_size: 300003 - config_name: humaneval-js-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 348187 num_examples: 152 download_size: 395298 dataset_size: 348187 - config_name: humaneval-js-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202217 num_examples: 150 download_size: 247885 dataset_size: 202217 - config_name: humaneval-js-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 249617 num_examples: 150 download_size: 296185 dataset_size: 249617 - config_name: humaneval-js-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 239985 num_examples: 148 download_size: 291915 dataset_size: 239985 - config_name: humaneval-js-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 286901 num_examples: 148 download_size: 339719 dataset_size: 286901 - config_name: humaneval-js-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 277292 num_examples: 152 download_size: 321769 dataset_size: 277292 - config_name: humaneval-js-java-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 285956 num_examples: 152 download_size: 330737 dataset_size: 285956 - config_name: humaneval-js-java-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 285956 num_examples: 152 download_size: 330737 dataset_size: 285956 - config_name: humaneval-js-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 325780 num_examples: 152 download_size: 371169 dataset_size: 325780 - config_name: humaneval-js-java-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 312708 num_examples: 152 download_size: 358097 dataset_size: 312708 - config_name: humaneval-js-java-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 367800 num_examples: 152 download_size: 413395 dataset_size: 367800 - config_name: humaneval-js-java-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 336876 num_examples: 152 download_size: 382265 dataset_size: 336876 - config_name: humaneval-js-java-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 534161 num_examples: 152 download_size: 581630 dataset_size: 534161 - config_name: humaneval-js-java-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 314836 num_examples: 152 download_size: 360225 dataset_size: 314836 - config_name: humaneval-js-java-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 464303 num_examples: 152 download_size: 511145 dataset_size: 464303 - config_name: humaneval-js-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 150973 num_examples: 152 download_size: 193904 dataset_size: 150973 - config_name: humaneval-js-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199613 num_examples: 152 download_size: 243456 dataset_size: 199613 - config_name: humaneval-js-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 174797 num_examples: 154 download_size: 216732 dataset_size: 174797 - config_name: humaneval-js-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 223769 num_examples: 154 download_size: 266628 dataset_size: 223769 - config_name: humaneval-js-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 206549 num_examples: 154 download_size: 253652 dataset_size: 206549 - config_name: humaneval-js-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 255521 num_examples: 154 download_size: 303548 dataset_size: 255521 - config_name: humaneval-js-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 272219 num_examples: 154 download_size: 324102 dataset_size: 272219 - config_name: humaneval-js-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 321345 num_examples: 154 download_size: 374152 dataset_size: 321345 - config_name: humaneval-js-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 162670 num_examples: 154 download_size: 205419 dataset_size: 162670 - config_name: humaneval-js-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212104 num_examples: 154 download_size: 255777 dataset_size: 212104 - config_name: humaneval-js-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 228027 num_examples: 154 download_size: 268576 dataset_size: 228027 - config_name: humaneval-js-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 276691 num_examples: 154 download_size: 318164 dataset_size: 276691 - config_name: humaneval-js-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 171064 num_examples: 154 download_size: 215387 dataset_size: 171064 - config_name: humaneval-js-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 220190 num_examples: 154 download_size: 265437 dataset_size: 220190 - config_name: humaneval-js-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 174966 num_examples: 154 download_size: 220059 dataset_size: 174966 - config_name: humaneval-js-rkt-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 184360 num_examples: 154 download_size: 229761 dataset_size: 184360 - config_name: humaneval-js-rkt-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 184360 num_examples: 154 download_size: 229761 dataset_size: 184360 - config_name: humaneval-js-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 224400 num_examples: 154 download_size: 270417 dataset_size: 224400 - config_name: humaneval-js-rkt-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 211156 num_examples: 154 download_size: 257173 dataset_size: 211156 - config_name: humaneval-js-rkt-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 276831 num_examples: 154 download_size: 323206 dataset_size: 276831 - config_name: humaneval-js-rkt-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 235642 num_examples: 154 download_size: 281659 dataset_size: 235642 - config_name: humaneval-js-rkt-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 437971 num_examples: 154 download_size: 486106 dataset_size: 437971 - config_name: humaneval-js-rkt-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 213312 num_examples: 154 download_size: 259329 dataset_size: 213312 - config_name: humaneval-js-rkt-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 381962 num_examples: 154 download_size: 429822 dataset_size: 381962 - config_name: humaneval-js-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 167765 num_examples: 150 download_size: 207572 dataset_size: 167765 - config_name: humaneval-js-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 215615 num_examples: 150 download_size: 256322 dataset_size: 215615 - config_name: humaneval-js-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 218633 num_examples: 153 download_size: 260371 dataset_size: 218633 - config_name: humaneval-js-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 267593 num_examples: 153 download_size: 310249 dataset_size: 267593 - config_name: humaneval-js-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 157703 num_examples: 151 download_size: 205439 dataset_size: 157703 - config_name: humaneval-js-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 205872 num_examples: 151 download_size: 254514 dataset_size: 205872 - config_name: humaneval-js-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199705 num_examples: 154 download_size: 242660 dataset_size: 199705 - config_name: humaneval-js-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 248985 num_examples: 154 download_size: 292864 dataset_size: 248985 - config_name: humaneval-js-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 173490 num_examples: 152 download_size: 217784 dataset_size: 173490 - config_name: humaneval-js-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 222890 num_examples: 152 download_size: 268096 dataset_size: 222890 - config_name: humaneval-lua-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 249105 num_examples: 145 download_size: 290745 dataset_size: 249105 - config_name: humaneval-lua-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 294200 num_examples: 145 download_size: 336710 dataset_size: 294200 - config_name: humaneval-lua-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 289858 num_examples: 143 download_size: 333845 dataset_size: 289858 - config_name: humaneval-lua-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 334188 num_examples: 143 download_size: 379033 dataset_size: 334188 - config_name: humaneval-lua-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199580 num_examples: 141 download_size: 242908 dataset_size: 199580 - config_name: humaneval-lua-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 243149 num_examples: 141 download_size: 287323 dataset_size: 243149 - config_name: humaneval-lua-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 237805 num_examples: 140 download_size: 287415 dataset_size: 237805 - config_name: humaneval-lua-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 281205 num_examples: 140 download_size: 331655 dataset_size: 281205 - config_name: humaneval-lua-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 268042 num_examples: 143 download_size: 310406 dataset_size: 268042 - config_name: humaneval-lua-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 312658 num_examples: 143 download_size: 355880 dataset_size: 312658 - config_name: humaneval-lua-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 154454 num_examples: 143 download_size: 195353 dataset_size: 154454 - config_name: humaneval-lua-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199213 num_examples: 143 download_size: 240970 dataset_size: 199213 - config_name: humaneval-lua-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 169380 num_examples: 145 download_size: 212090 dataset_size: 169380 - config_name: humaneval-lua-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 215490 num_examples: 145 download_size: 259070 dataset_size: 215490 - config_name: humaneval-lua-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 206221 num_examples: 145 download_size: 251074 dataset_size: 206221 - config_name: humaneval-lua-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 251316 num_examples: 145 download_size: 297039 dataset_size: 251316 - config_name: humaneval-lua-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 266302 num_examples: 145 download_size: 315582 dataset_size: 266302 - config_name: humaneval-lua-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 311542 num_examples: 145 download_size: 361692 dataset_size: 311542 - config_name: humaneval-lua-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 165858 num_examples: 145 download_size: 206699 dataset_size: 165858 - config_name: humaneval-lua-py-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 171049 num_examples: 144 download_size: 211886 dataset_size: 171049 - config_name: humaneval-lua-py-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 171049 num_examples: 144 download_size: 211886 dataset_size: 171049 - config_name: humaneval-lua-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 211388 num_examples: 145 download_size: 253099 dataset_size: 211388 - config_name: humaneval-lua-py-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 197113 num_examples: 144 download_size: 238526 dataset_size: 197113 - config_name: humaneval-lua-py-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 263725 num_examples: 144 download_size: 305500 dataset_size: 263725 - config_name: humaneval-lua-py-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 220009 num_examples: 144 download_size: 261422 dataset_size: 220009 - config_name: humaneval-lua-py-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 412455 num_examples: 144 download_size: 456952 dataset_size: 412455 - config_name: humaneval-lua-py-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199129 num_examples: 144 download_size: 240542 dataset_size: 199129 - config_name: humaneval-lua-py-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 379514 num_examples: 144 download_size: 424606 dataset_size: 379514 - config_name: humaneval-lua-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 227113 num_examples: 145 download_size: 265894 dataset_size: 227113 - config_name: humaneval-lua-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 271918 num_examples: 145 download_size: 311569 dataset_size: 271918 - config_name: humaneval-lua-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 173848 num_examples: 145 download_size: 216123 dataset_size: 173848 - config_name: humaneval-lua-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219088 num_examples: 145 download_size: 262233 dataset_size: 219088 - config_name: humaneval-lua-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 177245 num_examples: 145 download_size: 220245 dataset_size: 177245 - config_name: humaneval-lua-rkt-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 184060 num_examples: 145 download_size: 227350 dataset_size: 184060 - config_name: humaneval-lua-rkt-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 184060 num_examples: 145 download_size: 227350 dataset_size: 184060 - config_name: humaneval-lua-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 222775 num_examples: 145 download_size: 266645 dataset_size: 222775 - config_name: humaneval-lua-rkt-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 210305 num_examples: 145 download_size: 254175 dataset_size: 210305 - config_name: humaneval-lua-rkt-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 285056 num_examples: 145 download_size: 329143 dataset_size: 285056 - config_name: humaneval-lua-rkt-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 233360 num_examples: 145 download_size: 277230 dataset_size: 233360 - config_name: humaneval-lua-rkt-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 415065 num_examples: 145 download_size: 461247 dataset_size: 415065 - config_name: humaneval-lua-rkt-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212335 num_examples: 145 download_size: 256205 dataset_size: 212335 - config_name: humaneval-lua-rkt-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 305742 num_examples: 145 download_size: 350750 dataset_size: 305742 - config_name: humaneval-lua-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 168842 num_examples: 141 download_size: 206764 dataset_size: 168842 - config_name: humaneval-lua-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212834 num_examples: 141 download_size: 251602 dataset_size: 212834 - config_name: humaneval-lua-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 215564 num_examples: 144 download_size: 255388 dataset_size: 215564 - config_name: humaneval-lua-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260636 num_examples: 144 download_size: 301324 dataset_size: 260636 - config_name: humaneval-lua-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 161345 num_examples: 142 download_size: 206768 dataset_size: 161345 - config_name: humaneval-lua-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 205649 num_examples: 142 download_size: 251924 dataset_size: 205649 - config_name: humaneval-lua-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 200743 num_examples: 145 download_size: 241731 dataset_size: 200743 - config_name: humaneval-lua-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 246128 num_examples: 145 download_size: 287986 dataset_size: 246128 - config_name: humaneval-lua-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 175616 num_examples: 143 download_size: 217817 dataset_size: 175616 - config_name: humaneval-lua-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 221090 num_examples: 143 download_size: 264149 dataset_size: 221090 - config_name: humaneval-php-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 252020 num_examples: 147 download_size: 294327 dataset_size: 252020 - config_name: humaneval-php-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 297737 num_examples: 147 download_size: 340926 dataset_size: 297737 - config_name: humaneval-php-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 293992 num_examples: 145 download_size: 338711 dataset_size: 293992 - config_name: humaneval-php-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 338942 num_examples: 145 download_size: 384531 dataset_size: 338942 - config_name: humaneval-php-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202392 num_examples: 143 download_size: 246574 dataset_size: 202392 - config_name: humaneval-php-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 246579 num_examples: 143 download_size: 291619 dataset_size: 246579 - config_name: humaneval-php-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 238346 num_examples: 141 download_size: 288459 dataset_size: 238346 - config_name: humaneval-php-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 282056 num_examples: 141 download_size: 333015 dataset_size: 282056 - config_name: humaneval-php-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 271813 num_examples: 145 download_size: 314887 dataset_size: 271813 - config_name: humaneval-php-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 317053 num_examples: 145 download_size: 360997 dataset_size: 317053 - config_name: humaneval-php-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 153588 num_examples: 145 download_size: 195191 dataset_size: 153588 - config_name: humaneval-php-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 198973 num_examples: 145 download_size: 241446 dataset_size: 198973 - config_name: humaneval-php-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 168626 num_examples: 147 download_size: 212019 dataset_size: 168626 - config_name: humaneval-php-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 215372 num_examples: 147 download_size: 259647 dataset_size: 215372 - config_name: humaneval-php-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 176751 num_examples: 147 download_size: 217306 dataset_size: 176751 - config_name: humaneval-php-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 222468 num_examples: 147 download_size: 263905 dataset_size: 222468 - config_name: humaneval-php-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 270759 num_examples: 147 download_size: 321165 dataset_size: 270759 - config_name: humaneval-php-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 316623 num_examples: 147 download_size: 367911 dataset_size: 316623 - config_name: humaneval-php-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 164932 num_examples: 147 download_size: 206252 dataset_size: 164932 - config_name: humaneval-php-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 211090 num_examples: 147 download_size: 253292 dataset_size: 211090 - config_name: humaneval-php-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 227687 num_examples: 147 download_size: 266919 dataset_size: 227687 - config_name: humaneval-php-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 273110 num_examples: 147 download_size: 313224 dataset_size: 273110 - config_name: humaneval-php-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 173192 num_examples: 147 download_size: 216144 dataset_size: 173192 - config_name: humaneval-php-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219056 num_examples: 147 download_size: 262890 dataset_size: 219056 - config_name: humaneval-php-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 176812 num_examples: 147 download_size: 220499 dataset_size: 176812 - config_name: humaneval-php-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 222970 num_examples: 147 download_size: 267539 dataset_size: 222970 - config_name: humaneval-php-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 170170 num_examples: 143 download_size: 208757 dataset_size: 170170 - config_name: humaneval-php-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 214786 num_examples: 143 download_size: 254231 dataset_size: 214786 - config_name: humaneval-php-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 216712 num_examples: 146 download_size: 257181 dataset_size: 216712 - config_name: humaneval-php-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 262410 num_examples: 146 download_size: 303755 dataset_size: 262410 - config_name: humaneval-php-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 160168 num_examples: 144 download_size: 206338 dataset_size: 160168 - config_name: humaneval-php-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 205096 num_examples: 144 download_size: 252130 dataset_size: 205096 - config_name: humaneval-php-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 200558 num_examples: 147 download_size: 242205 dataset_size: 200558 - config_name: humaneval-php-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 246569 num_examples: 147 download_size: 289098 dataset_size: 246569 - config_name: humaneval-php-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 175286 num_examples: 145 download_size: 218176 dataset_size: 175286 - config_name: humaneval-php-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 221396 num_examples: 145 download_size: 265156 dataset_size: 221396 - config_name: humaneval-pl-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 238295 num_examples: 145 download_size: 279623 dataset_size: 238295 - config_name: humaneval-pl-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 283535 num_examples: 145 download_size: 325733 dataset_size: 283535 - config_name: humaneval-pl-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 278962 num_examples: 143 download_size: 322664 dataset_size: 278962 - config_name: humaneval-pl-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 323435 num_examples: 143 download_size: 367995 dataset_size: 323435 - config_name: humaneval-pl-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 194741 num_examples: 142 download_size: 238273 dataset_size: 194741 - config_name: humaneval-pl-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 238761 num_examples: 142 download_size: 283145 dataset_size: 238761 - config_name: humaneval-pl-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 227541 num_examples: 139 download_size: 276532 dataset_size: 227541 - config_name: humaneval-pl-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 270770 num_examples: 139 download_size: 320595 dataset_size: 270770 - config_name: humaneval-pl-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 256986 num_examples: 143 download_size: 299065 dataset_size: 256986 - config_name: humaneval-pl-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 301745 num_examples: 143 download_size: 344682 dataset_size: 301745 - config_name: humaneval-pl-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 146814 num_examples: 144 download_size: 187742 dataset_size: 146814 - config_name: humaneval-pl-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 192030 num_examples: 144 download_size: 233822 dataset_size: 192030 - config_name: humaneval-pl-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 159732 num_examples: 145 download_size: 202130 dataset_size: 159732 - config_name: humaneval-pl-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 205987 num_examples: 145 download_size: 249255 dataset_size: 205987 - config_name: humaneval-pl-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 167708 num_examples: 145 download_size: 207396 dataset_size: 167708 - config_name: humaneval-pl-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212948 num_examples: 145 download_size: 253506 dataset_size: 212948 - config_name: humaneval-pl-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 196337 num_examples: 145 download_size: 240866 dataset_size: 196337 - config_name: humaneval-pl-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 241577 num_examples: 145 download_size: 286976 dataset_size: 241577 - config_name: humaneval-pl-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 156274 num_examples: 145 download_size: 196717 dataset_size: 156274 - config_name: humaneval-pl-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 201949 num_examples: 145 download_size: 243262 dataset_size: 201949 - config_name: humaneval-pl-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 217351 num_examples: 145 download_size: 255734 dataset_size: 217351 - config_name: humaneval-pl-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 262301 num_examples: 145 download_size: 301554 dataset_size: 262301 - config_name: humaneval-pl-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 164119 num_examples: 145 download_size: 206082 dataset_size: 164119 - config_name: humaneval-pl-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 209504 num_examples: 145 download_size: 252337 dataset_size: 209504 - config_name: humaneval-pl-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 167768 num_examples: 145 download_size: 210456 dataset_size: 167768 - config_name: humaneval-pl-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 213443 num_examples: 145 download_size: 257001 dataset_size: 213443 - config_name: humaneval-pl-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 162618 num_examples: 142 download_size: 200593 dataset_size: 162618 - config_name: humaneval-pl-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 207064 num_examples: 142 download_size: 245891 dataset_size: 207064 - config_name: humaneval-pl-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 203872 num_examples: 144 download_size: 243383 dataset_size: 203872 - config_name: humaneval-pl-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 249088 num_examples: 144 download_size: 289463 dataset_size: 249088 - config_name: humaneval-pl-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 153173 num_examples: 143 download_size: 198416 dataset_size: 153173 - config_name: humaneval-pl-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 197932 num_examples: 143 download_size: 244033 dataset_size: 197932 - config_name: humaneval-pl-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 191184 num_examples: 145 download_size: 231860 dataset_size: 191184 - config_name: humaneval-pl-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 236714 num_examples: 145 download_size: 278260 dataset_size: 236714 - config_name: humaneval-pl-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 167916 num_examples: 144 download_size: 210163 dataset_size: 167916 - config_name: humaneval-pl-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 213852 num_examples: 144 download_size: 256963 dataset_size: 213852 - config_name: humaneval-py-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 269818 num_examples: 161 download_size: 315390 dataset_size: 269818 - config_name: humaneval-py-cpp-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 277385 num_examples: 161 download_size: 323279 dataset_size: 277385 - config_name: humaneval-py-cpp-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 277385 num_examples: 161 download_size: 323279 dataset_size: 277385 - config_name: humaneval-py-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 320372 num_examples: 161 download_size: 366910 dataset_size: 320372 - config_name: humaneval-py-cpp-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 371950 num_examples: 161 download_size: 418742 dataset_size: 371950 - config_name: humaneval-py-cpp-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 374409 num_examples: 161 download_size: 421208 dataset_size: 374409 - config_name: humaneval-py-cpp-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 541382 num_examples: 161 download_size: 590578 dataset_size: 541382 - config_name: humaneval-py-cpp-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 541460 num_examples: 161 download_size: 590509 dataset_size: 541460 - config_name: humaneval-py-cpp-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 505605 num_examples: 161 download_size: 555456 dataset_size: 505605 - config_name: humaneval-py-cpp-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 504425 num_examples: 161 download_size: 553938 dataset_size: 504425 - config_name: humaneval-py-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 313896 num_examples: 158 download_size: 361839 dataset_size: 313896 - config_name: humaneval-py-cs-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 321006 num_examples: 158 download_size: 369265 dataset_size: 321006 - config_name: humaneval-py-cs-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 321006 num_examples: 158 download_size: 369265 dataset_size: 321006 - config_name: humaneval-py-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 363350 num_examples: 158 download_size: 412241 dataset_size: 363350 - config_name: humaneval-py-cs-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 415186 num_examples: 158 download_size: 464331 dataset_size: 415186 - config_name: humaneval-py-cs-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 415186 num_examples: 158 download_size: 464331 dataset_size: 415186 - config_name: humaneval-py-cs-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 584141 num_examples: 158 download_size: 635690 dataset_size: 584141 - config_name: humaneval-py-cs-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 576517 num_examples: 158 download_size: 627893 dataset_size: 576517 - config_name: humaneval-py-cs-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 548799 num_examples: 158 download_size: 601003 dataset_size: 548799 - config_name: humaneval-py-cs-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 547009 num_examples: 158 download_size: 599130 dataset_size: 547009 - config_name: humaneval-py-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212776 num_examples: 156 download_size: 260187 dataset_size: 212776 - config_name: humaneval-py-d-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219796 num_examples: 156 download_size: 267519 dataset_size: 219796 - config_name: humaneval-py-d-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 218979 num_examples: 156 download_size: 266659 dataset_size: 218979 - config_name: humaneval-py-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 261448 num_examples: 156 download_size: 309795 dataset_size: 261448 - config_name: humaneval-py-d-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 309470 num_examples: 155 download_size: 357743 dataset_size: 309470 - config_name: humaneval-py-d-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 311119 num_examples: 156 download_size: 359680 dataset_size: 311119 - config_name: humaneval-py-d-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 479753 num_examples: 156 download_size: 530738 dataset_size: 479753 - config_name: humaneval-py-d-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 475465 num_examples: 156 download_size: 526239 dataset_size: 475465 - config_name: humaneval-py-d-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 444576 num_examples: 156 download_size: 496166 dataset_size: 444576 - config_name: humaneval-py-d-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 435804 num_examples: 156 download_size: 487352 dataset_size: 435804 - config_name: humaneval-py-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 252050 num_examples: 154 download_size: 306002 dataset_size: 252050 - config_name: humaneval-py-go-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 258980 num_examples: 154 download_size: 313240 dataset_size: 258980 - config_name: humaneval-py-go-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 258980 num_examples: 154 download_size: 313240 dataset_size: 258980 - config_name: humaneval-py-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 300252 num_examples: 154 download_size: 355128 dataset_size: 300252 - config_name: humaneval-py-go-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 350796 num_examples: 154 download_size: 405926 dataset_size: 350796 - config_name: humaneval-py-go-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 350796 num_examples: 154 download_size: 405926 dataset_size: 350796 - config_name: humaneval-py-go-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 516168 num_examples: 154 download_size: 573643 dataset_size: 516168 - config_name: humaneval-py-go-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 508544 num_examples: 154 download_size: 565846 dataset_size: 508544 - config_name: humaneval-py-go-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 480876 num_examples: 154 download_size: 538994 dataset_size: 480876 - config_name: humaneval-py-go-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 479086 num_examples: 154 download_size: 537121 dataset_size: 479086 - config_name: humaneval-py-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 290226 num_examples: 158 download_size: 336381 dataset_size: 290226 - config_name: humaneval-py-java-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 297968 num_examples: 158 download_size: 344439 dataset_size: 297968 - config_name: humaneval-py-java-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 297968 num_examples: 158 download_size: 344439 dataset_size: 297968 - config_name: humaneval-py-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 339996 num_examples: 158 download_size: 387099 dataset_size: 339996 - config_name: humaneval-py-java-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 326566 num_examples: 158 download_size: 373669 dataset_size: 326566 - config_name: humaneval-py-java-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 400643 num_examples: 158 download_size: 448121 dataset_size: 400643 - config_name: humaneval-py-java-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 348528 num_examples: 158 download_size: 395631 dataset_size: 348528 - config_name: humaneval-py-java-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 553308 num_examples: 158 download_size: 603112 dataset_size: 553308 - config_name: humaneval-py-java-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 328778 num_examples: 158 download_size: 375881 dataset_size: 328778 - config_name: humaneval-py-java-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 519914 num_examples: 158 download_size: 570421 dataset_size: 519914 - config_name: humaneval-py-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 160858 num_examples: 159 download_size: 205695 dataset_size: 160858 - config_name: humaneval-py-jl-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 169286 num_examples: 159 download_size: 214441 dataset_size: 169286 - config_name: humaneval-py-jl-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 169286 num_examples: 159 download_size: 214441 dataset_size: 169286 - config_name: humaneval-py-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 211102 num_examples: 159 download_size: 256893 dataset_size: 211102 - config_name: humaneval-py-jl-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260852 num_examples: 159 download_size: 306857 dataset_size: 260852 - config_name: humaneval-py-jl-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 261461 num_examples: 159 download_size: 307468 dataset_size: 261461 - config_name: humaneval-py-jl-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 430527 num_examples: 159 download_size: 478964 dataset_size: 430527 - config_name: humaneval-py-jl-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 429919 num_examples: 159 download_size: 478175 dataset_size: 429919 - config_name: humaneval-py-jl-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 394574 num_examples: 159 download_size: 443616 dataset_size: 394574 - config_name: humaneval-py-jl-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 389099 num_examples: 159 download_size: 438158 dataset_size: 389099 - config_name: humaneval-py-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 176840 num_examples: 161 download_size: 223588 dataset_size: 176840 - config_name: humaneval-py-js-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 186661 num_examples: 161 download_size: 233731 dataset_size: 186661 - config_name: humaneval-py-js-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 186661 num_examples: 161 download_size: 233731 dataset_size: 186661 - config_name: humaneval-py-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 228521 num_examples: 161 download_size: 276235 dataset_size: 228521 - config_name: humaneval-py-js-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 280099 num_examples: 161 download_size: 328067 dataset_size: 280099 - config_name: humaneval-py-js-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 281303 num_examples: 161 download_size: 329274 dataset_size: 281303 - config_name: humaneval-py-js-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 449531 num_examples: 161 download_size: 499903 dataset_size: 449531 - config_name: humaneval-py-js-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 445997 num_examples: 161 download_size: 496227 dataset_size: 445997 - config_name: humaneval-py-js-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 413754 num_examples: 161 download_size: 464781 dataset_size: 413754 - config_name: humaneval-py-js-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 413788 num_examples: 161 download_size: 464748 dataset_size: 413788 - config_name: humaneval-py-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 185721 num_examples: 161 download_size: 229429 dataset_size: 185721 - config_name: humaneval-py-lua-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 193288 num_examples: 161 download_size: 237318 dataset_size: 193288 - config_name: humaneval-py-lua-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 193289 num_examples: 161 download_size: 237319 dataset_size: 193289 - config_name: humaneval-py-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 236275 num_examples: 161 download_size: 280949 dataset_size: 236275 - config_name: humaneval-py-lua-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 287853 num_examples: 161 download_size: 332781 dataset_size: 287853 - config_name: humaneval-py-lua-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 289380 num_examples: 161 download_size: 334311 dataset_size: 289380 - config_name: humaneval-py-lua-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 457285 num_examples: 161 download_size: 504617 dataset_size: 457285 - config_name: humaneval-py-lua-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 456931 num_examples: 161 download_size: 504092 dataset_size: 456931 - config_name: humaneval-py-lua-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 421508 num_examples: 161 download_size: 469495 dataset_size: 421508 - config_name: humaneval-py-lua-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 416080 num_examples: 161 download_size: 464009 dataset_size: 416080 - config_name: humaneval-py-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 218780 num_examples: 161 download_size: 267949 dataset_size: 218780 - config_name: humaneval-py-php-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 226347 num_examples: 161 download_size: 275838 dataset_size: 226347 - config_name: humaneval-py-php-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 226347 num_examples: 161 download_size: 275838 dataset_size: 226347 - config_name: humaneval-py-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 269334 num_examples: 161 download_size: 319469 dataset_size: 269334 - config_name: humaneval-py-php-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 320912 num_examples: 161 download_size: 371301 dataset_size: 320912 - config_name: humaneval-py-php-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 322463 num_examples: 161 download_size: 372857 dataset_size: 322463 - config_name: humaneval-py-php-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 490344 num_examples: 161 download_size: 543137 dataset_size: 490344 - config_name: humaneval-py-php-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 486382 num_examples: 161 download_size: 539027 dataset_size: 486382 - config_name: humaneval-py-php-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 454567 num_examples: 161 download_size: 508015 dataset_size: 454567 - config_name: humaneval-py-php-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 455059 num_examples: 161 download_size: 508442 dataset_size: 455059 - config_name: humaneval-py-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 287584 num_examples: 161 download_size: 341821 dataset_size: 287584 - config_name: humaneval-py-pl-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 295473 num_examples: 161 download_size: 350032 dataset_size: 295473 - config_name: humaneval-py-pl-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 295473 num_examples: 161 download_size: 350032 dataset_size: 295473 - config_name: humaneval-py-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 338299 num_examples: 161 download_size: 393502 dataset_size: 338299 - config_name: humaneval-py-pl-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 389877 num_examples: 161 download_size: 445334 dataset_size: 389877 - config_name: humaneval-py-pl-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 391293 num_examples: 161 download_size: 446753 dataset_size: 391293 - config_name: humaneval-py-pl-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 559309 num_examples: 161 download_size: 617170 dataset_size: 559309 - config_name: humaneval-py-pl-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 556594 num_examples: 161 download_size: 614334 dataset_size: 556594 - config_name: humaneval-py-pl-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 523532 num_examples: 161 download_size: 582048 dataset_size: 523532 - config_name: humaneval-py-pl-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 524915 num_examples: 161 download_size: 583377 dataset_size: 524915 - config_name: humaneval-py-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 241329 num_examples: 161 download_size: 283588 dataset_size: 241329 - config_name: humaneval-py-r-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 242950 num_examples: 161 download_size: 285531 dataset_size: 242950 - config_name: humaneval-py-r-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 242950 num_examples: 161 download_size: 285531 dataset_size: 242950 - config_name: humaneval-py-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 291561 num_examples: 161 download_size: 334786 dataset_size: 291561 - config_name: humaneval-py-r-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 343139 num_examples: 161 download_size: 386618 dataset_size: 343139 - config_name: humaneval-py-r-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 344732 num_examples: 161 download_size: 388216 dataset_size: 344732 - config_name: humaneval-py-r-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 512571 num_examples: 161 download_size: 558454 dataset_size: 512571 - config_name: humaneval-py-r-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 512765 num_examples: 161 download_size: 558474 dataset_size: 512765 - config_name: humaneval-py-r-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 476794 num_examples: 161 download_size: 523332 dataset_size: 476794 - config_name: humaneval-py-r-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 472181 num_examples: 161 download_size: 518702 dataset_size: 472181 - config_name: humaneval-py-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 181804 num_examples: 161 download_size: 228069 dataset_size: 181804 - config_name: humaneval-py-rb-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 189693 num_examples: 161 download_size: 236280 dataset_size: 189693 - config_name: humaneval-py-rb-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 189693 num_examples: 161 download_size: 236280 dataset_size: 189693 - config_name: humaneval-py-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 232519 num_examples: 161 download_size: 279750 dataset_size: 232519 - config_name: humaneval-py-rb-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 284097 num_examples: 161 download_size: 331582 dataset_size: 284097 - config_name: humaneval-py-rb-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 285661 num_examples: 161 download_size: 333151 dataset_size: 285661 - config_name: humaneval-py-rb-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 453529 num_examples: 161 download_size: 503418 dataset_size: 453529 - config_name: humaneval-py-rb-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 450659 num_examples: 161 download_size: 500414 dataset_size: 450659 - config_name: humaneval-py-rb-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 417752 num_examples: 161 download_size: 468296 dataset_size: 417752 - config_name: humaneval-py-rb-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 417579 num_examples: 161 download_size: 468053 dataset_size: 417579 - config_name: humaneval-py-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 185786 num_examples: 161 download_size: 232856 dataset_size: 185786 - config_name: humaneval-py-rkt-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 195608 num_examples: 161 download_size: 243322 dataset_size: 195608 - config_name: humaneval-py-rkt-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 195608 num_examples: 161 download_size: 243322 dataset_size: 195608 - config_name: humaneval-py-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 236823 num_examples: 161 download_size: 284859 dataset_size: 236823 - config_name: humaneval-py-rkt-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 288401 num_examples: 161 download_size: 336691 dataset_size: 288401 - config_name: humaneval-py-rkt-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 289879 num_examples: 161 download_size: 338172 dataset_size: 289879 - config_name: humaneval-py-rkt-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 457833 num_examples: 161 download_size: 508527 dataset_size: 457833 - config_name: humaneval-py-rkt-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 458741 num_examples: 161 download_size: 509443 dataset_size: 458741 - config_name: humaneval-py-rkt-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 422056 num_examples: 161 download_size: 473405 dataset_size: 422056 - config_name: humaneval-py-rkt-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 417513 num_examples: 161 download_size: 469002 dataset_size: 417513 - config_name: humaneval-py-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 177453 num_examples: 156 download_size: 218775 dataset_size: 177453 - config_name: humaneval-py-rs-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 185097 num_examples: 156 download_size: 226731 dataset_size: 185097 - config_name: humaneval-py-rs-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 185097 num_examples: 156 download_size: 226731 dataset_size: 185097 - config_name: humaneval-py-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 226593 num_examples: 156 download_size: 268851 dataset_size: 226593 - config_name: humaneval-py-rs-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 276264 num_examples: 156 download_size: 318736 dataset_size: 276264 - config_name: humaneval-py-rs-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 276264 num_examples: 156 download_size: 318736 dataset_size: 276264 - config_name: humaneval-py-rs-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 444898 num_examples: 156 download_size: 489794 dataset_size: 444898 - config_name: humaneval-py-rs-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 440454 num_examples: 156 download_size: 485139 dataset_size: 440454 - config_name: humaneval-py-rs-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 409721 num_examples: 156 download_size: 455222 dataset_size: 409721 - config_name: humaneval-py-rs-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 400793 num_examples: 156 download_size: 446252 dataset_size: 400793 - config_name: humaneval-py-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 230813 num_examples: 160 download_size: 274387 dataset_size: 230813 - config_name: humaneval-py-scala-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 238973 num_examples: 160 download_size: 282867 dataset_size: 238973 - config_name: humaneval-py-scala-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 238973 num_examples: 160 download_size: 282867 dataset_size: 238973 - config_name: humaneval-py-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 281373 num_examples: 160 download_size: 325907 dataset_size: 281373 - config_name: humaneval-py-scala-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 333037 num_examples: 160 download_size: 377825 dataset_size: 333037 - config_name: humaneval-py-scala-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 333800 num_examples: 160 download_size: 378590 dataset_size: 333800 - config_name: humaneval-py-scala-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 502310 num_examples: 160 download_size: 549502 dataset_size: 502310 - config_name: humaneval-py-scala-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 501032 num_examples: 160 download_size: 548034 dataset_size: 501032 - config_name: humaneval-py-scala-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 466678 num_examples: 160 download_size: 514525 dataset_size: 466678 - config_name: humaneval-py-scala-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 460602 num_examples: 160 download_size: 508399 dataset_size: 460602 - config_name: humaneval-py-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 168044 num_examples: 158 download_size: 217907 dataset_size: 168044 - config_name: humaneval-py-sh-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 265467 num_examples: 158 download_size: 328031 dataset_size: 265467 - config_name: humaneval-py-sh-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 171047 num_examples: 158 download_size: 217118 dataset_size: 171047 - config_name: humaneval-py-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 217814 num_examples: 158 download_size: 268625 dataset_size: 217814 - config_name: humaneval-py-sh-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 268867 num_examples: 158 download_size: 319930 dataset_size: 268867 - config_name: humaneval-py-sh-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 269975 num_examples: 158 download_size: 321041 dataset_size: 269975 - config_name: humaneval-py-sh-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 435325 num_examples: 158 download_size: 488768 dataset_size: 435325 - config_name: humaneval-py-sh-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 435283 num_examples: 158 download_size: 488535 dataset_size: 435283 - config_name: humaneval-py-sh-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 399908 num_examples: 158 download_size: 453977 dataset_size: 399908 - config_name: humaneval-py-sh-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 394864 num_examples: 158 download_size: 448917 dataset_size: 394864 - config_name: humaneval-py-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 211908 num_examples: 161 download_size: 256746 dataset_size: 211908 - config_name: humaneval-py-swift-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 220120 num_examples: 161 download_size: 265280 dataset_size: 220120 - config_name: humaneval-py-swift-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 217795 num_examples: 161 download_size: 262805 dataset_size: 217795 - config_name: humaneval-py-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 262784 num_examples: 161 download_size: 308588 dataset_size: 262784 - config_name: humaneval-py-swift-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 314362 num_examples: 161 download_size: 360420 dataset_size: 314362 - config_name: humaneval-py-swift-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 316185 num_examples: 161 download_size: 362250 dataset_size: 316185 - config_name: humaneval-py-swift-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 483794 num_examples: 161 download_size: 532256 dataset_size: 483794 - config_name: humaneval-py-swift-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 483304 num_examples: 161 download_size: 531594 dataset_size: 483304 - config_name: humaneval-py-swift-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 448017 num_examples: 161 download_size: 497134 dataset_size: 448017 - config_name: humaneval-py-swift-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 439996 num_examples: 161 download_size: 489032 dataset_size: 439996 - config_name: humaneval-py-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 184431 num_examples: 159 download_size: 230695 dataset_size: 184431 - config_name: humaneval-py-ts-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 194130 num_examples: 159 download_size: 240712 dataset_size: 194130 - config_name: humaneval-py-ts-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 194130 num_examples: 159 download_size: 240712 dataset_size: 194130 - config_name: humaneval-py-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 235470 num_examples: 159 download_size: 282688 dataset_size: 235470 - config_name: humaneval-py-ts-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 221796 num_examples: 159 download_size: 269014 dataset_size: 221796 - config_name: humaneval-py-ts-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 289202 num_examples: 159 download_size: 336673 dataset_size: 289202 - config_name: humaneval-py-ts-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 455904 num_examples: 159 download_size: 505829 dataset_size: 455904 - config_name: humaneval-py-ts-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 450589 num_examples: 159 download_size: 500308 dataset_size: 450589 - config_name: humaneval-py-ts-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 419176 num_examples: 159 download_size: 469649 dataset_size: 419176 - config_name: humaneval-py-ts-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 417590 num_examples: 159 download_size: 467978 dataset_size: 417590 - config_name: humaneval-r-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212937 num_examples: 124 download_size: 249007 dataset_size: 212937 - config_name: humaneval-r-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 251253 num_examples: 124 download_size: 288067 dataset_size: 251253 - config_name: humaneval-r-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 246255 num_examples: 122 download_size: 284221 dataset_size: 246255 - config_name: humaneval-r-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 283831 num_examples: 122 download_size: 322529 dataset_size: 283831 - config_name: humaneval-r-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 175594 num_examples: 122 download_size: 213592 dataset_size: 175594 - config_name: humaneval-r-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 213048 num_examples: 122 download_size: 251778 dataset_size: 213048 - config_name: humaneval-r-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 203273 num_examples: 118 download_size: 245535 dataset_size: 203273 - config_name: humaneval-r-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 239617 num_examples: 118 download_size: 282587 dataset_size: 239617 - config_name: humaneval-r-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 226937 num_examples: 122 download_size: 263511 dataset_size: 226937 - config_name: humaneval-r-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 264757 num_examples: 122 download_size: 302063 dataset_size: 264757 - config_name: humaneval-r-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 135483 num_examples: 123 download_size: 171154 dataset_size: 135483 - config_name: humaneval-r-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 173736 num_examples: 123 download_size: 210145 dataset_size: 173736 - config_name: humaneval-r-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 147219 num_examples: 124 download_size: 184191 dataset_size: 147219 - config_name: humaneval-r-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 186403 num_examples: 124 download_size: 224119 dataset_size: 186403 - config_name: humaneval-r-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 154097 num_examples: 124 download_size: 188603 dataset_size: 154097 - config_name: humaneval-r-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 192413 num_examples: 124 download_size: 227663 dataset_size: 192413 - config_name: humaneval-r-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 178757 num_examples: 124 download_size: 217579 dataset_size: 178757 - config_name: humaneval-r-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 217073 num_examples: 124 download_size: 256639 dataset_size: 217073 - config_name: humaneval-r-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 230697 num_examples: 124 download_size: 273496 dataset_size: 230697 - config_name: humaneval-r-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 269137 num_examples: 124 download_size: 312680 dataset_size: 269137 - config_name: humaneval-r-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 143997 num_examples: 124 download_size: 179139 dataset_size: 143997 - config_name: humaneval-r-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 182685 num_examples: 124 download_size: 218571 dataset_size: 182685 - config_name: humaneval-r-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 151059 num_examples: 124 download_size: 187659 dataset_size: 151059 - config_name: humaneval-r-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 189499 num_examples: 124 download_size: 226843 dataset_size: 189499 - config_name: humaneval-r-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 154062 num_examples: 124 download_size: 191282 dataset_size: 154062 - config_name: humaneval-r-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 192750 num_examples: 124 download_size: 230714 dataset_size: 192750 - config_name: humaneval-r-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 150322 num_examples: 122 download_size: 183610 dataset_size: 150322 - config_name: humaneval-r-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 188142 num_examples: 122 download_size: 222162 dataset_size: 188142 - config_name: humaneval-r-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 183572 num_examples: 123 download_size: 218017 dataset_size: 183572 - config_name: humaneval-r-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 221825 num_examples: 123 download_size: 257008 dataset_size: 221825 - config_name: humaneval-r-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 139820 num_examples: 121 download_size: 178747 dataset_size: 139820 - config_name: humaneval-r-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 177330 num_examples: 121 download_size: 216983 dataset_size: 177330 - config_name: humaneval-r-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 174179 num_examples: 124 download_size: 209681 dataset_size: 174179 - config_name: humaneval-r-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212743 num_examples: 124 download_size: 248989 dataset_size: 212743 - config_name: humaneval-r-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 153893 num_examples: 123 download_size: 190675 dataset_size: 153893 - config_name: humaneval-r-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 192761 num_examples: 123 download_size: 230281 dataset_size: 192761 - config_name: humaneval-rb-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 231623 num_examples: 150 download_size: 273585 dataset_size: 231623 - config_name: humaneval-rb-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 278423 num_examples: 150 download_size: 321285 dataset_size: 278423 - config_name: humaneval-rb-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 274917 num_examples: 148 download_size: 319353 dataset_size: 274917 - config_name: humaneval-rb-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 320945 num_examples: 148 download_size: 366269 dataset_size: 320945 - config_name: humaneval-rb-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 183368 num_examples: 146 download_size: 227277 dataset_size: 183368 - config_name: humaneval-rb-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 228628 num_examples: 146 download_size: 273413 dataset_size: 228628 - config_name: humaneval-rb-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219876 num_examples: 144 download_size: 269862 dataset_size: 219876 - config_name: humaneval-rb-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 264660 num_examples: 144 download_size: 315510 dataset_size: 264660 - config_name: humaneval-rb-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 252457 num_examples: 148 download_size: 295215 dataset_size: 252457 - config_name: humaneval-rb-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 298781 num_examples: 148 download_size: 342427 dataset_size: 298781 - config_name: humaneval-rb-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 133568 num_examples: 148 download_size: 174824 dataset_size: 133568 - config_name: humaneval-rb-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 180040 num_examples: 148 download_size: 222184 dataset_size: 180040 - config_name: humaneval-rb-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 148669 num_examples: 150 download_size: 191741 dataset_size: 148669 - config_name: humaneval-rb-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 196519 num_examples: 150 download_size: 240491 dataset_size: 196519 - config_name: humaneval-rb-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 156903 num_examples: 150 download_size: 197249 dataset_size: 156903 - config_name: humaneval-rb-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 203703 num_examples: 150 download_size: 244949 dataset_size: 203703 - config_name: humaneval-rb-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 187561 num_examples: 150 download_size: 232895 dataset_size: 187561 - config_name: humaneval-rb-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 234361 num_examples: 150 download_size: 280595 dataset_size: 234361 - config_name: humaneval-rb-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 250848 num_examples: 150 download_size: 300782 dataset_size: 250848 - config_name: humaneval-rb-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 297798 num_examples: 150 download_size: 348632 dataset_size: 297798 - config_name: humaneval-rb-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 144887 num_examples: 150 download_size: 186015 dataset_size: 144887 - config_name: humaneval-rb-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 192137 num_examples: 150 download_size: 234165 dataset_size: 192137 - config_name: humaneval-rb-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 208692 num_examples: 150 download_size: 247688 dataset_size: 208692 - config_name: humaneval-rb-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 255192 num_examples: 150 download_size: 295088 dataset_size: 255192 - config_name: humaneval-rb-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 156934 num_examples: 150 download_size: 200306 dataset_size: 156934 - config_name: humaneval-rb-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 204184 num_examples: 150 download_size: 248456 dataset_size: 204184 - config_name: humaneval-rb-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 149391 num_examples: 146 download_size: 187587 dataset_size: 149391 - config_name: humaneval-rb-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 195089 num_examples: 146 download_size: 234161 dataset_size: 195089 - config_name: humaneval-rb-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 197038 num_examples: 149 download_size: 237135 dataset_size: 197038 - config_name: humaneval-rb-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 243824 num_examples: 149 download_size: 284815 dataset_size: 243824 - config_name: humaneval-rb-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 140661 num_examples: 147 download_size: 186640 dataset_size: 140661 - config_name: humaneval-rb-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 186672 num_examples: 147 download_size: 233533 dataset_size: 186672 - config_name: humaneval-rb-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 180695 num_examples: 150 download_size: 221985 dataset_size: 180695 - config_name: humaneval-rb-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 227795 num_examples: 150 download_size: 269985 dataset_size: 227795 - config_name: humaneval-rb-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 155703 num_examples: 148 download_size: 198284 dataset_size: 155703 - config_name: humaneval-rb-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202915 num_examples: 148 download_size: 246384 dataset_size: 202915 - config_name: humaneval-rkt-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 234376 num_examples: 138 download_size: 273371 dataset_size: 234376 - config_name: humaneval-rkt-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 277708 num_examples: 138 download_size: 317531 dataset_size: 277708 - config_name: humaneval-rkt-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 273011 num_examples: 136 download_size: 314174 dataset_size: 273011 - config_name: humaneval-rkt-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 315579 num_examples: 136 download_size: 357558 dataset_size: 315579 - config_name: humaneval-rkt-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 185556 num_examples: 135 download_size: 226552 dataset_size: 185556 - config_name: humaneval-rkt-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 227676 num_examples: 135 download_size: 269482 dataset_size: 227676 - config_name: humaneval-rkt-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219186 num_examples: 133 download_size: 265720 dataset_size: 219186 - config_name: humaneval-rkt-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260815 num_examples: 133 download_size: 308147 dataset_size: 260815 - config_name: humaneval-rkt-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 253685 num_examples: 136 download_size: 293352 dataset_size: 253685 - config_name: humaneval-rkt-java-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260349 num_examples: 136 download_size: 300288 dataset_size: 260349 - config_name: humaneval-rkt-java-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 260349 num_examples: 136 download_size: 300288 dataset_size: 260349 - config_name: humaneval-rkt-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 296525 num_examples: 136 download_size: 337008 dataset_size: 296525 - config_name: humaneval-rkt-java-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 284829 num_examples: 136 download_size: 325312 dataset_size: 284829 - config_name: humaneval-rkt-java-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 368367 num_examples: 136 download_size: 409227 dataset_size: 368367 - config_name: humaneval-rkt-java-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 306453 num_examples: 136 download_size: 346936 dataset_size: 306453 - config_name: humaneval-rkt-java-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 386788 num_examples: 136 download_size: 428036 dataset_size: 386788 - config_name: humaneval-rkt-java-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 286733 num_examples: 136 download_size: 327216 dataset_size: 286733 - config_name: humaneval-rkt-java-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 360256 num_examples: 136 download_size: 401396 dataset_size: 360256 - config_name: humaneval-rkt-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 139996 num_examples: 137 download_size: 178615 dataset_size: 139996 - config_name: humaneval-rkt-jl-direct-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 146983 num_examples: 137 download_size: 185876 dataset_size: 146983 - config_name: humaneval-rkt-jl-direct-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 146983 num_examples: 137 download_size: 185876 dataset_size: 146983 - config_name: humaneval-rkt-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 183288 num_examples: 137 download_size: 222729 dataset_size: 183288 - config_name: humaneval-rkt-jl-exp-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 171506 num_examples: 137 download_size: 210947 dataset_size: 171506 - config_name: humaneval-rkt-jl-exp-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 228800 num_examples: 137 download_size: 268464 dataset_size: 228800 - config_name: humaneval-rkt-jl-exp-lbl-d-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 193289 num_examples: 137 download_size: 232730 dataset_size: 193289 - config_name: humaneval-rkt-jl-exp-lbl-d-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 283282 num_examples: 137 download_size: 323590 dataset_size: 283282 - config_name: humaneval-rkt-jl-exp-lbl-gpt3.5 features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 173424 num_examples: 137 download_size: 212865 dataset_size: 173424 - config_name: humaneval-rkt-jl-exp-lbl-gpt3.5exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 229140 num_examples: 137 download_size: 269059 dataset_size: 229140 - config_name: humaneval-rkt-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 152504 num_examples: 138 download_size: 192513 dataset_size: 152504 - config_name: humaneval-rkt-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 196802 num_examples: 138 download_size: 237639 dataset_size: 196802 - config_name: humaneval-rkt-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 160077 num_examples: 138 download_size: 197572 dataset_size: 160077 - config_name: humaneval-rkt-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 203409 num_examples: 138 download_size: 241732 dataset_size: 203409 - config_name: humaneval-rkt-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 188793 num_examples: 138 download_size: 230884 dataset_size: 188793 - config_name: humaneval-rkt-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 232125 num_examples: 138 download_size: 275044 dataset_size: 232125 - config_name: humaneval-rkt-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 247633 num_examples: 138 download_size: 293988 dataset_size: 247633 - config_name: humaneval-rkt-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 291103 num_examples: 138 download_size: 338286 dataset_size: 291103 - config_name: humaneval-rkt-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 149500 num_examples: 138 download_size: 187741 dataset_size: 149500 - config_name: humaneval-rkt-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 193246 num_examples: 138 download_size: 232315 dataset_size: 193246 - config_name: humaneval-rkt-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 207864 num_examples: 138 download_size: 244117 dataset_size: 207864 - config_name: humaneval-rkt-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 250920 num_examples: 138 download_size: 288001 dataset_size: 250920 - config_name: humaneval-rkt-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 156746 num_examples: 138 download_size: 196341 dataset_size: 156746 - config_name: humaneval-rkt-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 200216 num_examples: 138 download_size: 240639 dataset_size: 200216 - config_name: humaneval-rkt-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 154865 num_examples: 135 download_size: 190614 dataset_size: 154865 - config_name: humaneval-rkt-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 197390 num_examples: 135 download_size: 233949 dataset_size: 197390 - config_name: humaneval-rkt-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 200104 num_examples: 137 download_size: 237348 dataset_size: 200104 - config_name: humaneval-rkt-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 243396 num_examples: 137 download_size: 281462 dataset_size: 243396 - config_name: humaneval-rkt-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 144413 num_examples: 135 download_size: 187013 dataset_size: 144413 - config_name: humaneval-rkt-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 186938 num_examples: 135 download_size: 230348 dataset_size: 186938 - config_name: humaneval-rkt-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 182332 num_examples: 138 download_size: 220703 dataset_size: 182332 - config_name: humaneval-rkt-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 225940 num_examples: 138 download_size: 265139 dataset_size: 225940 - config_name: humaneval-rkt-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 160290 num_examples: 137 download_size: 200137 dataset_size: 160290 - config_name: humaneval-rkt-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 204267 num_examples: 137 download_size: 244936 dataset_size: 204267 - config_name: humaneval-rs-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 239849 num_examples: 144 download_size: 280657 dataset_size: 239849 - config_name: humaneval-rs-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 284777 num_examples: 144 download_size: 326449 dataset_size: 284777 - config_name: humaneval-rs-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 287984 num_examples: 144 download_size: 331866 dataset_size: 287984 - config_name: humaneval-rs-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 332768 num_examples: 144 download_size: 377514 dataset_size: 332768 - config_name: humaneval-rs-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199820 num_examples: 144 download_size: 243773 dataset_size: 199820 - config_name: humaneval-rs-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 244460 num_examples: 144 download_size: 289277 dataset_size: 244460 - config_name: humaneval-rs-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 236009 num_examples: 142 download_size: 285919 dataset_size: 236009 - config_name: humaneval-rs-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 280171 num_examples: 142 download_size: 330933 dataset_size: 280171 - config_name: humaneval-rs-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 265698 num_examples: 144 download_size: 307946 dataset_size: 265698 - config_name: humaneval-rs-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 310770 num_examples: 144 download_size: 353882 dataset_size: 310770 - config_name: humaneval-rs-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 149700 num_examples: 144 download_size: 190503 dataset_size: 149700 - config_name: humaneval-rs-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 194916 num_examples: 144 download_size: 236583 dataset_size: 194916 - config_name: humaneval-rs-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 161725 num_examples: 144 download_size: 203685 dataset_size: 161725 - config_name: humaneval-rs-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 207661 num_examples: 144 download_size: 250485 dataset_size: 207661 - config_name: humaneval-rs-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 169676 num_examples: 144 download_size: 208936 dataset_size: 169676 - config_name: humaneval-rs-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 214604 num_examples: 144 download_size: 254728 dataset_size: 214604 - config_name: humaneval-rs-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 198815 num_examples: 144 download_size: 242921 dataset_size: 198815 - config_name: humaneval-rs-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 243743 num_examples: 144 download_size: 288713 dataset_size: 243743 - config_name: humaneval-rs-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 259499 num_examples: 144 download_size: 308130 dataset_size: 259499 - config_name: humaneval-rs-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 304571 num_examples: 144 download_size: 354066 dataset_size: 304571 - config_name: humaneval-rs-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 158142 num_examples: 144 download_size: 198158 dataset_size: 158142 - config_name: humaneval-rs-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 203502 num_examples: 144 download_size: 244382 dataset_size: 203502 - config_name: humaneval-rs-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219234 num_examples: 144 download_size: 257198 dataset_size: 219234 - config_name: humaneval-rs-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 263874 num_examples: 144 download_size: 302702 dataset_size: 263874 - config_name: humaneval-rs-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 166183 num_examples: 144 download_size: 207711 dataset_size: 166183 - config_name: humaneval-rs-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 211255 num_examples: 144 download_size: 253647 dataset_size: 211255 - config_name: humaneval-rs-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 169627 num_examples: 144 download_size: 211875 dataset_size: 169627 - config_name: humaneval-rs-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 214987 num_examples: 144 download_size: 258099 dataset_size: 214987 - config_name: humaneval-rs-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 210402 num_examples: 144 download_size: 249770 dataset_size: 210402 - config_name: humaneval-rs-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 255618 num_examples: 144 download_size: 295850 dataset_size: 255618 - config_name: humaneval-rs-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 153547 num_examples: 141 download_size: 198108 dataset_size: 153547 - config_name: humaneval-rs-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 197680 num_examples: 141 download_size: 243087 dataset_size: 197680 - config_name: humaneval-rs-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 192278 num_examples: 144 download_size: 232510 dataset_size: 192278 - config_name: humaneval-rs-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 237494 num_examples: 144 download_size: 278590 dataset_size: 237494 - config_name: humaneval-rs-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 170987 num_examples: 144 download_size: 213091 dataset_size: 170987 - config_name: humaneval-rs-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 216923 num_examples: 144 download_size: 259891 dataset_size: 216923 - config_name: humaneval-scala-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 237640 num_examples: 142 download_size: 278183 dataset_size: 237640 - config_name: humaneval-scala-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 282086 num_examples: 142 download_size: 323481 dataset_size: 282086 - config_name: humaneval-scala-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 284322 num_examples: 142 download_size: 327897 dataset_size: 284322 - config_name: humaneval-scala-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 328626 num_examples: 142 download_size: 373053 dataset_size: 328626 - config_name: humaneval-scala-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 196365 num_examples: 140 download_size: 239218 dataset_size: 196365 - config_name: humaneval-scala-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 239905 num_examples: 140 download_size: 283598 dataset_size: 239905 - config_name: humaneval-scala-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 240142 num_examples: 142 download_size: 290341 dataset_size: 240142 - config_name: humaneval-scala-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 284446 num_examples: 142 download_size: 335497 dataset_size: 284446 - config_name: humaneval-scala-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 261864 num_examples: 142 download_size: 303827 dataset_size: 261864 - config_name: humaneval-scala-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 306452 num_examples: 142 download_size: 349267 dataset_size: 306452 - config_name: humaneval-scala-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 151189 num_examples: 141 download_size: 191361 dataset_size: 151189 - config_name: humaneval-scala-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 195604 num_examples: 141 download_size: 236622 dataset_size: 195604 - config_name: humaneval-scala-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 164558 num_examples: 142 download_size: 206237 dataset_size: 164558 - config_name: humaneval-scala-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 209998 num_examples: 142 download_size: 252529 dataset_size: 209998 - config_name: humaneval-scala-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 172439 num_examples: 142 download_size: 211396 dataset_size: 172439 - config_name: humaneval-scala-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 216885 num_examples: 142 download_size: 256694 dataset_size: 216885 - config_name: humaneval-scala-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 200283 num_examples: 142 download_size: 244038 dataset_size: 200283 - config_name: humaneval-scala-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 244729 num_examples: 142 download_size: 289336 dataset_size: 244729 - config_name: humaneval-scala-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 258416 num_examples: 142 download_size: 306608 dataset_size: 258416 - config_name: humaneval-scala-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 303004 num_examples: 142 download_size: 352048 dataset_size: 303004 - config_name: humaneval-scala-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 160912 num_examples: 142 download_size: 200603 dataset_size: 160912 - config_name: humaneval-scala-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 205784 num_examples: 142 download_size: 246327 dataset_size: 205784 - config_name: humaneval-scala-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 220989 num_examples: 142 download_size: 258668 dataset_size: 220989 - config_name: humaneval-scala-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 265151 num_examples: 142 download_size: 303682 dataset_size: 265151 - config_name: humaneval-scala-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 168923 num_examples: 142 download_size: 210176 dataset_size: 168923 - config_name: humaneval-scala-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 213511 num_examples: 142 download_size: 255616 dataset_size: 213511 - config_name: humaneval-scala-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 172465 num_examples: 142 download_size: 214428 dataset_size: 172465 - config_name: humaneval-scala-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 217337 num_examples: 142 download_size: 260152 dataset_size: 217337 - config_name: humaneval-scala-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 167996 num_examples: 140 download_size: 205553 dataset_size: 167996 - config_name: humaneval-scala-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 211956 num_examples: 140 download_size: 250353 dataset_size: 211956 - config_name: humaneval-scala-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 156724 num_examples: 139 download_size: 200831 dataset_size: 156724 - config_name: humaneval-scala-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 200370 num_examples: 139 download_size: 245311 dataset_size: 200370 - config_name: humaneval-scala-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 194899 num_examples: 142 download_size: 234874 dataset_size: 194899 - config_name: humaneval-scala-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 239629 num_examples: 142 download_size: 280456 dataset_size: 239629 - config_name: humaneval-scala-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 171888 num_examples: 141 download_size: 213356 dataset_size: 171888 - config_name: humaneval-scala-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 217008 num_examples: 141 download_size: 259322 dataset_size: 217008 - config_name: humaneval-sh-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 251470 num_examples: 146 download_size: 294288 dataset_size: 251470 - config_name: humaneval-sh-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 297022 num_examples: 146 download_size: 340716 dataset_size: 297022 - config_name: humaneval-sh-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 293082 num_examples: 144 download_size: 338211 dataset_size: 293082 - config_name: humaneval-sh-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 337866 num_examples: 144 download_size: 383859 dataset_size: 337866 - config_name: humaneval-sh-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202928 num_examples: 142 download_size: 247505 dataset_size: 202928 - config_name: humaneval-sh-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 246948 num_examples: 142 download_size: 292377 dataset_size: 246948 - config_name: humaneval-sh-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 238508 num_examples: 140 download_size: 289006 dataset_size: 238508 - config_name: humaneval-sh-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 282048 num_examples: 140 download_size: 333386 dataset_size: 282048 - config_name: humaneval-sh-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 271913 num_examples: 144 download_size: 315458 dataset_size: 271913 - config_name: humaneval-sh-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 316985 num_examples: 144 download_size: 361394 dataset_size: 316985 - config_name: humaneval-sh-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 154739 num_examples: 144 download_size: 196813 dataset_size: 154739 - config_name: humaneval-sh-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199955 num_examples: 144 download_size: 242893 dataset_size: 199955 - config_name: humaneval-sh-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 170002 num_examples: 146 download_size: 213898 dataset_size: 170002 - config_name: humaneval-sh-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 216576 num_examples: 146 download_size: 261348 dataset_size: 216576 - config_name: humaneval-sh-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 178026 num_examples: 146 download_size: 219182 dataset_size: 178026 - config_name: humaneval-sh-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 223578 num_examples: 146 download_size: 265610 dataset_size: 223578 - config_name: humaneval-sh-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 208413 num_examples: 146 download_size: 254543 dataset_size: 208413 - config_name: humaneval-sh-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 253965 num_examples: 146 download_size: 300971 dataset_size: 253965 - config_name: humaneval-sh-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 271336 num_examples: 146 download_size: 322136 dataset_size: 271336 - config_name: humaneval-sh-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 317034 num_examples: 146 download_size: 368710 dataset_size: 317034 - config_name: humaneval-sh-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 166204 num_examples: 146 download_size: 208122 dataset_size: 166204 - config_name: humaneval-sh-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212194 num_examples: 146 download_size: 254988 dataset_size: 212194 - config_name: humaneval-sh-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 228638 num_examples: 146 download_size: 268480 dataset_size: 228638 - config_name: humaneval-sh-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 273898 num_examples: 146 download_size: 314616 dataset_size: 273898 - config_name: humaneval-sh-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 174509 num_examples: 146 download_size: 217967 dataset_size: 174509 - config_name: humaneval-sh-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 220207 num_examples: 146 download_size: 264541 dataset_size: 220207 - config_name: humaneval-sh-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 177929 num_examples: 146 download_size: 222117 dataset_size: 177929 - config_name: humaneval-sh-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 223919 num_examples: 146 download_size: 268983 dataset_size: 223919 - config_name: humaneval-sh-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 170658 num_examples: 142 download_size: 209706 dataset_size: 170658 - config_name: humaneval-sh-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 215104 num_examples: 142 download_size: 255004 dataset_size: 215104 - config_name: humaneval-sh-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 218375 num_examples: 145 download_size: 259347 dataset_size: 218375 - config_name: humaneval-sh-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 263905 num_examples: 145 download_size: 305747 dataset_size: 263905 - config_name: humaneval-sh-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 201387 num_examples: 146 download_size: 243549 dataset_size: 201387 - config_name: humaneval-sh-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 247231 num_examples: 146 download_size: 290269 dataset_size: 247231 - config_name: humaneval-sh-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 176498 num_examples: 144 download_size: 219845 dataset_size: 176498 - config_name: humaneval-sh-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 222434 num_examples: 144 download_size: 266645 dataset_size: 222434 - config_name: humaneval-swift-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 244248 num_examples: 145 download_size: 285400 dataset_size: 244248 - config_name: humaneval-swift-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 289633 num_examples: 145 download_size: 331655 dataset_size: 289633 - config_name: humaneval-swift-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 289740 num_examples: 144 download_size: 333633 dataset_size: 289740 - config_name: humaneval-swift-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 334668 num_examples: 144 download_size: 379425 dataset_size: 334668 - config_name: humaneval-swift-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 196918 num_examples: 142 download_size: 240264 dataset_size: 196918 - config_name: humaneval-swift-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 241080 num_examples: 142 download_size: 285278 dataset_size: 241080 - config_name: humaneval-swift-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 233850 num_examples: 141 download_size: 283438 dataset_size: 233850 - config_name: humaneval-swift-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 277842 num_examples: 141 download_size: 328276 dataset_size: 277842 - config_name: humaneval-swift-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 268335 num_examples: 144 download_size: 310594 dataset_size: 268335 - config_name: humaneval-swift-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 313551 num_examples: 144 download_size: 356674 dataset_size: 313551 - config_name: humaneval-swift-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 148838 num_examples: 144 download_size: 189610 dataset_size: 148838 - config_name: humaneval-swift-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 194198 num_examples: 144 download_size: 235834 dataset_size: 194198 - config_name: humaneval-swift-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 162180 num_examples: 145 download_size: 204470 dataset_size: 162180 - config_name: humaneval-swift-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 208580 num_examples: 145 download_size: 251740 dataset_size: 208580 - config_name: humaneval-swift-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 170170 num_examples: 145 download_size: 209790 dataset_size: 170170 - config_name: humaneval-swift-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 215555 num_examples: 145 download_size: 256045 dataset_size: 215555 - config_name: humaneval-swift-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 199430 num_examples: 145 download_size: 243859 dataset_size: 199430 - config_name: humaneval-swift-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 244815 num_examples: 145 download_size: 290114 dataset_size: 244815 - config_name: humaneval-swift-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 259658 num_examples: 145 download_size: 308558 dataset_size: 259658 - config_name: humaneval-swift-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 305188 num_examples: 145 download_size: 354958 dataset_size: 305188 - config_name: humaneval-swift-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 158817 num_examples: 145 download_size: 199200 dataset_size: 158817 - config_name: humaneval-swift-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 204637 num_examples: 145 download_size: 245890 dataset_size: 204637 - config_name: humaneval-swift-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 220024 num_examples: 145 download_size: 258339 dataset_size: 220024 - config_name: humaneval-swift-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 265119 num_examples: 145 download_size: 304304 dataset_size: 265119 - config_name: humaneval-swift-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 166624 num_examples: 145 download_size: 208479 dataset_size: 166624 - config_name: humaneval-swift-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212154 num_examples: 145 download_size: 254879 dataset_size: 212154 - config_name: humaneval-swift-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 170204 num_examples: 145 download_size: 212784 dataset_size: 170204 - config_name: humaneval-swift-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 216024 num_examples: 145 download_size: 259474 dataset_size: 216024 - config_name: humaneval-swift-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 164268 num_examples: 142 download_size: 202073 dataset_size: 164268 - config_name: humaneval-swift-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 208856 num_examples: 142 download_size: 247513 dataset_size: 208856 - config_name: humaneval-swift-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 212749 num_examples: 145 download_size: 252429 dataset_size: 212749 - config_name: humaneval-swift-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 258424 num_examples: 145 download_size: 298974 dataset_size: 258424 - config_name: humaneval-swift-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 155205 num_examples: 143 download_size: 200491 dataset_size: 155205 - config_name: humaneval-swift-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 200107 num_examples: 143 download_size: 246251 dataset_size: 200107 - config_name: humaneval-swift-ts-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 169982 num_examples: 144 download_size: 212066 dataset_size: 169982 - config_name: humaneval-swift-ts-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 216062 num_examples: 144 download_size: 259010 dataset_size: 216062 - config_name: humaneval-ts-cpp-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 256270 num_examples: 153 download_size: 299449 dataset_size: 256270 - config_name: humaneval-ts-cpp-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 304924 num_examples: 153 download_size: 349021 dataset_size: 304924 - config_name: humaneval-ts-cs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 304810 num_examples: 152 download_size: 350894 dataset_size: 304810 - config_name: humaneval-ts-cs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 352994 num_examples: 152 download_size: 399990 dataset_size: 352994 - config_name: humaneval-ts-d-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 208777 num_examples: 151 download_size: 254693 dataset_size: 208777 - config_name: humaneval-ts-d-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 256493 num_examples: 151 download_size: 303315 dataset_size: 256493 - config_name: humaneval-ts-go-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 244584 num_examples: 148 download_size: 296399 dataset_size: 244584 - config_name: humaneval-ts-go-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 291500 num_examples: 148 download_size: 344203 dataset_size: 291500 - config_name: humaneval-ts-java-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 282120 num_examples: 152 download_size: 326482 dataset_size: 282120 - config_name: humaneval-ts-java-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 330608 num_examples: 152 download_size: 375882 dataset_size: 330608 - config_name: humaneval-ts-jl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 157354 num_examples: 153 download_size: 200508 dataset_size: 157354 - config_name: humaneval-ts-jl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 206314 num_examples: 153 download_size: 250386 dataset_size: 206314 - config_name: humaneval-ts-js-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 170073 num_examples: 153 download_size: 214454 dataset_size: 170073 - config_name: humaneval-ts-js-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 219798 num_examples: 153 download_size: 265097 dataset_size: 219798 - config_name: humaneval-ts-lua-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 178511 num_examples: 153 download_size: 220064 dataset_size: 178511 - config_name: humaneval-ts-lua-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 227165 num_examples: 153 download_size: 269636 dataset_size: 227165 - config_name: humaneval-ts-php-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 209789 num_examples: 153 download_size: 256455 dataset_size: 209789 - config_name: humaneval-ts-php-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 258443 num_examples: 153 download_size: 306027 dataset_size: 258443 - config_name: humaneval-ts-pl-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 274663 num_examples: 153 download_size: 326053 dataset_size: 274663 - config_name: humaneval-ts-pl-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 323470 num_examples: 153 download_size: 375778 dataset_size: 323470 - config_name: humaneval-ts-py-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 166367 num_examples: 153 download_size: 208727 dataset_size: 166367 - config_name: humaneval-ts-py-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 215480 num_examples: 153 download_size: 258758 dataset_size: 215480 - config_name: humaneval-ts-r-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 231284 num_examples: 153 download_size: 271460 dataset_size: 231284 - config_name: humaneval-ts-r-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 279632 num_examples: 153 download_size: 320726 dataset_size: 279632 - config_name: humaneval-ts-rb-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 174790 num_examples: 153 download_size: 218712 dataset_size: 174790 - config_name: humaneval-ts-rb-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 223597 num_examples: 153 download_size: 268437 dataset_size: 223597 - config_name: humaneval-ts-rkt-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 178653 num_examples: 153 download_size: 223340 dataset_size: 178653 - config_name: humaneval-ts-rkt-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 227766 num_examples: 153 download_size: 273371 dataset_size: 227766 - config_name: humaneval-ts-rs-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 174451 num_examples: 151 download_size: 214468 dataset_size: 174451 - config_name: humaneval-ts-rs-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 222620 num_examples: 151 download_size: 263543 dataset_size: 222620 - config_name: humaneval-ts-scala-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 223457 num_examples: 153 download_size: 265084 dataset_size: 223457 - config_name: humaneval-ts-scala-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 272417 num_examples: 153 download_size: 314962 dataset_size: 272417 - config_name: humaneval-ts-sh-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 161391 num_examples: 150 download_size: 208649 dataset_size: 161391 - config_name: humaneval-ts-sh-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 209241 num_examples: 150 download_size: 257399 dataset_size: 209241 - config_name: humaneval-ts-swift-direct features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 202575 num_examples: 153 download_size: 245124 dataset_size: 202575 - config_name: humaneval-ts-swift-exp features: - name: name dtype: string - name: language dtype: string - name: prompt dtype: string - name: doctests dtype: string - name: original dtype: string - name: prompt_terminology dtype: string - name: source_program dtype: string - name: target_signature dtype: string - name: tests dtype: string - name: stop_tokens sequence: string - name: translation_prompt dtype: string splits: - name: test num_bytes: 251535 num_examples: 153 download_size: 295002 dataset_size: 251535 --- # Dataset Card for MultiPL-C2C ## Dataset Description - **Repository:** https://github.com/PootieT/explain-then-translate - **Paper:** https://openreview.net/forum?id=uyUO80sbm0 - **Point of Contact:** zilutang@bu.edu ## Dataset Summary MultiPL-C2C is a dataset for evaluating large language models for code translation between any of the 19 programming languages. It is an extension from MultiPL-E dataset, which takes the OpenAI "HumanEval" and the MBPP Python benchmarks and uses little compilers to translate them to other languages. On top of MultiPL-E, it is easy to add support for new languages and benchmarks. For MultiPL-C2C, we release only the subset for HumanEval. ## Subsets To evaluate program translation with MultiPL-C2C, you need mainly three arguments: *SRC_LANG*, *TGT_LANG*, and *EXP*. We use the canonical file extension for each language to identify the language, e.g., "py" for Python, "cpp" for C++, "lua" for Lua, and so on. *EXP* specifies what type of prompting is done for the translation: `direct` is direct translation. {`exp`, `exp-lbl`, `exp-lbl-d`} are other explain-then-translate method. See paper for detailed description. Our paper ["explain-then-translate"](https://openreview.net/forum?id=uyUO80sbm0) explores a Chain-of-Thought like 2-stage prompting methods to improve 0-shot translation accuracy. We also provide a few other variations: - *SRCDATA-SRC_LANG-TGT_LANG-EXP-gpt3.5* is the same as *SRCDATA-SRC_LANG-TGT_LANG-EXP*, but are the prompts we used to evaluated GPT-3.5 in March 2023. For example, in Python-to-X direction, we cached Python-Java intermediate generations (explanations), and reused them in other Python-to-X directions. - *SRCDATA-SRC_LANG-TGT_LANG-EXP-gpt3.5exp* is a step further to control reproducibility for our experiments on top of *SRCDATA-SRC_LANG-TGT_LANG-EXP-gpt3.5*. We additionally cached all intermediate generations (explanations) in the prompts. Different from *SRCDATA-SRC_LANG-TGT_LANG-EXP-gpt3.5*, translation direction such as `Lua-Python` will have explanations generated by GPT-3.5 Note that we did not experiment with MBPP even thought it is available with MultiPL-E. Feel free to PR and we can include them to further improve code-translation datasets. ## Example The following script uses the Salesforce/codegen model to translate Python-Lua and MultiPL-E to produce a script with unit tests for luaunit. ```python import datasets from transformers import AutoTokenizer, AutoModelForCausalLM SRC_LANG = "py" TGT_LANG = "lua" EXP = "direct" MODEL_NAME = "Salesforce/codegen-350M-multi" TOFILL_TOKEN = "{{{TOFILL}}}" tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) model = AutoModelForCausalLM.from_pretrained(MODEL_NAME).half().cuda() problems = datasets.load_dataset("zilu-peter-tang/MultiPL-C2C", f"humaneval-{SRC_LANG}-{TGT_LANG}-{EXP}") def stop_at_stop_token(decoded_string, problem): """ Truncates the output at stop tokens, taking care to skip the prompt which may have stop tokens. """ min_stop_index = len(decoded_string) for stop_token in problem["stop_tokens"]: stop_index = decoded_string.find(stop_token, len(problem["translation_prompt"])) if stop_index != -1 and stop_index < min_stop_index: min_stop_index = stop_index return decoded_string[:min_stop_index] def generate(problem, tokenizer, model): input_ids = tokenizer( problem["translation_prompt"], return_tensors="pt", ).input_ids.cuda() generated_ids = model.generate( input_ids, max_length=512, pad_token_id=tokenizer.eos_token_id + 2 ) truncated_string = stop_at_stop_token(tokenizer.decode(generated_ids[0]), problem) return truncated_string for problem in problems["test"]: # generate each intermediate steps (explanation) sequentially, if any stop = problem["stop_tokens"] prompt = problem["translation_prompt"] while TOFILL_TOKEN in problem["translation_prompt"]: incomplete_prompt = prompt[:prompt.find(TOFILL_TOKEN)] problem["stop_tokens"] = ["\n#"] problem["translation_prompt"] = incomplete_prompt intermediate_completion = generate(problem, tokenizer, model).replace(incomplete_prompt, "") prompt = prompt.replace(TOFILL_TOKEN, intermediate_completion, 1) # finally generate translation problem["translation_prompt"] = prompt problem["stop_tokens"] = stop completion = generate(problem, tokenizer, model).replace(prompt, "") filename = problem["name"] + "-" + SRC_LANG + "-" + TGT_LANG + "." + TGT_LANG with open(filename, "w") as f: print(f"Created {filename}") f.write(problem["prompt"]) f.write(completion) f.write("\n") f.write(problem["tests"]) ``` Depending on the quality of the model, you may or may not find some extraneous natural langauge or program commands at the end of the generated functions. To trim them using programmatic rules, see our [repo](https://github.com/PootieT/explain-then-translate/blob/main/MultiPL-C2C/inference/__main__.py) for a more detailed inference script that takes care of this in post-processing. For few shot examples checkout our all other translation prompts used [here](https://github.com/PootieT/explain-then-translate/tree/main/MultiPL-C2C/translation_prompts/translation_prompts_chatgpt_with_explanation)
提供机构：
zilu-peter-tang
原始信息汇总

数据集概述

数据集配置

humaneval-cpp-cs-direct

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 323975
      • num_examples: 148
  • 下载大小: 370379
  • 数据集大小: 323975

humaneval-cpp-cs-exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 369855
      • num_examples: 148
  • 下载大小: 417147
  • 数据集大小: 369855

humaneval-cpp-d-direct

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 228556
      • num_examples: 146
  • 下载大小: 274393
  • 数据集大小: 228556

humaneval-cpp-d-exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 273670
      • num_examples: 146
  • 下载大小: 320383
  • 数据集大小: 273670

humaneval-cpp-go-direct

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 266725
      • num_examples: 145
  • 下载大小: 318988
  • 数据集大小: 266725

humaneval-cpp-go-exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 311675
      • num_examples: 145
  • 下载大小: 364808
  • 数据集大小: 311675

humaneval-cpp-java-direct

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 301674
      • num_examples: 148
  • 下载大小: 346400
  • 数据集大小: 301674

humaneval-cpp-java-exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 347850
      • num_examples: 148
  • 下载大小: 393464
  • 数据集大小: 347850

humaneval-cpp-jl-direct

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 180272
      • num_examples: 148
  • 下载大小: 223520
  • 数据集大小: 180272

humaneval-cpp-jl-exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 226596
      • num_examples: 148
  • 下载大小: 270732
  • 数据集大小: 226596

humaneval-cpp-js-direct

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 194639
      • num_examples: 149
  • 下载大小: 239420
  • 数据集大小: 194639

humaneval-cpp-js-exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 242021
      • num_examples: 149
  • 下载大小: 287696
  • 数据集大小: 242021

humaneval-cpp-lua-direct

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 202873
      • num_examples: 149
  • 下载大小: 244832
  • 数据集大小: 202873

humaneval-cpp-lua-direct-gpt3.5

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 208982
      • num_examples: 149
  • 下载大小: 251239
  • 数据集大小: 208982

humaneval-cpp-lua-direct-gpt3.5exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 208982
      • num_examples: 149
  • 下载大小: 251239
  • 数据集大小: 208982

humaneval-cpp-lua-exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 249212
      • num_examples: 149
  • 下载大小: 292065
  • 数据集大小: 249212

humaneval-cpp-lua-exp-gpt3.5

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 236398
      • num_examples: 149
  • 下载大小: 279251
  • 数据集大小: 236398

humaneval-cpp-lua-exp-gpt3.5exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 299368
      • num_examples: 149
  • 下载大小: 342412
  • 数据集大小: 299368

humaneval-cpp-lua-exp-lbl-d-gpt3.5

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 260089
      • num_examples: 149
  • 下载大小: 302942
  • 数据集大小: 260089

humaneval-cpp-lua-exp-lbl-d-gpt3.5exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 498101
      • num_examples: 149
  • 下载大小: 543921
  • 数据集大小: 498101

humaneval-cpp-lua-exp-lbl-gpt3.5

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 238484
      • num_examples: 149
  • 下载大小: 281337
  • 数据集大小: 238484

humaneval-cpp-lua-exp-lbl-gpt3.5exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 402215
      • num_examples: 149
  • 下载大小: 447326
  • 数据集大小: 402215

humaneval-cpp-php-direct

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 233624
      • num_examples: 149
  • 下载大小: 280652
  • 数据集大小: 233624

humaneval-cpp-php-exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 279963
      • num_examples: 149
  • 下载大小: 327885
  • 数据集大小: 279963

humaneval-cpp-pl-direct

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
    • tests: string
    • stop_tokens: sequence of string
    • translation_prompt: string
  • 分割:
    • test:
      • num_bytes: 297359
      • num_examples: 149
  • 下载大小: 349118
  • 数据集大小: 297359

humaneval-cpp-pl-exp

  • 特征:
    • name: string
    • language: string
    • prompt: string
    • doctests: string
    • original: string
    • prompt_terminology: string
    • source_program: string
    • target_signature: string
5,000+
优质数据集
54 个
任务类型
进入经典数据集
关于我们
遇见数据集——让每个数据集都被发现，让每一次遇见都有价值。
数发科官网www.sfktec.com.cn
联系我们
youjian
selectdataset@iotsh.com.cn
商业合作
数据驱动未来，携手共赢发展
二维码
社区交流群

面向社区/商业的数据集话题

二维码
科研交流群

面向高校/科研机构的开源数据集话题

数据驱动未来

携手共赢发展

商业合作