CoIR-Retrieval/codefeedback-mt-queries-corpus

--- dataset_info: features: - name: _id dtype: string - name: partition dtype: string - name: text dtype: string - name: title dtype: string splits: - name: queries num_bytes: 295280604 num_examples: 66383 - name: corpus num_bytes: 99230769 num_examples: 66383 download_size: 176595250 dataset_size: 394511373 --- Employing the CoIR evaluation framework's dataset version, utilize the code below for assessment: ```python import coir from coir.data_loader import get_tasks from coir.evaluation import COIR from coir.models import YourCustomDEModel model_name = "intfloat/e5-base-v2" # Load the model model = YourCustomDEModel(model_name=model_name) # Get tasks #all task ["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic- # text2sql","cosqa","codesearchnet","codesearchnet-ccr"] tasks = get_tasks(tasks=["codetrans-dl"]) # Initialize evaluation evaluation = COIR(tasks=tasks,batch_size=128) # Run evaluation results = evaluation.run(model, output_folder=f"results/{model_name}") print(results) ```数据集信息： 特征字段： - 字段名：_id，数据类型：字符串 - 字段名：partition，数据类型：字符串 - 字段名：text，数据类型：字符串 - 字段名：title，数据类型：字符串 数据划分： - 划分名称：queries，占用字节数：295280604，样本数：66383 - 划分名称：corpus，占用字节数：99230769，样本数：66383 下载大小：176595250 数据集总大小：394511373 --- 采用CoIR评估框架的数据集版本，请使用下述代码开展评估： python import coir from coir.data_loader import get_tasks from coir.evaluation import COIR from coir.models import YourCustomDEModel model_name = "intfloat/e5-base-v2" # 加载模型 model = YourCustomDEModel(model_name=model_name) # 获取任务 # 所有任务包括["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic-text2sql","cosqa","codesearchnet","codesearchnet-ccr"] tasks = get_tasks(tasks=["codetrans-dl"]) # 初始化评估 evaluation = COIR(tasks=tasks, batch_size=128) # 执行评估 results = evaluation.run(model, output_folder=f"results/{model_name}") print(results)

原始信息汇总

数据集概述

数据集信息

• 特征字段:
  • _id: 类型为字符串
  • partition: 类型为字符串
  • text: 类型为字符串
  • title: 类型为字符串

数据分割

• queries:
  • 字节数: 295280604
  • 样本数: 66383
• corpus:
  • 字节数: 99230769
  • 样本数: 66383

数据集大小

• 下载大小: 176595250 字节
• 数据集总大小: 394511373 字节