ali77sina/SEC-QA-sorted-chunks

---
license: mit
task_categories:
- question-answering
language:
- en
tags:
- finance
size_categories:
- 1K<n<10K
---
This data comprises synthetic question and answer pairs created by GPT-4-turbo on SEC filings for 29 companies. The dataset has the following columns:

questions, answers, chunks and sorted_chunks.

questions: the list of questions, there were 5 questions created for a 2000 word section of different SEC filings.

answers: the answer generated by GPT-4.

chunks: these are the bits of text that are segmented.

sorted_chunks: these are the chunks being sorted, using Dense Passage Retrieval (DPR).

The sorting is done by using the embeddings extracted from the BERT models fine-tuned for query and context embedding for DPR. The sorting is then done by using FAISS using IndexFlatL2.

The following is an example of how one can sort the chunks using DPR pre-trained embeddings and FAISS:

First the DPR models should be imported as such:

```python
from transformers import DPRConfig, DPRContextEncoder, DPRQuestionEncoder, DPRQuestionEncoderTokenizer, DPRContextEncoderTokenizer

# context tokenizer and encoder
tokenizer_c = DPRContextEncoderTokenizer.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base")
model_c = DPRContextEncoder.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base", device_map = 'cuda')

# question tokenizer and encoder
tokenizer_q = DPRQuestionEncoderTokenizer.from_pretrained("facebook/dpr-question_encoder-single-nq-base")
model_q = DPRQuestionEncoder.from_pretrained("facebook/dpr-question_encoder-single-nq-base", device_map = 'cuda')
```

The created chunks can then be sorted using FAISS as such:

```python
import faiss

sorted_chunks = []

print(f'----- total rows: {len(answers)}\n')

for q,a,c in tqdm(zip(questions[:], answers[:], chunks[:])):
index = faiss.IndexFlatL2(768) # Assuming embeddings are of size 768
# Encode texts using context encoder
encoded_texts = tokenizer_c(c, return_tensors='pt', padding = 'max_length')['input_ids']
encoded_texts = model_c(encoded_texts.cuda()).pooler_output

# Encode question using question encoder
encoded_question = tokenizer_q(q, return_tensors='pt')['input_ids']
encoded_question = model_q(encoded_question.cuda()).pooler_output

if len(c)>4:
# Perform FAISS search to get top 5 texts
faiss_texts = encoded_texts.detach().cpu().numpy()
faiss_question = encoded_question.detach().cpu().numpy()
# faiss_question = encoded_question.detach().cpu().numpy()
index.add(np.array(faiss_texts))
_, faiss_topk = index.search(faiss_question, k=4)
top_texts = [c[j] for j in faiss_topk[0]]
top_t = ''
for j in top_texts:
top_t += j + ', '
sorted_chunks.append(top_t)

else:
# Perform FAISS search to get top 5 texts
faiss_texts = encoded_texts.detach().cpu().numpy()
faiss_question = encoded_question.detach().cpu().numpy()
# faiss_question = encoded_question.detach().cpu().numpy()
index.add(np.array(faiss_texts))
_, faiss_topk = index.search(faiss_question, k=len(c))
top_texts = [c[j] for j in faiss_topk[0]]
top_t = ''
for j in top_texts:
top_t += j + ', '
sorted_chunks.append(top_t)

df = pd.DataFrame(data = {'questions':questions[:],
'answers':answers[:], 'chunks':chunks[:], 'sorted_chunks':sorted_chunks})
```

原始信息汇总

数据集概述

基本信息

• 许可证：MIT
• 任务类别：问答
• 语言：英语
• 标签：金融
• 数据量：1K<n<10K

数据内容

• 列信息：
  • questions：问题列表，每个2000字的SEC文件部分生成5个问题。
  • answers：由GPT-4生成的答案。
  • chunks：文本片段。
  • sorted_chunks：使用Dense Passage Retrieval (DPR)排序的文本片段。

数据处理

• 排序方法：
  • 使用BERT模型微调的查询和上下文嵌入进行DPR处理。
  • 使用FAISS和IndexFlatL2进行排序。

示例代码

• DPR模型导入： python from transformers import DPRConfig, DPRContextEncoder, DPRQuestionEncoder, DPRQuestionEncoderTokenizer, DPRContextEncoderTokenizer

  上下文编码器和分词器

  tokenizer_c = DPRContextEncoderTokenizer.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base") model_c = DPRContextEncoder.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base", device_map = cuda)

  问题编码器和分词器

  tokenizer_q = DPRQuestionEncoderTokenizer.from_pretrained("facebook/dpr-question_encoder-single-nq-base") model_q = DPRQuestionEncoder.from_pretrained("facebook/dpr-question_encoder-single-nq-base", device_map = cuda)

• 使用FAISS排序： python import faiss

  sorted_chunks = []

  print(f----- 总行数: {len(answers)} )

  for q,a,c in tqdm(zip(questions[:], answers[:], chunks[:])): index = faiss.IndexFlatL2(768) # 假设嵌入大小为768

  使用上下文编码器编码文本

  encoded_texts = tokenizer_c(c, return_tensors=pt, padding = max_length)[input_ids] encoded_texts = model_c(encoded_texts.cuda()).pooler_output

  使用问题编码器编码问题

  encoded_question = tokenizer_q(q, return_tensors=pt)[input_ids] encoded_question = model_q(encoded_question.cuda()).pooler_output

  if len(c)>4: # 执行FAISS搜索以获取前5个文本 faiss_texts = encoded_texts.detach().cpu().numpy() faiss_question = encoded_question.detach().cpu().numpy() index.add(np.array(faiss_texts)) _, faiss_topk = index.search(faiss_question, k=4) top_texts = [c[j] for j in faiss_topk[0]] top_t = for j in top_texts: top_t += j + , sorted_chunks.append(top_t)

  else: # 执行FAISS搜索以获取前5个文本 faiss_texts = encoded_texts.detach().cpu().numpy() faiss_question = encoded_question.detach().cpu().numpy() index.add(np.array(faiss_texts)) _, faiss_topk = index.search(faiss_question, k=len(c)) top_texts = [c[j] for j in faiss_topk[0]] top_t = for j in top_texts: top_t += j + , sorted_chunks.append(top_t)

  df = pd.DataFrame(data = {questions:questions[:], answers:answers[:], chunks:chunks[:], sorted_chunks:sorted_chunks})