kstevica/llm-comparison

---
license: mit
task_categories:
- text-generation
language:
- en
tags:
- stories
pretty_name: LLM Comparison
size_categories:
- n<1K
---
# Fine tuning progress validation - RedPajama 3B, StableLM Alpha 7B, Open-LLaMA

This repository contains the progress of fine-tuning models: RedPajama 3B, StableLM Alpha 7B, Open-LLaMA. These models have been fine-tuned on a specific text dataset and the results of the fine-tuning process are provided in the text file included in this repository.

## Fine-Tuning Details

- **Model: RedPajama 3B, size: 3 billion parameters, method: adapter**
- **Model: StableLM Alpha 7B, size: 7 billion parameters, method: adapter**
- **Model: Open-LLaMA 7B 300B, size: 7 billion parameters (300B tokens), method: LoRA**
- **Model: Open-LLaMA 7B 300B, size: 7 billion parameters (300B tokens), method: adapter**

## Dataset

The text source used for fine-tuning these models has a size of 25MB, which has been split into 174,000 data inputs.

## Fine-Tuning Process

The fine-tuning process was conducted with the following details:

- **Epochs:** 1
- **Validation Frequency:** Every 1% of the training data
- **Training Data:** 174,000 data inputs

## Acknowledgments #1

I would like to acknowledge @stabilityai, @togethercompute and OpenLM Research for providing the base models. Their groundbreaking work in the field of natural language processing has made projects like this possible.

## Acknowledgments #2

I would like to acknowledge @LightningAI for providing the lit-parrot fine-tuning framework.

## Disclaimer

There might be NSFW results in the results.

## License

This repository and the fine-tuned models are licensed under the [MIT License](LICENSE). Feel free to modify and use them according to the terms of the license.

原始信息汇总

数据集概述

数据集名称

• 名称: LLM Comparison

数据集描述

• 目的: 用于验证RedPajama 3B, StableLM Alpha 7B, Open-LLaMA模型的微调进度。
• 内容: 包含这些模型的微调结果，微调基于特定的文本数据集。

模型详情

• RedPajama 3B:
  • 参数大小: 30亿
  • 微调方法: 适配器
• StableLM Alpha 7B:
  • 参数大小: 70亿
  • 微调方法: 适配器
• Open-LLaMA 7B 300B:
  • 参数大小: 70亿（300B tokens）
  • 微调方法: LoRA 和 适配器

数据集详情

• 数据源大小: 25MB
• 数据输入数量: 174,000

微调过程

• 训练周期: 1
• 验证频率: 每1%的训练数据
• 训练数据量: 174,000数据输入

许可证

• 许可证: MIT License