uer_t5-small-chinese-cluecorpussmall

# Chinese T5

## Model description

This is the set of Chinese T5 models pre-trained by [UER-py](https://github.com/dbiir/UER-py/), which is introduced in [this paper](https://arxiv.org/abs/1909.05658). Besides, the models could also be pre-trained by [TencentPretrain](https://github.com/Tencent/TencentPretrain) introduced in [this paper](https://arxiv.org/abs/2212.06385), which inherits UER-py to support models with parameters above one billion, and extends it to a multimodal pre-training framework.

# 中文T5

## 模型描述

本系列为基于[UER-py](https://github.com/dbiir/UER-py/)预训练的中文T5模型，相关研究可参见[该论文](https://arxiv.org/abs/1909.05658)。此外，该系列模型也可通过[腾讯预训练框架(TencentPretrain)](https://github.com/Tencent/TencentPretrain)进行预训练，相关介绍可参见[此论文](https://arxiv.org/abs/2212.06385)。该框架继承了UER-py的能力，支持参数量超过十亿的模型预训练，并将其拓展至多模态预训练框架。

搜集汇总

背景与挑战

背景概述

该数据集是一个中文T5小型模型，基于CLUECorpusSmall训练，适用于文本生成任务。模型由UER-py和TencentPretrain预训练，提供了从预处理到模型转换的完整训练流程。