chinese-bert-wwm-ext

hfl
/
chinese-bert-wwm-ext Copied
like
72
Fill-Mask
PyTorch
TensorFlow
JAX
Transformers
Chinese
bert
AutoTrain Compatible

arxiv:
1906.08101

arxiv:
2004.13922
License:
apache-2.0
Model card
Files and versions
Chinese BERT with Whole Word Masking
For further accelerating Chinese natural language processing, we provide Chinese pre-trained BERT with Whole Word Masking.

Pre-Training with Whole Word Masking for Chinese BERT
Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu

This repository is developed based on：https://github.com/google-research/bert

You may also interested in,

Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm
Chinese MacBERT: https://github.com/ymcui/MacBERT
Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA
Chinese XLNet: https://github.com/ymcui/Chinese-XLNet
Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer
More resources by HFL: https://github.com/ymcui/HFL-Anthology

Citation
If you find the technical report or resource is useful, please cite the following technical report in your paper.

Primary: https://arxiv.org/abs/2004.13922
@inproceedings{cui-etal-2020-revisiting,
title = "Revisiting Pre-Trained Models for {C}hinese Natural Language Processing",
author = "Cui, Yiming and
Che, Wanxiang and
Liu, Ting and
Qin, Bing and
Wang, Shijin and
Hu, Guoping",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings",
month = nov,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/2020.findings-emnlp.58",
pages = "657--668",
}

Secondary: https://arxiv.org/abs/1906.08101
@article{chinese-bert-wwm,
title={Pre-Training with Whole Word Masking for Chinese BERT},
author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},
journal={arXiv preprint arXiv:1906.08101},
year={2019}
}

哈工大讯飞联合实验室（HFL）的chinese-bert-wwm-ext模型 已被复制，获赞72次。支持掩码填充（Fill-Mask）任务，兼容PyTorch、TensorFlow、JAX框架，适配Transformers库，面向中文场景，基于BERT模型，支持AutoTrain训练。
arXiv: 1906.08101
arXiv: 2004.13922
许可证: Apache 2.0
模型卡片、文件与版本

带全词掩码的中文BERT
为进一步加速中文自然语言处理研究，我们发布了基于全词掩码的中文预训练BERT模型。

《中文BERT的全词掩码预训练》
作者：崔一鸣、车万翔、刘挺、秦兵、杨子青、王石金、胡国平

本仓库基于谷歌研究院的bert仓库（https://github.com/google-research/bert）开发。

您可能还会感兴趣的资源：
中文BERT系列模型：https://github.com/ymcui/Chinese-BERT-wwm
中文MacBERT：https://github.com/ymcui/MacBERT
中文ELECTRA：https://github.com/ymcui/Chinese-ELECTRA
中文XLNet：https://github.com/ymcui/Chinese-XLNet
知识蒸馏工具包TextBrewer：https://github.com/airaria/TextBrewer
HFL发布的更多资源：https://github.com/ymcui/HFL-Anthology

引用
若您认为本技术报告或资源对研究有所帮助，请在论文中引用以下文献：

主要引用文献：https://arxiv.org/abs/2004.13922
@inproceedings{cui-etal-2020-revisiting,
title = "Revisiting Pre-Trained Models for {C}hinese Natural Language Processing",
author = "Cui, Yiming and
Che, Wanxiang and
Liu, Ting and
Qin, Bing and
Wang, Shijin and
Hu, Guoping",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings",
month = nov,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/2020.findings-emnlp.58",
pages = "657--668",
}

次要引用文献：https://arxiv.org/abs/1906.08101
@article{chinese-bert-wwm,
title={Pre-Training with Whole Word Masking for Chinese BERT},
author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},
journal={arXiv preprint arXiv:1906.08101},
year = "2019"
}

搜集汇总

背景与挑战

背景概述

该数据集是一个中文预训练的BERT模型，采用全词掩码技术，适用于中文自然语言处理任务。由hfl团队开发，支持多种深度学习框架，并提供了相关的技术报告和开源项目链接。