chinese-roberta-wwm-ext.zip

hfl/chinese-roberta-wwm-ext · Hugging Face
https://huggingface.co/hfl/chinese-roberta-wwm-ext
网页Chinese BERT with Whole Word Masking. For further accelerating Chinese natural language processing, we provide Chinese pre-trained BERT with Whole Word Masking. Pre …

hfl
/
chinese-roberta-wwm-ext Copied
like
114
Fill-Mask
PyTorch
TensorFlow
JAX
Transformers
Chinese
bert
AutoTrain Compatible

arxiv:
1906.08101

arxiv:
2004.13922
License:
apache-2.0
Model card
Files and versions
Please use 'Bert' related functions to load this model!
Chinese BERT with Whole Word Masking
For further accelerating Chinese natural language processing, we provide Chinese pre-trained BERT with Whole Word Masking.

Pre-Training with Whole Word Masking for Chinese BERT
Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu

This repository is developed based on：https://github.com/google-research/bert

You may also interested in,

Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm
Chinese MacBERT: https://github.com/ymcui/MacBERT
Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA
Chinese XLNet: https://github.com/ymcui/Chinese-XLNet
Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer
More resources by HFL: https://github.com/ymcui/HFL-Anthology

Citation
If you find the technical report or resource is useful, please cite the following technical report in your paper.

Primary: https://arxiv.org/abs/2004.13922
@inproceedings{cui-etal-2020-revisiting,
title = "Revisiting Pre-Trained Models for {C}hinese Natural Language Processing",
author = "Cui, Yiming and
Che, Wanxiang and
Liu, Ting and
Qin, Bing and
Wang, Shijin and
Hu, Guoping",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings",
month = nov,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/2020.findings-emnlp.58",
pages = "657--668",
}

Secondary: https://arxiv.org/abs/1906.08101
@article{chinese-bert-wwm,
title={Pre-Training with Whole Word Masking for Chinese BERT},
author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},
journal={arXiv preprint arXiv:1906.08101},
year={2019}
}

搜集汇总

背景与挑战

背景概述

该数据集是一个基于全词掩码技术的中文预训练BERT模型，专为加速中文自然语言处理任务而设计。模型源自Google Research的BERT实现，并提供了相关的学术论文引用和资源链接。