bert-base-chinese.zip

bert-base-chinese Copied
like
295
Fill-Mask
PyTorch
TensorFlow
JAX
Safetensors
Transformers
Chinese
bert
AutoTrain Compatible

arxiv:
1810.04805
Model card
Files and versions
Community
9
Bert-base-chinese
Table of Contents
Model Details
Uses
Risks, Limitations and Biases
Training
Evaluation
How to Get Started With the Model
Model Details
Model Description
This model has been pre-trained for Chinese, training and random input masking has been applied independently to word pieces (as in the original BERT paper).

Developed by: HuggingFace team
Model Type: Fill-Mask
Language(s): Chinese
License: [More Information needed]
Parent Model: See the BERT base uncased model for more information about the BERT base model.
Model Sources
Paper: BERT
Uses
Direct Use
This model can be used for masked language modeling

Risks, Limitations and Biases
CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)).

Training
Training Procedure
type_vocab_size: 2
vocab_size: 21128
num_hidden_layers: 12
Training Data
[More Information Needed]

Evaluation
Results
[More Information Needed]

How to Get Started With the Model
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("bert-base-chinese")

model = AutoModelForMaskedLM.from_pretrained("bert-base-chinese"

搜集汇总

背景与挑战

背景概述

bert-base-chinese.zip 是一个由HuggingFace团队开发的中文预训练语言模型，属于填充掩码（Fill-Mask）类型，采用与原始BERT相同的词片段随机掩码训练方法。该模型包含12个隐藏层，词汇表大小为21128，专门用于中文掩码语言建模任务。