Decopy: Detect and correct with Pinyin for Chinese spelling correction

Chinese Spelling Correction (CSC) is a crucial task. Previous studies have
been affected by issues such as misleading error information,
over-reliance on high-frequency characters, and scarcity of training data.
This paper proposes a new CSC model, named Decopy, which employs an
advanced detection-correction framework and a novel error masking strategy
with pinyin features. Decopy can not only capture semantic information
(word embeddings) and positional information (position embeddings) of
words, but also recognize the phonetic features (pinyin embeddings) of
words. It can start directly from the phonetics of words, connect
similarities at the pinyin level, and make the most of useful phonetic
information, thereby reducing reliance on confusions and minimizing
misleading information. Additionally, to address the scarcity of training
data, we have constructed a new CSC dataset based on THUCNews and used it
for pre-training Decopy. This enables Decopy to have a more comprehensive
understanding of the input information, especially the additional pinyin
information. Experiments on SIGHAN15 and three domain-specific datasets,
namely LAW, Medical (Med), and Official Document Writing (Odw), show that
Decopy achieves significant improvements and outperforms the previous
state-of-the-art methods. Finally, we tested and analyzed several
high-performance LLMs on the CSC task, and fine-tuned ChatGLM3-6B for the
CSC task to further evaluate the capabilities of LLMs in this field.