无监督词库构建

利用基于信息熵和内部凝固度的无监督词库挖掘算法，结合各个场景下模型训练经验，能够实现在大量语料中建模分析出特有词汇，且没有长度的限制，除了2-4字的常用词，还能挖掘出场景化的短语。再与TF-IDF和LDA等算法结合后，能够进一步增加新词识别的可靠度。

Leveraging unsupervised vocabulary mining algorithms based on information entropy and internal cohesion, combined with model training experience across various scenarios, this method enables modeling and analysis of domain-specific vocabulary from large-scale corpora without length restrictions. Besides common 2- to 4-character words, it can also extract scenario-specific phrases. When further combined with algorithms such as TF-IDF and LDA, it can enhance the reliability of new word recognition.