LDNet/Chat2Find-Corpus

Chat2Find Corpus是一个高质量的三语对话数据集，源自Chat2Find平台的真实世界交互。它包含约2.55亿个token，涵盖僧伽罗语（සිංහල）、泰米尔语（தமிழ்）和英语，并包括大量Singlish和Tanglish（音译和代码混合语言）实例。这使得它成为在低资源多语言环境下进行大型语言模型的持续预训练（CPT）和监督微调（SFT）的理想资源。数据集内容涉及信息查询、角色扮演以及南亚地区（主要是斯里兰卡和印度）的文化和地方知识，采用JSON Lines格式存储，包含约279,248条记录。关键特征包括非合成性（捕获自然语言模式和本地细微差别）、三语和代码混合（无缝切换语言），并作为Chat2Find未来模型套件的主要训练数据。

The Chat2Find Corpus is a high-quality trilingual conversational dataset derived from real-world interactions on the Chat2Find platform. It contains approximately 255 million tokens in Sinhala, Tamil, and English, including significant instances of Singlish and Tanglish (transliterated and code-mixed speech). This makes it an ideal resource for Continual Pre-training (CPT) and Supervised Fine-Tuning (SFT) of Large Language Models in low-resource and multilingual contexts. The dataset covers information seeking, role-playing, and cultural/local knowledge specific to the South Asian region (primarily Sri Lanka and India), with around 279,248 records in JSON Lines format. Key features include non-synthetic origin (capturing natural language patterns and local nuances), trilingual and code-mixed content, and serving as the primary training data for Chat2Finds upcoming model suite.