Albanian corpus used for pertaining large language models

The corpus used for pertaining large language models is an SQLite (v3) database with the following tables: corp_src: the sources of the Albanian text corp_doc: the corpus source (names) and source files doc: joins from sentences to corpus document source (`corp_doc`) sent: the Albanian sentences with tokenization and token length This query shows how to get the corpus sources and constituent counts: select cs.id as name, cs.url, count(*) as count from corp_src as cs, corp_doc as cd, doc as d, sent as s where cd.name = cs.id and cd.doc_id = d.rowid and cd.doc_id = s.doc_id group by cs.id;用于预训练大语言模型（Large Language Model，LLM）的语料库为SQLite（版本3）数据库，包含以下数据表： 表corp_src：阿尔巴尼亚语文本的来源 表corp_doc：语料库来源（名称）及源文件信息 表doc：连接句子与语料库文档来源（`corp_doc`）的关联表 表sent：经过分词处理并标注Token长度的阿尔巴尼亚语句子 以下查询语句可用于获取语料库来源及其构成条目数： select cs.id as name, cs.url, count(*) as count from corp_src as cs, corp_doc as cd, doc as d, sent as s where cd.name = cs.id and cd.doc_id = d.rowid and cd.doc_id = s.doc_id group by cs.id;