Xing4.0-29B-Intent-oss

<h1 align="center">Xing4.0-29B-Intent-oss</h1> <p align="center"> 🦉 <a href="https://github.com/XingChen-AGI/Xing4.0-29B-Intent">GitHub</a> • 🤗 <a href="https://huggingface.co/XingChen-AGI/Xing4.0-29B-Intent">Model</a> • 📚 <a href="https://huggingface.co/datasets/XingChen-AGI/Xing4.0-29B-Intent-oss/tree/main">Dataset Files</a> </p> **Xing4.0-29B-Intent-oss** is the companion dataset for [Xing4.0-29B-Intent](https://huggingface.co/XingChen-AGI/Xing4.0-29B-Intent), a model for schema-guided intent recognition and structured information extraction. It is curated and released by the Xingchen General Artificial Intelligence Laboratory of China Telecom Artificial Intelligence Technology Co., Ltd. For complete schema definitions, conversion rules, and training/inference example assembly instructions, see the dataset documentation on GitHub: [English](https://github.com/XingChen-AGI/Xing4.0-29B-Intent/blob/main/data/README.md) | [简体中文](https://github.com/XingChen-AGI/Xing4.0-29B-Intent/blob/main/data/README_zh.md). ## Overview The dataset combines two sources: - **General**: 61 public information extraction and spoken language understanding datasets reconstructed into a unified schema-guided format. - **XingChen**: China Telecom's in-house intent and slot annotations for smart-device assistant scenarios. It supports supervised fine-tuning, schema-guided semantic understanding evaluation, and research on generalization across tasks, domains, and languages. Reproducing a specific evaluation result also requires matching data versions, evaluation scripts, and inference settings. ## Data Distribution | Statistic | Value | | --- | --- | | Total samples | 2,753,000 | | Reconstructed public datasets | 61 | | Domains | 19 | | Languages | 52, including Chinese and English | | Task types | NER, RE, EE, and intent recognition / slot filling | | Task | Samples | Share | Representative datasets | | --- | ---: | ---: | --- | | Named entity recognition (NER) | 823,000 | 29.9% | MSRA, OntoNotes, CoNLL2003, CMeEE, WeiboNER | | Relation extraction (RE) | 905,000 | 32.9% | DuIE, NYT, TACRED, BioRED, SemEval | | Event extraction (EE) | 56,000 | 2.0% | ACE05, RAMS, FewFC, MAVEN, DuEE | | Intent recognition / slot filling (SLU) | 969,000 | 35.2% | Snips, ATIS, MASSIVE | | **Total** | **2,753,000** | **100%** | — | ## Data Format Dataset files are JSON arrays, with one sample per element. Each sample includes its input conversation, a schema reference, and a serialized reference output. | Fields | Description | | --- | --- | | `id`, `turn` | Sample and conversation-turn identifiers | | `task`, `source`, `domain`, `language` | Task and provenance metadata | | `schema_path`, `schema` | Schema file path or an embedded schema array; a non-empty embedded array takes precedence | | `prompt_path`, `prompt` | Optional prompt file or embedded prompt template | | `dialogs` | Input turns containing `role` and `content` | | `output` | Reference answer stored as a JSON string | Illustrative sample: ```json { "id": "idx", "turn": "idx-0", "task": "Intent Recognition", "source": "XingChen", "domain": "Phone", "language": "Chinese", "schema_path": "./schema_description/schema.json", "schema": "", "prompt_path": "", "prompt": "", "dialogs": [ {"role": "user", "content": "明天的紫外线强度"} ], "output": "[{\"category\": \"查询天气\", \"args\": [{\"时间\": \"明天\", \"天气指数\": \"紫外线指数\"}]}]" } ``` The input schema defines candidate categories and their argument names and descriptions. The parsed output uses a common structure across all tasks: ```json [ { "category": "查询天气", "args": [{"时间": "明天", "天气指数": "紫外线指数"}] } ] ``` Here, `category` identifies the intent, entity type, relation type, or event type. Each object in `args` contains one set of extracted arguments. Multiple categories use separate top-level objects; multiple instances of a category use separate objects in `args`. Use `[]` when there are no arguments to fill. The `output` field must be parsed with `json.loads(output)` to obtain this structure. Schema and prompt paths are resolved relative to the corresponding data root, such as `General/test/` or `XingChen/`, rather than the individual JSON file's directory. ## Download and Use Browse and download the published files from the [Files and versions tab](https://huggingface.co/datasets/XingChen-AGI/Xing4.0-29B-Intent-oss/tree/main). Keep dataset files together with their associated schema and prompt files, preserving their relative paths. The GitHub repository provides [build_examples.py](https://github.com/XingChen-AGI/Xing4.0-29B-Intent/blob/main/data/build_examples.py) to assemble records into `messages` and `output` pairs. Follow the [setup and usage examples](https://github.com/XingChen-AGI/Xing4.0-29B-Intent/blob/main/data/README.md#usage-examples) to arrange the downloaded data and run the script from the model repository root. - **Training**: convert the assembled records to the format required by your training framework, for example by appending `output` as an assistant reply. - **Inference**: use `messages` as the request's message field and add parameters such as `model`. Never include reference answers in inference requests. - **Evaluation**: preserve task-specific single-label or multi-label constraints, construct each sample independently, and compare parsed output structures rather than raw JSON strings. ## License This dataset is released under the [Apache License 2.0](https://github.com/XingChen-AGI/Xing4.0-29B-Intent/blob/main/LICENSE). The original license terms of the public source datasets also apply; check their respective terms before commercial use. ## Citation If this project helps your research or applications, please consider citing it and the related work on its base model. ```bibtex @misc{li2026xinguspcrossfamilysemantic, title={Xing USP: A Cross-Family Semantic Contract for Information Extraction and Intent-Slot Parsing}, author={Zhongqiu Li and Boyang Wang and Ruiyu Fang and Shiquan Wang and Di Wu and Jiayi Liang and Feilong Zhang and Hongfeng Du and Xiaomeng Huang and Mengjiao Bao and Luwen Pu and Xuewei He and Yu Zhao and Chao Wang and Kaidong Yu and Bingkai Yang and Qingjun Zhu and Shaoyi Du and Shuangyong Song and Yongxiang Li}, year={2026}, eprint={2610.34343}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2610.34343}, } @misc{liu2025trainingreporttelechat3moe, title={Training Report of TeleChat3-MoE}, author={Xinzhang Liu and Chao Wang and Zhihao Yang and Zhuo Jiang and Xuncheng Zhao and Haoran Wang and Lei Li and Dongdong He and Luobin Liu and Kaizhe Yuan and Han Gao and Zihan Wang and Yitong Yao and Sishi Xiong and Wenmin Deng and Haowei He and Kaidong Yu and Yu Zhao and Ruiyu Fang and Yuhao Jiang and Yingyan Li and Xiaohui Hu and Xi Yu and Jingqi Li and Yanwei Liu and Qingli Li and Xinyu Shi and Junhao Niu and Chengnuo Huang and Yao Xiao and Ruiwen Wang and Fengkai Li and Luwen Pu and Kaipeng Jia and Fubei Yao and Yuyao Huang and Xuewei He and Zhuoru Jiang and Ruiting Song and Rui Xue and Qiyi Xie and Jie Zhang and Zilu Huang and Zhaoxi Zhang and Zhilong Lu and Yanhan Zhang and Yin Zhang and Yanlei Xue and Zhu Yuan and Teng Su and Xin Jiang and Shuangyong Song and Yongxiang Li and Xuelong Li}, year={2025}, eprint={2512.24157}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2512.24157}, } @misc{wang2025technicalreporttelechat2telechat25, title={Technical Report of TeleChat2, TeleChat2.5 and T1}, author={Zihan Wang and Xinzhang Liu and Yitong Yao and Chao Wang and Yu Zhao and Zhihao Yang and Wenmin Deng and Kaipeng Jia and Jiaxin Peng and Yuyao Huang and Sishi Xiong and Zhuo Jiang and Kaidong Yu and Xiaohui Hu and Fubei Yao and Ruiyu Fang and Zhuoru Jiang and Ruiting Song and Qiyi Xie and Rui Xue and Xuewei He and Yanlei Xue and Zhu Yuan and Zhaoxi Zhang and Zilu Huang and Shiquan Wang and Xin Wang and Hanming Wu and Mingyuan Wang and Xufeng Zhan and Yuhan Sun and Zhaohu Xing and Yuhao Jiang and Bingkai Yang and Shuangyong Song and Yongxiang Li and Zhongjiang He and Xuelong Li}, year={2025}, eprint={2507.18013}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2507.18013}, } @misc{he2024telechattechnicalreport, title={TeleChat Technical Report}, author={Zhongjiang He and Zihan Wang and Xinzhang Liu and Shixuan Liu and Yitong Yao and Yuyao Huang and Xuelong Li and Yongxiang Li and Zhonghao Che and Zhaoxi Zhang and Yan Wang and Xin Wang and Luwen Pu and Huinan Xu and Ruiyu Fang and Yu Zhao and Jie Zhang and Xiaomeng Huang and Zhilong Lu and Jiaxin Peng and Wenjun Zheng and Shiquan Wang and Bingkai Yang and Xuewei he and Zhuoru Jiang and Qiyi Xie and Yanhan Zhang and Zhongqiu Li and Lingling Shi and Weiwei Fu and Yin Zhang and Zilu Huang and Sishi Xiong and Yuxiang Zhang and Chao Wang and Shuangyong Song}, year={2024}, eprint={2401.03804}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2401.03804}, } ```