Ethosoft/nedo-turkish-sft-mixtures

---
language:
- tr
task_categories:
- text-generation
- question-answering
- summarization
- text2text-generation
pretty_name: NEDO Turkish SFT Mixtures
size_categories:
- 10K<n<100K
---

# NEDO Turkish SFT Mixtures

This dataset repository contains Turkish supervised fine-tuning datasets prepared for the NEDO Turkish SLM project.

The datasets were built to fine-tune NEDOQwen-style Turkish decoder-only language models after base pretraining on the NEDO Turkish 65K tokenized web corpus.

Main related pretraining dataset:

Ethosoft/nedo-turkish-65k-tokenized-60b

## Summary

This repository contains two main SFT mixtures:

| File | Examples | Recommended? | Description |
|---|---:|---|---|
| data/tr_sft_clean_20k.jsonl | 20,000 | Yes | Cleaner Turkish instruction subset used for better SFT behavior |
| data/tr_sft_hf_mix.jsonl | 104,078 | Experimental | Larger but noisier Turkish instruction mixture |

A small smoke-test dataset was used internally during development but is not the main release.

## Files

data/tr_sft_clean_20k.jsonl
data/tr_sft_hf_mix.jsonl
metadata/sft_dataset_info.json
scripts/46_build_tr_sft_mix.py
scripts/50_build_clean_sft.py

## Recommended starting point

For most users, start with:

data/tr_sft_clean_20k.jsonl

This file is smaller, cleaner, and was more useful in early NEDO SFT experiments.

The larger file:

data/tr_sft_hf_mix.jsonl

is broader but noisier. It may be useful for experimentation, filtering, or continued dataset curation, but should not be treated as a high-quality benchmark.

## Dataset variants

### tr_sft_clean_20k.jsonl

- Examples: 20,000
- Format: JSONL
- Fields: instruction, input, output, source
- Status: recommended clean subset

Upstream sources used during construction:

- SoAp9035/turkish_instructions
- Dbmaxwell/turkish-finance-instruction-dataset

This dataset was built after filtering out noisier translation-style data from the broader SFT mixture. In internal NEDO experiments, it produced better basic QA and instruction-format behavior than the larger HF-mix dataset.

### tr_sft_hf_mix.jsonl

- Examples: 104,078
- Format: JSONL
- Fields: instruction, input, output, source
- Status: broader experimental mixture

Upstream sources used during construction:

- NovusResearch/turkish_instructions
- SoAp9035/turkish_instructions
- Dbmaxwell/turkish-finance-instruction-dataset

This mixture is more diverse, but internal experiments showed that it can produce weaker structured-prompt behavior without further cleaning.

## Format

Each line is a JSON object:

{
"instruction": "...",
"input": "",
"output": "...",
"source": "..."
}

The expected fields are:

- instruction: user instruction or task
- input: optional extra context; can be empty
- output: target assistant response
- source: upstream dataset/source identifier

## Prompt format used in NEDO SFT

When input is empty, examples were formatted as:

Kullanıcı talimatı:
{instruction}

Asistan cevabı:
{output}

When input is non-empty, examples were formatted as:

Kullanıcı talimatı:
{instruction}

Ek bilgi:
{input}

Asistan cevabı:
{output}

During training, the prompt portion was masked with `-100` labels and loss was computed only on assistant answer tokens.

## Example loading

import json

path = "data/tr_sft_clean_20k.jsonl"

with open(path, "r", encoding="utf-8") as f:
for line in f:
example = json.loads(line)
print(example["instruction"])
print(example["output"])
break

## Relationship to NEDOQwen

These SFT mixtures were prepared for NEDOQwen 0.8B Turkish SLM experiments.

Related resources:

- Pretraining dataset: Ethosoft/nedo-turkish-65k-tokenized-60b
- Base model: planned NEDOQwen 0.8B base release
- Clean SFT model: planned NEDOQwen 0.8B clean-SFT release

## Internal experiment notes

In the NEDO project handoff:

- HF-mix SFT completed technically but produced weak structured-prompt behavior.
- Clean 20K SFT performed better than HF-mix.
- Clean SFT could answer simple factual questions such as “Fransa'nın başkenti nedir?”
- Domain-specific SFT is still needed for strong SLM/LLM/agent-domain behavior.

## Data quality notes

This repository should be treated as an SFT research artifact.

Known characteristics:

- Turkish instruction-following format
- mixed task types: rewriting, QA, explanation, classification, summarization, formatting, finance-oriented instructions
- some examples may contain translation artifacts
- some examples may contain factual errors or awkward wording
- broader HF-mix file is noisier than the clean 20K subset

## License and provenance

This repository is a mixture/derivative of upstream Turkish instruction datasets.

Known upstream sources include:

- NovusResearch/turkish_instructions
- SoAp9035/turkish_instructions
- Dbmaxwell/turkish-finance-instruction-dataset

Downstream users are responsible for checking and complying with upstream dataset licenses and terms, especially for commercial redistribution or production use.

## Intended use

This dataset is intended for:

- Turkish supervised fine-tuning
- small language model alignment experiments
- instruction-following research
- Turkish SLM evaluation and ablation studies
- reproducibility of the NEDO Turkish SLM SFT pipeline

## Out-of-scope use

This dataset is not intended for:

- use as a factual benchmark
- high-stakes deployment without filtering and evaluation
- commercial redistribution without checking upstream licenses
- treating the broader HF-mix file as fully cleaned data

## Citation and attribution

If you use this dataset, please attribute the NEDO Turkish SLM project and the upstream datasets.

Suggested attribution:

NEDO Turkish SFT Mixtures.
Built for Turkish supervised fine-tuning experiments in the NEDO Turkish SLM project.
Includes cleaned and broader mixtures derived from Turkish instruction datasets.

Please also cite or link the relevant upstream datasets used in your experiment:

NovusResearch/turkish_instructions
SoAp9035/turkish_instructions
Dbmaxwell/turkish-finance-instruction-dataset

## Contact

For questions, issues, or removal requests, please open a discussion on this dataset repository.