davanstrien/test-transformers-cb-smoltalk2-regen

--- tags: - generated - transformers - continuous-batching - uv-script --- # Generated Responses Dataset This dataset contains generated responses for prompts from [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2). ## Generation Details - **Source Dataset**: [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) - **Input Column**: `messages` (chat messages) - **Model**: [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) - **Backend**: transformers continuous batching - **Number of Examples**: 10 - **Generation Date**: 2026-03-25T13:13:34.055879 ### Generation Parameters - **Temperature**: 0.7 - **Top P**: 0.8 - **Top K**: 20 - **Max New Tokens**: 256 - **Max Batch Tokens**: 512 - **Repetition Penalty**: 1.0 ### Hardware Configuration - **GPUs**: 1 - **Attention Implementation**: paged|sdpa ## Dataset Structure The dataset contains all columns from the source dataset plus: - `response`: The generated response from the model ## Generation Script Generated using the transformers continuous batching script from [uv-scripts/transformers-inference](https://huggingface.co/datasets/uv-scripts/transformers-inference). To reproduce this generation: ```bash uv run https://huggingface.co/datasets/uv-scripts/transformers-inference/raw/main/generate-responses.py \ HuggingFaceTB/smoltalk2 \ <output-dataset> \ --model-id Qwen/Qwen3-4B-Instruct-2507 \ --messages-column messages \ --temperature 0.7 \ --top-p 0.8 \ --top-k 20 \ --max-tokens 256 ```