Research of Methods for Evaluating the Efficiency of Transformer Architectures in Natural Language Text Generation Tasks

This dataset contains the experimental data from a study comparing 14 large language models (LLMs) in the task of generating structured text in Russian. The study evaluated both commercial (GigaChat, AliceAI, DeepSeek, GPT-5.1) and local (Qwen2.5-7B, Zephyr 7B, Mistral 7B, etc.) models.

Contents:
- 70 generated text responses (14 models × 5 business scenarios)
- Expert evaluations from 3 human experts
- LLM-based evaluations from 2 LLM arbiters (DeepSeek and GPT-5.1)
- Aggregated quality scores with standard deviations

Evaluation criteria (K1-K4):
- K1: Relevance to target business niche
- K2: Structural completeness
- K3: Linguistic quality and personalization
- K4: Instruction adherence

The data is provided in JSON format and can be used to reproduce the results presented in the accompanying article, as well as for further research in LLM evaluation and Russian language text generation.