Phase-Technologies/hindi-translation-dataset

--- {} --- # Hindi Translation Dataset ## Dataset Description This dataset consists of English sentences paired with their corresponding Hindi translations. It serves as a foundational resource for linguistic research and machine translation tasks. ### Dataset Summary - **Total Instances:** 2618 - **Source Language:** English (en-US) - **Target Language:** Hindi (hi-IN) - **Format:** CSV UTF-8 - **Curation:** Synthetic generation with duplicate removal. ## Technical Specifications ### Model Used This dataset was generated using the **Sarvam-30B** model, specifically optimized for Indian languages and high-quality Hindi output. ### Data Fields - `english`: The source sentence in English. - `hindi`: The target translation in Devanagari script, optimized for natural flow. ### Generation Pipeline 1. **Extraction:** Initial prompts were processed by Sarvam-30B. 2. **Validation:** Responses were parsed for valid JSON structure. 3. **Post-Processing:** Merging of session outputs and deduplication using pandas. 4. **Final Export:** Normalized CSV generation. ## Intended Use - Training and fine-tuning neural machine translation models. - Evaluation of multilingual LLMs on translation accuracy. ## Ethical Considerations - **Bias:** Reflects biases inherent in the Sarvam-30B model. - **Privacy:** No PII was intentionally included. ## Licensing Licensed under Apache License 2.0.