five

somosnlp/es-inclusive-language-it

收藏
Hugging Face2024-05-14 更新2024-06-12 收录
下载链接:
https://hf-mirror.com/datasets/somosnlp/es-inclusive-language-it
下载链接
链接失效反馈
官方服务:
资源简介:
--- dataset_info: features: - name: pregunta dtype: string - name: respuesta dtype: string splits: - name: train num_bytes: 922215 num_examples: 4196 download_size: 404937 dataset_size: 922215 configs: - config_name: default data_files: - split: train path: data/train-* license: cc-by-nc-sa-4.0 language: - es --- # Dataset Card for Es-Inclusive-Language-it This dataset is the instructions version of the dataset [Es-Inclusive-Language](https://huggingface.co/datasets/somosnlp/es-inclusive-language). In this version there are two columns of data: `pregunta` y `respuesta`. The column `pregunta` contains the input as a prompt giving instructions to rewrite a piece of spanish text to the inclusive version of the same text. The expected output, i.e. the text in inclusive language, is given in the column `respuesta`. Find below the dataset card for es-inclusive-language dataset. # Dataset Card for Es-Inclusive-Language Languages are powerful tools to communicate ideas, but their use is not impartial. The selection of words carries inherent biases and reflects subjective perspectives. In some cases, language is wielded to enforce ideologies, marginalize certain groups, or promote specific political agendas. Spanish is not the exception to that. For instance, when we say “los alumnos” or “los ingenieros”, we are excluding women from those groups. Similarly, expressions such as “los gitanos” o “los musulmanes” perpetuate discrimination against these communities. In response to these linguistic challenges, this dataset offers neutral alternatives in accordance with official guidelines on inclusive language from various Spanish speaking countries. Its purpose is to provide grammatically correct and inclusive solutions to situations where our language choices might otherwise be exclusive. This dataset consists of pairs of texts with one entry featuring exclusive language and the other one its corresponding inclusive rewrite. All pairs are tagged with the origin (source) of the data and, in order to account for completeness of inclusive translation, also with labels for translation difficulty. This is a tool that contributes to the Sustainable Development Goals number five (_Achieve gender equality and empower all women and girls_) and ten (_Reduce inequality within and among countries_). ## Dataset Details ### Dataset Description - Curated by: Andrés Martínez Fernández-Salguero, Gaia Quintana Fleitas, Miguel López Pérez, Imanuel Rozenberg and Josué Sauca - Funded by: SomosNLP, HuggingFace, Argilla - Language(s) (NLP): Spanish (`es-ES`, `es-AR`, `es-MX`, `es-CR`, `es-CL`) - License: cc-by-nc-sa-4.0 ### Dataset Sources - Repository: https://github.com/Andresmfs/es-inclusive-language-dataset-creation - Video presentation: https://www.youtube.com/watch?v=7rrNGJIXEHU ## Uses ### Direct Use This dataset can be used to fine-tune LLMs to perform text2text generation tasks, specifically to train models that are able to rewrite Spanish texts using inclusive language. ### Out-of-Scope Use This dataset is specifically designed for translating Spanish texts to Spanish texts in inclusive language. Using the dataset for unrelated tasks is considered out of scope. This dataset can not be used with commercial purposes, it is intended for research or educational purposes only. ## Dataset Structure This dataset consists of pairs of texts with one entry featuring exclusive language and the other one its corresponding inclusive rewrite. All pairs are tagged with the origin (source) of the data and, in order to account for completeness of inclusive translation, also with labels for translation difficulty. The dataset has a total of 4196 rows and contains the following columns: - `gender_exclusive` (input): text in non inclusive language - `gender_inclusive` (target): text in inclusive language - `difficulty`: translation difficulty category. Descriptions and distribution below. - `origin`: data source. Descriptions and distribution below. ### Difficulty tags descriptions We used different labels, most of them gender related, and can be describe like this: | Tag | Description | Example | |-----------------------|---------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | no_cambia | No changes are needed | "Los alumnos Carlos y Manuel son muy problemáticos" cannot be translated as "El alumnado Carlos y Manuel son muy problemáticos” | | plural_complejo | Plural words for which there is not a neutral term. There are different formulas that will vary according to the context. | "Los agricultores" -> "La comunidad agrícola", "Los y las agricultoras". “Las limpiadoras” -> “El equipo de limpieza”. More: "El grupo de...", "El sector de...", "El personal de..." | | plural_neutro | Change the plural for a generic noun. | "Los alumnos" -> "El alumnado" | | culturas | People and cultures | "Los andaluces" -> "El pueblo andaluz", "La comunidad andaluza" | | feminizar_profesiones | Professions with androcentric feminine forms | “La médico” -> "La médica". “La técnico de sonido” -> "La técnica de sonido" | | nombres_propios | Proper names | "Los alumnos Carlos y Manuel son muy problemáticos" cannot be translated as "El alumnado es muy problemático | | persona_generica | Reference to a generic person | "Nota al lector" -> "Nota a quien lee", "Nota a la persona que lee" | | dificultades_variadas | Mix of difficulties (to tag big chunks of diverse data) | | | plurales | Mix of neutral and complex plurals | | | falsa_concordancia | Androcentric agreement errors | "Estas siete parejas van a dar lo mejor de sí mismos" -> "Estas siete parejas van a dar lo mejor de sí mismas." | | omision | The subject or some pronouns are omitted, or the phrase is restructured with verboids. | "los participantes mantendrán un debate" -> "habrá un debate", "Si los científicos trabajan adecuadamente" -> "Trabajando adecuadamente, "los estudiantes" -> "estudiantes | | terminologia | Correction of terms with ableist, racist, or other types of discrimination bias. | | | parafrasis | Avoid words with generic connotations by reformulating the phrase | | | otros | Difficulties that don’t fit in the other labels | | <img src="https://cdn-uploads.huggingface.co/production/uploads/65d9bf5b41325e422e9fa704/BuwIZc3bOS0Seruz-zPce.png" alt="difficulties_distribution.JPG" width="1500"/> ### Origin tags descriptions Data quality can depend on their origin, so data are tagged with origin labels according to this table: | Tag | Description | Link to origin | |---------------------------|----------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------| | neutral_es | Curated and refined version of neutral-es dataset | https://huggingface.co/datasets/hackathon-pln-es/neutral-es | | GPT-3.5_fewshot | Chat GPT-3.5 generated with few shot technique | | | GPT-3.5_CaDi <sup>*</sup> | Data created based on the dataset used for developing CaDi project<sup>*</sup> | https://lenguaje-incluyente.ibero.mx/ | | GPT-3.5_fs_multiplication | Data multiplicated from GPT-3.5_fewshot using GPT-3.5 | | | guia_CCGG | Examples from Spanish General Courts language inclusive Guide | https://www.congreso.es/docu/igualdad/Recomendaciones_uso_no_sexista_lenguajeCC.GG..pdf | | guia_TAI | Examples from Trenes Argentinos' Guide to the use of inclusive language | https://www.argentina.gob.ar/sites/default/files/guia_para_uso_de_lenguaje_inclusivo_v1.pdf | | guia_CONICET | Examples from Guide to inclusive, non-sexist language (CONICET) | https://cenpat.conicet.gov.ar/wp-content/uploads/sites/91/2020/08/Guia-lenguaje-inclusivo-no-sexista-CENPAT_final-1.pdf | | guia_INAES | Examples of Guidelines for Inclusive Language Recommendations (INAES) | https://www.argentina.gob.ar/sites/default/files/2020/10/lenguaje_inclusivo_inaes_2021.pdf | | guia_CHRYSALLIS | Examples from Practical Guide to Inclusive Language (Chrysallis) | https://www.lgbtqiahealtheducation.org/wp-content/uploads/2020/04/Guia-practica-de-lenguaje-inclusivo-Chrysallis.pdf | | guia_ONU | Examples from Guidance for the use of gender-inclusive language (UN) | https://www.unwomen.org/sites/default/files/Headquarters/Attachments/Sections/Library/Gender-inclusive%20language/Guidelines-on-gender-inclusive-language-es.pdf | | guia_MX | Examples from Manual for the use of inclusive and gender-sensitive language (MX) | https://www.gob.mx/cms/uploads/attachment/file/183695/Manual_Lenguaje_Incluyente_con_perspectiva_de_g_nero-octubre-2016.pdf | | guia_CL | Examples from Gender Inclusive Language Guide of the Government of Chile | https://www.cultura.gob.cl/wp-content/uploads/2023/01/guia-de-lenguaje-inclusivo-de-genero.pdf | | guia_IEM | Examples from Uso del Lenguaje Inclusivo de Género | https://secretariagenero.poder-judicial.go.cr/images/Documentos/LenguajeInclusivo/Documentos/Uso-de-lenguaje-inclusivo-de-Genero-IEM-UNA.pdf | | human_combinatory | Combinatorics of text fragments generated with GPT3.5 | | | GPT-4_human | Chat GPT-4 generated and human revised | | | human | Human created data based on information from inclusive language guidelines | | <sup>*</sup>©Universidad Iberoamericana, A.C. , Ciudad de México, México <sup>*</sup>©Capitolina Díaz Martínez, Elvia María Guadalupe González del Pliego Dorantes, Marco Antonio López Hernández, Alberto López Medina, Héctor Celallos Avalos, Laura Mejía Hernández <img src="https://cdn-uploads.huggingface.co/production/uploads/65d9bf5b41325e422e9fa704/Y5BCoOYej06rpTK6kfm2k.png" alt="origins_distribution.JPG" width="600"/> ## Dataset Creation ### Curation Rationale The selection of words carries inherent biases and reflects subjective perspectives. In some cases, language is wielded to enforce ideologies, marginalize certain groups, or promote specific political agendas. In response to these linguistic challenges, this dataset offers neutral alternatives in accordance with official guidelines on inclusive language from various Spanish-speaking countries. Its purpose is to provide grammatically correct and inclusive solutions to situations where our language choices might otherwise be exclusive. This dataset has been created to train the model ["Traductor Inclusivo"](https://huggingface.co/somosnlp/es-inclusivo-translator) which aims to tackle this problem. ## Source Data ### Data Collection and Processing The data used for training the model has been sourced from various origins. The first and more important source was a curated and refined version of [es_neutral](https://huggingface.co/datasets/hackathon-pln-es/neutral-es) (link to the dataset curation notebook [here](https://github.com/Andresmfs/Neutral_es-Text-data-cleaning) ) In addition, we manually generated data based on Official Guidelines from different Spanish speaking countries. Finally, we augmented this data by experimenting with various prompts and Few-Shot learning techniques. We needed to be as explicit as possible, otherwise we wouldn’t get good results. You can see some examples below. Example 1: <img src="https://cdn-uploads.huggingface.co/production/uploads/65d9bf5b41325e422e9fa704/48ipmlxyEHgkNLxLvWnUp.jpeg" alt="foto1.JPG" width="600"/> Example 2: <img src="https://cdn-uploads.huggingface.co/production/uploads/65d9bf5b41325e422e9fa704/rwkDR3FrFyLLOMmofCMFI.jpeg" alt="foto2.JPG" width="600"/> Example 3: <img src="https://cdn-uploads.huggingface.co/production/uploads/65d9bf5b41325e422e9fa704/rHCV4UwitTbmQD0r2WS6V.jpeg" alt="foto3.JPG" width="700"/> We tried to be as inclusive as possible, paying close attention to the classification of difficulties that one could encounter in texts like these. Moreover, we took care to incorporate numerous counterexamples, recognizing that there are instances where neutrality is not required in a sentence. For instance, “Las arquitectas María Nuñez y Rosa Loria presentaron el proyecto” should not be rewritten as “El equipo de arquitectura María Nuñez y Rosa Loria presentó el proyecto”. It’s important to highlight that the Traductor Inclusivo not only promotes gender inclusivity but also addresses other forms of discrimination such as ableism, racism, xenophobia, and more. You can access scripts used during data creation [here](https://github.com/Andresmfs/es-inclusive-language-dataset-creation) ### Who are the source data producers? Data have been produced by different producers: - Official Spanish Inclusive Language Guidelines: - - [Recomendaciones para un uso no sexista del lenguaje en la Administracio n parlamentaria (España)](https://www.congreso.es/docu/igualdad/Recomendaciones_uso_no_sexista_lenguajeCC.GG..pdf) - [Guía para uso de lenguaje inclusivo (Argentina)](https://www.argentina.gob.ar/sites/default/files/guia_para_uso_de_lenguaje_inclusivo_v1.pdf) - [Guía de lenguaje inclusivo no sexista CCT CONICET-CENPAT (Argentina)](https://cenpat.conicet.gov.ar/wp-content/uploads/sites/91/2020/08/Guia-lenguaje-inclusivo-no-sexista-CENPAT_final-1.pdf) - [Guía de recomendaciones para lenguaje inclusivo (Argentina)](https://www.argentina.gob.ar/sites/default/files/2020/10/lenguaje_inclusivo_inaes_2021.pdf) - [Guía práctica de lenguaje inclusivo (España)](https://www.lgbtqiahealtheducation.org/wp-content/uploads/2020/04/Guia-practica-de-lenguaje-inclusivo-Chrysallis.pdf) - [Guía para el uso de un lenguaje inclusivo al género (ONU)](https://www.unwomen.org/sites/default/files/Headquarters/Attachments/Sections/Library/Gender-inclusive%20language/Guidelines-on-gender-inclusive-language-es.pdf) - [Manual para el uso de un lenguaje incluyente y con perspectiva de género (México)](https://www.gob.mx/cms/uploads/attachment/file/183695/Manual_Lenguaje_Incluyente_con_perspectiva_de_g_nero-octubre-2016.pdf) - [Guía de lenguaje inclusivo de Género (Chile)](https://www.cultura.gob.cl/wp-content/uploads/2023/01/guia-de-lenguaje-inclusivo-de-genero.pdf) - [Uso del Lenguaje Inclusivo de Género, IEM (Costa Rica)](https://secretariagenero.poder-judicial.go.cr/images/Documentos/LenguajeInclusivo/Documentos/Uso-de-lenguaje-inclusivo-de-Genero-IEM-UNA.pdf) - [Uso no sexista de la lengua, UOC (España)](https://www.uoc.edu/portal/es/servei-linguistic/redaccio/tractament-generes/index.html) - [neutral-es dataset](https://huggingface.co/datasets/hackathon-pln-es/neutral-es) - ChatGPT-3.5 based examples and structures from official guidelines and dataset used for developing [CaDi project](https://lenguaje-incluyente.ibero.mx/) - ChatGPT-4 ## Annotations ### Annotation process Most data were created using ChatGPT-3.5 based on examples and structures form Official Inclusive Language Guidelines and then supervised and corrected by the annotators always following the indications from the Official Guidelines. You can find more information about this process in the Data Collection and Processing section. Other data were created using pairs of words taken from CaDi project. Data taken from neutral-es dataset were not modified beyond correcting spelling or grammar mistakes. Regarding difficulty labels, we stablished different groups of data using the information from Official Spanish Inclusive Language Guidelines. We created a list of difficulties that could cover all kind of examples. The difficulties list can be found above together with their descriptions. Data from Official Guidelines were manually matched to the different difficulties according to the descriptions provided in the list and then we used those examples as a base to create more data using ChatGPT difficulty by difficulty, so new data would belong to a specific difficulty and therefore tagged with that difficulty. ### Who are the annotators? - [Gaia Quintana Fleitas](https://huggingface.co/gaiaquintana) - [Miguel López Pérez](https://huggingface.co/Wizmik12) - [Andrés Martínez Fernández-Salguero](https://huggingface.co/Andresmfs) ### Personal and Sensitive Information This dataset does not contain any personal or sensitive information. ## Bias, Risks, and Limitations - Data are based on the Spanish Inclusive Language Guidelines mentioned in the source section and they may inherit any bias comming from these guidelines and institutions behind them. These are official and updated guidelines that should not contain strong biases. Data from neutral-es are based as well on Official Spanish Inclusive Language Guidelines - Data based on CaDi may inherit biases from the institution behind it. It has been created by a group of specialists in the field from Universidad Iberoamericana - Dataset contains a significantly large amount of data related to workspace compared to other areas. - This dataset covers only difficulties that appear on the difficulties list - Dataset is composed mostly by sentences where the terms to be modified are at the beginning of the sentence. - Dataset does not contain long-complex texts. ### Recommendations Users should be made aware of the risks, biases and limitations of the dataset. ## License Creative Commons (cc-by-nc-sa-4.0) This kind of licensed is inherited from CaDi project dataset. ## Citation **BibTeX:** ``` @software{GAMIJ2024es-inclusive-language, author = {Gaia Quintana Fleitas, Andrés Martínez Fernández-Salguero, Miguel López Pérez, Imanuel Rozenberg, Josué Sauca}, title = {Traductor Inclusivo}, month = April, year = 2024, url = {https://huggingface.co/datasets/somosnlp/es-inclusive-language} } ``` ## More Information This project was developed during the [Hackathon #Somos600M](https://somosnlp.org/hackathon) organized by SomosNLP. The dataset was created using distilabel by Argilla and endpoints sponsored by HuggingFace. **Team**: - [**Gaia Quintana Fleitas**](https://huggingface.co/gaiaquintana) - [**Andrés Martínez Fernández-Salguero**](https://huggingface.co/Andresmfs) - **Imanuel Rozenberg** - [**Miguel López Pérez**](https://huggingface.co/Wizmik12) - **Josué Sauca** ## Contact - [**Gaia Quintana Fleitas**](https://www.linkedin.com/in/gaiaquintana/) (gaiaquintana11@gmail.com) - [**Andrés Martínez Fernández-Salguero**](www.linkedin.com/in/andrés-martínez-fernández-salguero-725674214) (andresmfs@gmail.com) - [**Miguel López Pérez**](https://www.linkedin.com/in/miguel-lopez-perezz/)
提供机构:
somosnlp
原始信息汇总

数据集概述

数据集名称

  • Es-Inclusive-Language-it

数据集内容

  • 特征
    • pregunta:输入提示,指导如何将西班牙语文本重写为包容性版本。
    • respuesta:预期的输出,即包容性语言的文本。

数据集结构

  • 列信息
    • gender_exclusive (输入): 非包容性语言的文本。
    • gender_inclusive (目标): 包容性语言的文本。
    • difficulty: 翻译难度类别。
    • origin: 数据来源。

数据集大小

  • 训练集
    • num_examples: 4196
    • num_bytes: 922215
    • download_size: 404937
    • dataset_size: 922215

许可证

  • cc-by-nc-sa-4.0

语言

  • 西班牙语 (es-ES, es-AR, es-MX, es-CR, es-CL)

数据来源

  • Repository: https://github.com/Andresmfs/es-inclusive-language-dataset-creation
  • Video presentation: https://www.youtube.com/watch?v=7rrNGJIXEHU

使用场景

  • 直接使用:用于微调大型语言模型(LLMs)以执行文本到文本生成任务,特别是训练能够使用包容性语言重写西班牙语文本的模型。
  • 超出范围的使用:该数据集专门设计用于将西班牙语文本翻译为包容性语言的西班牙语文本。使用该数据集进行无关任务被视为超出范围。该数据集不可用于商业目的,仅限于研究和教育用途。

数据集创建

  • 数据收集和处理
    • 主要来源:es_neutral
    • 手动生成数据,基于官方指南。
    • 通过实验使用各种提示和少量学习技术增强数据。

源数据生产者

  • 官方西班牙包容性语言指南。
  • neutral-es 数据集
  • ChatGPT-3.5 基于官方指南的示例和结构。
搜集汇总
数据集介绍
main_image_url
构建方式
在自然语言处理领域,语言偏见问题日益受到关注,而西班牙语中存在的性别歧视与非包容性表达尤为突出。为此,somosnlp/es-inclusive-language-it数据集应运而生,旨在通过指令微调形式推动西班牙语的包容性改写。该数据集基于原始Es-Inclusive-Language语料库进行重构,将原始的双列结构(性别排他性与包容性文本对)转化为指令型格式,新增了'pregunta'(输入指令)和'respuesta'(预期输出)两列。数据来源涵盖多个维度:首先是对neutral-es数据集的精炼与整理,其次依据西班牙语国家官方包容性语言指南(如西班牙议会、阿根廷政府、联合国等机构发布的指导文件)人工生成样例,最后借助GPT-3.5与GPT-4模型通过少样本学习与数据增强技术进行扩展。所有数据均标注了难度标签(如复数中性化、职业女性化等)与来源标签,确保构建过程的透明性与可追溯性。
使用方法
该数据集专为文本到文本生成任务设计,主要用于微调大语言模型以实现西班牙语的包容性改写。用户可通过HuggingFace平台直接加载数据集,使用默认配置训练集进行模型训练,其中'pregunta'列作为输入提示(如将排他性文本改写为包容性版本的指令),'respuesta'列作为目标输出。推荐结合Transformers库的Seq2Seq训练框架,例如基于T5或BART架构的模型进行微调。由于数据集已提供明确的指令格式,用户可将其直接应用于对话式改写任务,或通过调整提示模板适配不同应用场景(如正式文档、教育材料或公共宣传文本的自动包容性转换)。需注意,该数据集不适用于商业用途,且应避免将其用于无关任务(如情感分析或机器翻译),以确保使用场景的伦理一致性。
背景与挑战
背景概述
语言作为思想交流的载体,其词汇选择往往隐含着深层的社会偏见与权力结构。西班牙语中,诸如“los alumnos”或“los ingenieros”等表达天然排除了女性群体,而“los gitanos”或“los musulmanes”等称谓则可能固化对特定社群的歧视。为应对这一语言不平等现象,由Andrés Martínez Fernández-Salguero、Gaia Quintana Fleitas等研究人员于2023年创建的es-inclusive-language-it数据集应运而生。该数据集受SomosNLP、HuggingFace与Argilla资助,旨在提供符合多个西语国家官方指南的语法正确且包容性强的语言替代方案,直接服务于联合国可持续发展目标第五项(性别平等)与第十项(减少不平等)。其核心研究问题聚焦于如何通过指令微调大型语言模型,使其能够将排他性西语文本精准改写为包容性版本,从而推动自然语言处理技术在语言公平领域的应用。
当前挑战
该数据集面临的核心挑战在于语言包容性转换本身的复杂性与多维度性。首先,从领域问题看,西班牙语中的性别排他性表达形式多样,包括复数名词的替换(如“los agricultores”需变为“la comunidad agrícola”)、职业名称的女性化(如“la médico”需修正为“la médica”)、以及针对种族、残障等歧视性术语的纠正,这些均要求模型具备细腻的语义理解与上下文感知能力。其次,构建过程中,数据来源涵盖官方指南、人工创作与GPT-3.5/4生成,但不同来源的语言规范与质量参差不齐,需通过难度标签(如“plural_complejo”、“falsa_concordancia”)和来源标签(如“guia_ONU”、“GPT-3.5_fewshot”)进行精细标注与质量控制。此外,如何确保改写后的文本在保持语法正确的同时,不因过度中性化而损失原意(如专有名词“Carlos y Manuel”不可被抽象化),成为数据验证中的关键难点。最后,模型需避免将包容性改写机械应用于所有场景,需识别无需改写的反例(如特定女性人名句),这对训练数据的平衡性与多样性提出了严苛要求。
常用场景
经典使用场景
在自然语言处理与计算社会语言学的交叉领域中,somosnlp/es-inclusive-language-it数据集被广泛用于微调大语言模型,以执行西班牙语文本的包容性改写任务。该数据集包含近4200对文本,每对由一段原始排他性表述及其对应的包容性改写构成,覆盖了从性别中性化、职业名称女性化到消除种族歧视性术语等多种语言转换场景。研究者通常利用其指令版格式,将'pregunta'列中的提示作为模型输入,'respuesta'列中的包容性文本作为目标输出,从而训练模型掌握西班牙语包容性表达的语法规则与语境适应能力,为自动化文本包容性转换提供了标准化的训练基准。
解决学术问题
该数据集直击社会语言学中语言偏见与歧视的量化研究难题,系统性地解决了如何通过计算模型识别并修正西班牙语文本中隐含的性别、种族及能力歧视性表达。它依据多个西语国家的官方包容性语言指南,构建了涵盖十二种难度标签的平行语料,使学术界能够深入分析排他性语言的结构模式与转换策略。这一工作不仅为计算语言学研究提供了实证基础,还推动了自然语言处理技术向联合国可持续发展目标第五项(性别平等)与第十项(减少不平等)的实质性迈进,弥合了语言学理论与算法实践之间的鸿沟。
实际应用
在实际应用中,该数据集赋能了智能写作辅助系统与内容审核工具的开发,使政府公文、教育材料及媒体内容能够自动转换为符合包容性语言规范的形式。例如,阿根廷与墨西哥的官方机构已将其用于培训自动化校对系统,以在公共文档中消除'los alumnos'等排他性措辞。此外,基于该数据集训练的'Traductor Inclusivo'模型已在社交媒体监测平台中部署,实时检测并建议替换歧视性术语,显著提升了数字空间的语言平等性。其跨方言特性(涵盖西班牙、阿根廷、墨西哥等六种变体)更使其成为跨国企业全球化内容本地化的关键工具。
数据集最近研究
最新研究方向
在自然语言处理领域,包容性语言研究正成为消解文本中性别、种族及能力歧视的关键前沿方向。somosnlp/es-inclusive-language-it数据集作为西班牙语包容性改写指令集的代表,其最新研究聚焦于利用大语言模型实现从排他性表达向中立、包容性文本的精准转换。该数据集融合了来自西班牙语国家官方指南、GPT-3.5增强生成及人工校验的多源语料,覆盖复数中性化、职业名词女性化、歧视性术语修正等十余类语言转换难题。这一工作与联合国可持续发展目标5(性别平等)和10(减少不平等)紧密呼应,为构建公平、多元的数字语言环境提供了可复用的技术基础。当前,基于该数据集的“Traductor Inclusivo”模型已在HuggingFace开源,推动了西班牙语包容性语言自动处理从理论规范向实用工具的跨越。
以上内容由遇见数据集搜集并总结生成
二维码
社区交流群
二维码
科研交流群
商业服务