SINAI/ALIA-es-cultural-heritage-triplets
收藏资源简介：
--- language: - es license: cc-by-sa-4.0 size_categories: - 1M<n<10M tags: - cultural - heritage - humanities - retrieval - hard-negatives - embeddings - spanish task_categories: - text-generation - question-answering configs: - config_name: hard-negatives data_files: - split: train path: "ALIA-es-cultural-hard-negatives-train.parquet" - config_name: evaluation data_files: - split: test path: "ALIA-es-cultural-hard-negatives-eval.parquet" --- # Dataset Introduction The dataset **ALIA Spanish Cultural and Heritage Hard Negatives Corpus** contains **hard negatives for dense retrieval training** generated from <query, passage> pairs contained in [SINAI/ALIA-es-cultural-heritage-pairs](https://huggingface.co/datasets/SINAI/ALIA-es-cultural-heritage-pairs).\ The dataset was created as part of the **ALIA project** to improve the training of embedding models and dense retrievers specialized in Spanish cultural heritage language. Hard negatives are passages that are **semantically similar to a query but not correct answers**, making them particularly useful for training robust retrieval systems. Each query is paired with **multiple hard negatives** generated automatically using embedding similarity mining. This structure enables training retrieval models using contrastive or ranking losses that benefit from **multiple negatives per query**. The dataset was generated automatically using a **SentenceTransformers-based hard negative mining pipeline** with FAISS similarity search and the **Qwen3-Embedding-0.6B** embedding model. ------------------------------------------------------------------------ # Dataset Details ## Dataset Description The **ALIA Spanish Cultural and Heritage Hard Negatives Corpus** provides challenging negative passages for training dense retrieval models in the Spanish cultural heritage domain. In dense retrieval systems, hard negatives are passages that are **semantically close to the query but do not contain the correct answer**. Training with such negatives helps models learn fine‑grained semantic distinctions and improves ranking performance. This dataset is derived from cultural heritage query--passage pairs used within the ALIA project and automatically augmented with hard negatives using embedding similarity mining. ------------------------------------------------------------------------ # Dataset Structure The dataset has **two distinct configurations**, each with a different purpose and structure: ## Config 1: Hard Negatives (Training) **Purpose:** Multi-negative training for dense retrieval models. ### Data Instances Each training instance contains: - a **query** (in conversational format) - **one positive passage** (correct answer) - **multiple hard negative passages** mined from the corpus (typically 5 per query) - the **train phase** and **difficulty** label Example: ```json { "train_phase": "phase_1", "difficulty": "university", "messages": [ { "role": "user", "content": "¿Qué funciones sociales y comunicativas tiene el Toque Manual de Campanas declarado en 2019 y de qué manera se está protegiendo este patrimonio inmaterial en España?" } ], "positive_messages": [ [ { "role": "user", "content": "El Toque Manual de Campanas, declarado en 2019, es un lenguaje sonoro que ha funcionado a lo largo de los siglos como un medio de comunicación, cumpliendo un conjunto de funciones sociales para la comunidad: informar, coordinar, delimitar el territorio y proteger. ..." } ] ], "negative_messages": [ [ { "role": "user", "content": "En la campana se distinguen las siguientes inscripciones: ... Campana Wamba: Fundida en 1219, es la campana más antigua en funcionamiento de España. ..." } ], [ { "role": "user", "content": "Esto significa que a partir de ahora las declaraciones de bienes españoles se van a otorgar con cuenta gotas entre los bienes aprobados en la Lista Indicativa. ..." } ] ] } ``` ### Data Fields (Hard Negatives) | Field | Type | Description | | :--- | :--- | :--- | | **train_phase** | `string` | Phase label for training organization. | | **difficulty** | `string` | Difficulty level: `high_school`, `university`, or `phd`. | | **messages** | `list` | Query in conversational format (single message). | | **positive_messages** | `list` | List containing one positive passage in conversational format. | | **negative_messages** | `list` | List of hard negative passages in conversational format. | ## Config 2: Evaluation (Triplets) **Purpose:** Evaluation of retrieval models using query-passage-answer triplets. ### Data Instances Each evaluation instance is a **triplet** containing: - a **query** (plain text) - a **passage** (the retrieval candidate) - an **answer** (reference/expected answer for the query) - metadata: source, difficulty, character type, etc. Example: ```json { "dataset": "triplets_7275", "source_id": "Revista_Hipogrifo", "id_passage_query": "Revista_Hipogrifo-The_Topic_of_the_Jerusalem_War_in_Luis_de_Miranda_and_Martin_del_Barco_Centenera-13106_3", "character": "17th century Jesuit history researcher", "type": "background", "difficulty": "university", "query": "¿Qué papel desempeña el tema del asedio de Jerusalén en los textos coloniales tempranos del Río de la Plata y cómo lo expresan Luis de Miranda y Martín del Barco Centenera?", "id_passage": "Revista_Hipogrifo-The_Topic_of_the_Jerusalem_War_in_Luis_de_Miranda_and_Martin_del_Barco_Centenera-13106", "passage": "This analysis, whose point of departure is the discursive complexity of colonial productions... [full passage text]", "answer": "El análisis muestra que el asedio de Jerusalén, tomado de la obra de Flavio Josefo, se rearticula en la escena política colonial temprana del Río de la Plata..." } ``` ### Data Fields (Evaluation) | Field | Type | Description | | :--- | :--- | :--- | | **dataset** | `string` | Dataset identifier. | | **source_id** | `string` | Source repository of the document. | | **id_passage_query** | `string` | Unique identifier for the query-passage pair. | | **character** | `string` | Character or perspective for the query context. | | **type** | `string` | Type of query context (e.g., "background", "summary"). | | **difficulty** | `string` | Difficulty level: `high_school`, `university`, or `phd`. | | **query** | `string` | Plain text query. | | **id_passage** | `string` | Unique identifier for the passage. | | **passage** | `string` | The full passage text. | | **answer** | `string` | Reference answer or expected retrieval result. | ------------------------------------------------------------------------ # Example Usage The dataset can be loaded using the HuggingFace **datasets** library. ## Load All Configs ``` python from datasets import load_dataset # Load hard negatives (training) hard_neg_train = load_dataset("SINAI/ALIA-es-cultural-heritage-triplets", "hard-negatives") print(hard_neg_train["train"][0]) # Load evaluation triplets eval_triplets = load_dataset("SINAI/ALIA-es-cultural-heritage-triplets", "evaluation") print(eval_triplets["test"][0]) ``` ## Access Hard Negatives (Training Config) ``` python example = hard_neg_train["train"][0] query = example["messages"][0]["content"] positive = example["positive_messages"][0][0]["content"] negatives = [n[0]["content"] for n in example["negative_messages"]] print("Query:", query) print("Positive:", positive[:200]) print(f"Hard Negatives: {len(negatives)} passages") ``` ## Example for Contrastive Training ``` python query = example["messages"][0]["content"] positive = example["positive_messages"][0][0]["content"] negatives = [n[0]["content"] for n in example["negative_messages"]] training_example = { "query": query, "positive": positive, "negatives": negatives, "difficulty": example["difficulty"] } ``` This format can be used to train retrieval models such as: - SentenceTransformers dense retrievers with multi-negatives - dual‑encoder retrieval models - RAG retrievers with contrastive learning ## Access Evaluation Triplets ``` python example = eval_triplets["test"][0] query = example["query"] passage = example["passage"] answer = example["answer"] print("Query:", query) print("Passage:", passage[:200]) print("Expected Answer:", answer[:200]) ``` ## Example for Retrieval Evaluation ``` python eval_example = { "query": example["query"], "passage": example["passage"], "answer": example["answer"], "difficulty": example["difficulty"], "source": example["source_id"] } ``` This triplet format is suitable for: - Evaluating retrieval model ranking quality - Assessing passage-query relevance - Benchmarking dense retrieval systems # Difficulty Levels ## Motivation The inclusion of multiple difficulty levels is designed to improve the robustness and generalization of retrieval models. Different difficulty levels introduce variation in: - linguistic complexity - domain-specific terminology - reasoning depth required to distinguish correct vs incorrect passages ## Description of Each Level - **high_school**: Contains simpler queries with more explicit wording and lower semantic ambiguity. Negatives are easier to distinguish from the correct passage. - **university**: Includes moderately complex queries with more specialized vocabulary and increased semantic overlap between positives and negatives. - **phd**: Contains highly complex queries from the biomedical domain, often requiring fine-grained semantic understanding. Hard negatives in this setting are very close to the correct answer, making the retrieval task significantly more challenging. These levels correspond to the **complexity of the original query–passage pairs** from which hard negatives were generated and evaluation triplets selected. ## Training Implications Using multiple difficulty levels allows: - curriculum learning strategies (easy → hard) - more robust embedding models - evaluation across different complexity regimes This design is aligned with the hard negative mining pipeline. ------------------------------------------------------------------------ # Dataset Creation ## Hard Negative Mining Pipeline Hard negatives were generated using the SentenceTransformers utility: ``` python from sentence_transformers.util import mine_hard_negatives ``` Mining procedure: 1. Encode queries and passages with an embedding model 2. Build a FAISS similarity index 3. Retrieve semantically similar passages 4. Apply similarity filtering constraints 5. Select passages close to the query embedding but incorrect Two sampling selection strategies were used: * **Phase 1**: Randomly samples negatives from the candidate pool * **Phase 2**: Most similar negatives ------------------------------------------------------------------------ ## Embedding Model | Parameter | Value | |---------------------|---------------------------| | Model | Qwen3-Embedding-0.6B | | Framework | SentenceTransformers | | Similarity Search | FAISS | ------------------------------------------------------------------------ ## Mining Parameters ``` python mine_hard_negatives( dataset=hf_dataset, model=model, output_scores=True, range_min=10, range_max=50, max_score=0.8, relative_margin=0.05, num_negatives=5, batch_size=16, use_faiss=True, anchor_column_name="query", positive_column_name="passage" ) ``` ------------------------------------------------------------------------ # Statistics | Difficulty | Hard Negatives | Evaluation | | :--- | ---: | ---: | | **high_school** | 414,752 | 1,818 | | **university** | 766,416 | 3,660 | | **phd** | 202,704 | 1,084 | | **TOTAL** | **1,383,872** | **6,562** | ------------------------------------------------------------------------ ## Annotations No manual annotations exist. ## Personal and Sensitive Information The data comes from publicly available cultural heritage and institutional sources. Any personally identifiable information (if originally present) has been filtered in the pre-processing stages to ensure privacy and compliance. --- # Considerations for Using the Data ## Social Impact of Dataset Facilitates the development of cultural heritage encoders in Spanish, improving natural language processing and retrieval systems for heritage documentation, cultural information access, and digital humanities research. ## Discussion of Biases Reflects: - biases from heritage, institutional, and historical documentation, - inherent limitations of the generating model, ## Other Known Limitations - Interpretation of specialized heritage terminology may vary - Synthetic hard negatives may occasionally include factual heritage information that is plausible but technically incorrect for the specific query - Synthetic hard negatives may occasionally be semantically very close to the positive passage, making some examples especially challenging ## Citation ```bibtex @misc{ALIA-es-cultural-heritage-triplets, title={ALIA Spanish Cultural and Heritage Hard Negatives Corpus}, author={SINAI Research Group}, year={2026}, publisher={HuggingFace}, howpublished={\url{https://huggingface.co/datasets/SINAI/ALIA-es-cultural-heritage-triplets}} } ``` --- ### Funding This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project [ALIA](https://alia.gob.es). ### Acknowledgments This dataset has been generated thanks to [SCAYLE](https://www.scayle.es/) (Centro de Supercomputación de Castilla y León) which provided the needed computational resources on its CALENDULA supercomputing cluster. --- **Contact:** [ALIA Project](https://www.alia.gob.es/) - [SINAI Research Group](https://sinai.ujaen.es) - [Universidad de Jaén](https://www.ujaen.es/) **More Information:** [SINAI Research Group](https://sinai.ujaen.es) | [ALIA-UJA Project](https://github.com/sinai-uja/ALIA-UJA)
The ALIA Spanish Cultural and Heritage Hard Negatives Corpus is a dataset designed for training dense retrieval models in the Spanish cultural heritage domain. It contains hard negatives automatically generated from query-passage pairs in the SINAI/ALIA-es-cultural-heritage-pairs dataset. Hard negatives are passages that are semantically similar to a query but not correct answers, making them particularly useful for training robust retrieval systems. The dataset was created as part of the ALIA project to improve the training of embedding models and dense retrievers specialized in Spanish cultural heritage language. The dataset has two main configurations: the training configuration (hard-negatives) contains queries, one positive passage, multiple hard negative passages (typically 5 per query), along with train phase and difficulty labels; the evaluation configuration (evaluation) contains query-passage-answer triplets for evaluating retrieval models. The dataset includes three difficulty levels: high_school, university, and phd, corresponding to different levels of query complexity and semantic ambiguity. Data was generated automatically using a SentenceTransformers-based hard negative mining pipeline with the Qwen3-Embedding-0.6B embedding model and FAISS similarity search. In total, there are 1,383,872 hard negatives and 6,562 evaluation triplets.
