MALANTIN

# Multilingual Corporate Corpus — Vectorized Datasets **Project MALANTIN — Université Paris 8 / LIASD**This repository contains the fully anonymized and vectorized derivative datasets produced from the multilingual corporate corpus developed in the MALANTIN project. All textual content has been removed to comply with copyright and data-protection requirements. Only non-reversible dense vector representations and non-sensitive metadata are released.The repository includes three dataset families: 1. **French vector dataset** (`french/`) 2. **English vector dataset** (`english/`) 3. **Machine learning dataset** (`machine_learning_dataset/`) — curated, labeled, sentence-level dataset designed for NLP experiments.---## 1. Dataset Contents### **1.1 French Vector Dataset (`french/`)**Files:- `fr_embeddings_pca_117.npy` - `fr_embeddings_umap_64.npy` - `fr_metadata_no_text.csv`Content:- PCA-reduced vectors (117 dimensions, respecting PCA constraints) - UMAP projections (64 dimensions) - Metadata (language, sector, keywords, text_length, doc_id if applicable)### **1.2 English Vector Dataset (`english/`)**Files:- `en_embeddings_pca_128.npy` - `en_embeddings_umap_64.npy` - `en_metadata_no_text.csv`Content:- PCA embeddings (128 dimensions) - UMAP embeddings (64 dimensions) - Metadata (same schema as French dataset)### **1.3 Machine Learning Dataset (`machine_learning_dataset/`)**Files:- `ml_sentence_embeddings_pca_128.npy` - `ml_sentence_embeddings_umap_64.npy` - `ml_sentence_metadata_no_text.csv`Content:- Sentence-level anonymized representations - Labels (`word_labels`) and thematic keywords - Document source identifier (`source`) - PCA and UMAP embeddings optimized for supervised learningThis dataset is the recommended resource for classification, clustering, and semantic modeling.---## 2. Methodology Summary- Texts were collected from corporate websites and annual reports. - Sensitive entities (PERSON, ORGANIZATION, LOCATION, EMAIL, PHONE, etc.) were anonymized using **Microsoft Presidio**. - Sentences were encoded using **Sentence-Transformers** (`paraphrase-multilingual-MiniLM-L12-v2`). - Dimensionality reduction was applied: - PCA (max 128 components, respecting `min(n_samples, n_features)`) - UMAP (64 dimensions) - Only vectors and non-sensitive metadata are released. - No raw text or anonymized text is included.---## 3. Metadata SchemaAll metadata files include only non-sensitive fields:| Column name | Description ||------------------|-------------------------------------------------------|| `language` | Document language || `sector` | Economic sector || `keywords` | Lexical or thematic keywords || `word_labels`* | Label used in ML dataset (*machine_learning_dataset*) || `source` | Identifier for the document or website || `doc_id`* | Optional unique integer identifier || `text_length` | Length of original text (token count) |No textual content is stored.---## 4. File Formats- Vector embeddings: `.npy` (NumPy arrays) - Metadata: `.csv` (UTF-8) The datasets are lightweight, structured, and easy to load in Python.Example usage:```pythonimport numpy as npimport pandas as pdemb = np.load("machine_learning_dataset/ml_sentence_embeddings_pca_128.npy")meta = pd.read_csv("machine_learning_dataset/ml_sentence_metadata_no_text.csv")```---## 5. LicensingThis dataset is released under the **Open Data Commons Attribution License (ODC-BY 1.0)**. See `LICENSE.txt` for full terms.Users are free to use, share, and adapt the dataset for research, provided appropriate attribution is given.---## 6. CitationIf you use this dataset, please cite:Revekka Kyriakoglou amp; Anna Pappa (2025). **Multilingual Corporate Corpus for the Study of Emerging Concepts in SHS.** Université Paris 8 — LIASD. Dataset version: 1.0.Persistent Identifier: *https://doi.org/10.57745/M9UHMX*---## 7. ContactFor questions: - **Revekka Kyriakoglou** — [e-mail] - **Anna Pappa** — [e-mail] LIASD, Université Paris 8.