Dataset of English Translations of Non-English Abstracts of OpenAlex Works Cited in Overton Policy Documents
收藏资源简介:
Overview Target: OpenAlex works cited in Overton policy documents Data: Overton (2025-02-11 snapshot), OpenAlex (2025-05-30 snapshot) Objective: Select abstracts detected as non-English (or with English probability < 0.8) and provide English translations Models used: Open-source models (fastText for language detection, gpt-oss-120b for translation) Files (3 total): Overton_OpenAlex_matched_works_id.csv — Overton–OpenAlex matched IDs total_lang_detect.csv — Language detection results (English probability < 0.8 + clearly non-English) translate.csv — English translations of abstracts tagged as non-English 1) Overton_OpenAlex_matched_works_id.csv Description: Match results between Overton policy documents and OpenAlex worksRow count: 6,532,365 Schema Column Type Description oaid_w string OpenAlex Work ID doi string DOI 2) total_lang_detect.csv Description: From ~6.6M matches, language detection results for OpenAlex abstracts with English probability < 0.8 or clearly non-EnglishRow count: 1,569,586 Schema (standardized column names) Column Type Description oaid_w string OpenAlex Work ID doi string DOI lang string Language detected by fastText lang_detect_llm string Detection value based on gpt-oss-120b lang_score float fastText language score Translated bool Whether translation was performed 3) translate.csv Description: Subset of total_lang_detect.csv with English translations for abstracts tagged as non-EnglishRow count: 474,130 Schema (final standardized column names) Column Type Description oaid_w string OpenAlex Work ID doi string DOI lang string Language detected by fastText lang_detect_llm string Detection result based on gpt-oss-120b lang_score float fastText language score abstract_en string English translated abstract # Acknowledgement This dataset is a research output produced under KISTI’s 2025 Basic Project (Project No.: K25L4M2C3).



