NTIS-WoS Matched Published Paper

NTIS–WoS Matched Papers This document accompanies the Zenodo release that links National Science & Technology Information Service (NTIS)-funded publications to Web of Science (WoS) metadata. It summarises the source data, cleansing pipeline, matching logic, key variables, and considerations for public sharing. Dataset Overview Purpose: expose a reproducible mapping between NTIS paper records and WoS identifiers so downstream users can validate or extend the linkage. Public payload: the Zenodo table contains only six columns — NTIS project ID, DOI_NTIS, DOI_NTIS_cleaned, validity_flag, DOI_WoS, uid_WoS. Everything else (titles, bibliometrics, provenance flags) remains internal. Matching keys: cleansing is still performed on the NTIS DOI; when a DOI cannot be recovered, an internal title_vol comparison is used to source the WoS DOI/uid before the public table is trimmed down. Source Material Dataset Description Contains NTIS Korean Funding DB NTIS at November in 2023 NTIS project IDs (과제고유번호), published paper's titles, raw DOIs WoS Web of Science DB at 39th weeks in 2024 WoS uid, title, vol, issue, citation indicators, etc. Only the fields required for linking (year, title, DOI, SCI flag, NTIS project ID) are pulled into the working Polars DataFrame. Processing Summary DOI cleansing (Polars) Simple replacements (looped sequentially): '99999999999'→'' ' '→'' '\\\\n'→'' '?'→'' '링크'→'' 'ㄴ'→'' ','→'.' ']'→'', '%28'→'(' '%29'→')' Regex replacements (Rust-style backrefs): .*?(10\\\\.\\\\d{4,9}/[-._;()/:a-z0-9]+) → $1 (<https://dx>\\\\.doi\\\\.org/|http://dx\\\\.doi\\\\.org/|dx\\\\.doi\\\\.org/) → `` ^0\\\\. → 10. (10\\\\.\\\\d{4,9})\\\\. → $1/ (10\\\\.\\\\d{4,9})([a-z0-9]+) → $1/$2 %2f → / ^(\\\\d{2})(\\\\d{4}/) → $1.$2 (^/|^:) → `` Validation flag: doi_pattern = r'(?i)^10.\\\\d{4,9}/[-._;()/:a-z0-9]+$' WoS title normalisation HTML-like markup is stripped so titles align with NTIS records: pattern = re.compile(r"<[^>]*>") Matching workflow Primary DOI join — DOI_NTIS_cleaned (NTIS) ↔ DOI_WoS (WoS). Fallback exact-title join — for rows still missing DOI, match TITLE+VOL_NTIS ↔ TITLE+VOL_WoS. Table Description Column label Description NTIS project ID NTIS project identifier mapped to the paper DOI_NTIS Original DOI string supplied by NTIS DOI_NTIS_cleaned Normalised DOI after the cleansing rules Validity flag Boolean flag indicating the cleaned DOI meets the canonical regex DOI_WoS DOI harvested from the WoS record used for the match uid_WoS WoS unique identifier that anchors all downstream metrics All other columns (titles, title_vol, bibliometrics, internal match strategy) are retained only in the private diagnostic parquet. Validation & Coverage Checks cnt_valid_doi = df_pap_sub.group_by('is_valid_doi').len() — proportion of records with syntactically valid DOIs. df_merged.drop_nulls(subset='uid').height / df_merged.height — overall linkage rate after applying DOI-first then title_vol. df_wos.filter(pl.col('wos_title_clean').is_in(...)) — WoS slice that now maps to NTIS, supporting manual audits. Limitations & Caveats Hangul keyboard recovery rules from earlier experiments are disabled; enable them if new NTIS dumps show IME artefacts. Title fallback relies on exact equality. A string-grouper prototype exists later in the notebook but is not part of the published pipeline. When multiple DOIs appear in a single cell, only the first DOI is retained (design choice). Sharing Considerations The published six-column table carries only project/publication identifiers and DOI variants; no personal data or institutional review details are exposed. Keep the richer diagnostic parquet internal (it includes titles, bibliometrics, and intermediate flags). Consider adding a short note in the Zenodo description that non-essential NTIS/WoS metadata has been withheld intentionally. This documentation should be packaged with the Zenodo upload so users understand the provenance of the matched dataset and the meaning of the key fields. Acknowledgement This dataset is a research output produced under KISTI’s 2025 Basic Project (Project No.: K25L4M2C3).