CACP-Net Culture-Aware Cross-Modal Prompt Learning and Intelligent Distribution Network for Ethnic Film and Television

CACP-Net Culture-Aware Cross-Modal Prompt Learning and Intelligent Distribution Network for Ethnic Film and Television This repository hosts the reference PyTorch implementation of CACP-Net and the companion EthnicVid-3K benchmark introduced in the paper. --- Repository contents Package Description Size `CACP-Net.zip` PyTorch implementation + pretrained weights + evaluation pipeline ~21 MB `EthnicVid-3K.zip` Multimodal dataset (3,200 clips, 10 ethnic communities) with recommendation subset ~19 MB --- 1. Paper summary CACP-Net is a culture-aware cross-modal network that keeps three pretrained encoders (CLIP ViT-B/16, CLIP Transformer, Wav2Vec2-Base) frozen and jointly optimises three cooperating modules: CAPL — Culture-Aware Prompt Learning. Hierarchical soft prompts combined with embeddings of 1,200 ethnic cultural entities fill the knowledge gaps of general vision-language models on ethnic cultural symbols. CMSA — Cross-Modal Semantic Alignment. A trimodal contrastive objective reweights negative pairs by cultural affinity distance, giving culturally close negatives stronger supervision. UCDT — User-Content Dual-Tower Distribution. A cultural affinity gate blends collaborative matching with a culture-based prior and opens an exposure channel for cold-start and long-tail works. The three modules share a cultural entity embedding space and are optimised end to end with a staged loss-weight schedule. On the self-constructed EthnicVid-3K benchmark the full model updates only 8.7 M parameters and reaches R@1 = 44.3% on text-to-video retrieval and HR@10 = 0.358 on content distribution, while the exposure Gini coefficient drops from 0.62 to 0.34 over the strongest baseline. --- 2. Dataset at a glance (EthnicVid-3K) Attribute Value Number of clips 3,200 Ethnic communities 10 (Tibetan, Miao, Mongolian, Uyghur, Dai, Yi, Dong, Korean, Zhuang, Kazakh) Average clip duration ~48 seconds English descriptions per clip 3 Cultural labels per clip (mean) 2.3 Cultural dictionary entities 1,200 Train / Val / Test split 2,240 / 320 / 640 Vocabulary size 8,724 Users (recommendation subset) 8,532 Interaction records 216,358 Interaction density 0.79% Mean interactions per user ~25.4 See `EthnicVid-3K/README.md` for the full dataset specification, field dictionaries, and ethics statement. --- 3. Model summary (CACP-Net) Component Role Trainable Visual encoder CLIP ViT-B/16 — 12 frames per clip, mean-pooled over time (Eq. 1) no Text encoder CLIP Transformer — receives culture-aware prompt sequence (Eq. 2) no Audio encoder Wav2Vec2-Base — 16 kHz waveform, mean-pooled, linear projection to d (Eq. 3) projection only CAPL module Hierarchical soft prompts + cultural dictionary (Eqs. 5–9) yes CMSA module Culture-aware weighted InfoNCE + trimodal fusion (Eqs. 10–14) yes UCDT module Dual-tower + cultural affinity gate (Eqs. 15–20) yes Total trainable: 8.7 M. Total parameters including the three frozen encoders: 252.6 M. --- 4. Quick start 4.1 Set up the dataset ```bash unzip EthnicVid-3K.zip cd EthnicVid-3K # Resample audio to 16 kHz mono and extract 12 frames per clip python scripts/extract_audio.py python scripts/extract_frames.py python scripts/verify_dataset.py cd .. ``` 4.2 Install and run the model ```bash unzip CACP-Net.zip cd CACP-Net pip install -r requirements.txt pip install -e . bash scripts/run_all.sh ``` 4.3 Train from scratch ```bash python scripts/train.py \ --data-root ../EthnicVid-3K \ --num-epochs 40 \ --batch-size 128 \ --lr 1e-4 ``` --- 5. Reproduced results Table / Figure Content Metric highlight Table 3 Text-to-video retrieval on EthnicVid-3K and MSR-VTT CACP-Net R@1 = 44.3 (EthnicVid-3K), 48.0 (MSR-VTT) Table 4 Video-to-text retrieval on EthnicVid-3K CACP-Net R@1 = 42.9 Table 5 Efficiency measurements CACP-Net trainable = 8.7 M, two-stage latency = 12.4 ms Table 6 Content distribution CACP-Net HR@10 = 0.358, NDCG@10 = 0.209, MRR@10 = 0.164 Table 7 Ablation (five variants) w/o CAPL: R@1 -5.9; w/o UCDT: HR@10 -7.7 points Table 8 Hyperparameter sensitivity (M, d_c, tau_2) Best at M = 8, d_c = 128, tau_2 = 0.07 Table 9 Cold-start and long-tail scenarios Cold-start-users HR@10 = 0.234 (+45.3% over HGCL) Figure 7 Normalised retrieval metrics (radar) CACP-Net dominates on all five metrics Figure 8 Ablation variants relative to the full model — Figure 9 Training dynamics with the stage-wise schedule — Figure 10 Cumulative exposure + Gini coefficient Gini drops from 0.62 to 0.34 --- 6. Equation-to-code map Equation Description File (1)-(3) Three branch encoders `cacpnet/models/encoders.py` (4) Weighted multitask objective + L2 `cacpnet/losses.py` (5) Hierarchical soft prompts `cacpnet/models/capl.py` (6) Cultural dictionary embedding `cacpnet/models/capl.py` (7)-(8) Gated fusion of prompts `cacpnet/models/capl.py` (9) CAPL loss `cacpnet/models/capl.py` (10) Cultural affinity distance `cacpnet/models/cmsa.py` (11) Negative pair weight `cacpnet/models/cmsa.py` (12) Weighted InfoNCE `cacpnet/models/cmsa.py` (13) CMSA total loss `cacpnet/models/cmsa.py` (14) Retrieval score `cacpnet/models/cmsa.py` (15) User cultural profile `cacpnet/models/ucdt.py` (16)-(17) Dual-tower outputs `cacpnet/models/ucdt.py` (18) Cultural affinity gate `cacpnet/models/ucdt.py` (19) Distribution score `cacpnet/models/ucdt.py` (20) UCDT loss `cacpnet/models/ucdt.py` See `CACP-Net/docs/METHODS.md` for the detailed mapping with line references. --- 7. Requirements Python >= 3.8 PyTorch >= 2.0 open_clip_torch, transformers, numpy, matplotlib, pandas, PyYAML, Pillow ffmpeg (for audio/frame extraction from the dataset) A single NVIDIA A100 80 GB GPU is sufficient for full training (40 epochs in ~5.2 hours). --- 8. Ethics statement The dataset used publicly accessible video resources and anonymised viewing logs that contain no personally identifiable information. User identifiers were hashed by the data provider before delivery and no demographic attributes were retained. --- 9. Citation ```bibtex @article{Wei2026CACPNet, author = {Wei, Jingjing}, title = {CACP-Net: Culture-Aware Cross-Modal Prompt Learning and Intelligent Distribution Network for Ethnic Film and Television}, year = {2026}, } ``` --- 10. License Research use only. See the paper for the full license and usage terms.