Data and codes for "Disentangling Multi-view Representations Beyond Inductive Bias"

This record contains the data and codes for this paper: Guanzhou Ke, Yang Yu, Guoqing Chao, Xiaoli Wang, Chenyang Xu, and Shengfeng He. 2023. "Disentangling Multi-view Representations Beyond Inductive Bias." In Proceedings of the 31st ACM International Conference on Multimedia (MM '23), October 29–November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3581783.3611794 dmrib-weights is the file for pre-trained weights. DMRIB-main is a copy of the project's GitHub Repository at https://github.com/Guanzhou-Ke/DMRIB The official repos for ""Disentangling Multi-view Representations Beyond Inductive Bias"" (DMRIB) Status: Accepted in ACM MM 2023.Training stepWe show that how DMRIB train on the EdgeMnist dataset. Before the training step, you need to set the CUDA_VISIBLE_DEVICES, because of the faiss will use all gpu. It means that it will cause some error if you using tensor.to() to set a specific device. set environment.export CUDA_VISIBLE_DEVICES=0 train the pretext model. First, we need to run the pretext training script src/train_pretext.py. We use simclr-style to training a self-supervised learning model to mine neighbors information. The pretext config commonly put at configs/pretext. You just need to run the following command in you terminal:python train_pretext.py -f ./configs/pretext/pretext_EdgeMnist.yaml train the self-label clustering model. Then, we could use the pretext model to training clustering model via src/train_scan.py.python train_scan.py -f ./configs/scan/scan_EdgeMnist.yaml After that, we use the fine-tune script to train clustering model scr/train_selflabel.py. python train_selflabel.py -f ./configs/scan/selflabel_EdgeMnist.yaml training the view-specific encoder and disentangled. Finally, we could set the self-label clustering model as the consisten encoder. And train the second stage via src/train_dmrib.py.python train_dmrib.py -f ./configs/dmrib/dmrib_EdgeMnist.yaml ValidationNote: you can find the pre-train weights in the file dmrib-weights. And put the pretrained models into the following folders path to/{config.train.log_dir}/{results}/{config.dataset.name}/eid-{config.experiment_id}/dmrib/final_model.pth, respectively. For example, if you try to validate the EdgeMnist dataset, the default folder is ./experiments/results/EdgeMnist/eid-0/dmrib. And then, put the pretrained model edge-mnist.pth into this folder and rename it to final_model.pth. If you do not want to use the default setting, you have to modify the line 58 of the validate.py. python validate.py -f ./configs/dmrib/dmrib_EdgeMnist.yaml CreditThanks: Van Gansbeke, Wouter, et al. "Scan: Learning to classify images without labels." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X. Cham: Springer International Publishing, 2020. CitationGuanzhou Ke, Yang Yu, Guoqing Chao, Xiaoli Wang, Chenyang Xu, and Shengfeng He. 2023. Disentangling Multi-view Representations Be- yond Inductive Bias. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23), October 29–November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3581783.3611794本数据集包含对应论文的数据与代码： Guanzhou Ke、Yang Yu、Guoqing Chao、Xiaoli Wang、Chenyang Xu与Shengfeng He于2023年发表的《超越归纳偏倚的多视图表征解耦（Disentangling Multi-view Representations Beyond Inductive Bias）》，收录于第31届ACM国际多媒体大会（ACM MM '23）论文集，会议时间为2023年10月29日至11月3日，举办地为加拿大安大略省渥太华，由ACM出版社出版，美国纽约，共9页，DOI:10.1145/3581783.3611794。 其中dmrib-weights为预训练权重文件；DMRIB-main是本项目GitHub仓库的副本，仓库地址为https://github.com/Guanzhou-Ke/DMRIB，本仓库对应论文《超越归纳偏倚的多视图表征解耦（Disentangling Multi-view Representations Beyond Inductive Bias）》（简称DMRIB），本论文已被ACM MM 2023收录。 ## 训练流程 本节展示DMRIB在EdgeMnist数据集上的训练步骤： ### 训练前环境配置 由于faiss库会占用全部GPU资源，若直接使用tensor.to()指定设备可能引发错误，因此需提前设置CUDA_VISIBLE_DEVICES。执行如下命令配置环境： export CUDA_VISIBLE_DEVICES=0 ### 分步训练 1. 训练前置任务模型（pretext model）：运行src/train_pretext.py脚本，采用SimCLR风格训练自监督学习模型以挖掘邻域样本信息，前置任务配置文件通常存放在configs/pretext目录下，只需在终端执行如下命令： python train_pretext.py -f ./configs/pretext/pretext_EdgeMnist.yaml 2. 训练自标记聚类模型：利用上述前置任务模型，通过src/train_scan.py训练聚类模型，执行如下命令： python train_scan.py -f ./configs/scan/scan_EdgeMnist.yaml 3. 微调自标记聚类模型：运行微调脚本src/train_selflabel.py，执行如下命令： python train_selflabel.py -f ./configs/scan/selflabel_EdgeMnist.yaml 4. 训练视图专属编码器与解耦模块：将上述自标记聚类模型作为一致性编码器，通过src/train_dmrib.py执行第二阶段训练，执行如下命令： python train_dmrib.py -f ./configs/dmrib/dmrib_EdgeMnist.yaml ## 验证流程 注意：可从dmrib-weights文件中获取预训练权重，需将预训练模型放置至对应文件夹路径：{config.train.log_dir}/{results}/{config.dataset.name}/eid-{config.experiment_id}/dmrib/final_model.pth。以EdgeMnist数据集为例，默认路径为./experiments/results/EdgeMnist/eid-0/dmrib，将预训练模型edge-mnist.pth放入该文件夹并重命名为final_model.pth。若需修改默认设置，请修改validate.py第58行代码。验证命令： python validate.py -f ./configs/dmrib/dmrib_EdgeMnist.yaml ## 致谢 感谢Van Gansbeke、Wouter等人的论文《SCAN: Learning to classify images without labels》，该论文收录于《Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X》，由Springer International Publishing于2020年出版。 ## 引用信息 Guanzhou Ke, Yang Yu, Guoqing Chao, Xiaoli Wang, Chenyang Xu, and Shengfeng He. 2023. Disentangling Multi-view Representations Beyond Inductive Bias. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23), October 29–November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3581783.3611794