Pretraining dataset for t5chem

The pubchem molecules that used to pretrain t5chem models in this work: https://pubs.acs.org/doi/abs/10.1021/acs.jcim.1c01467Github source code: https://github.com/HelloJocelynLu/t5chem/tree/main Due to computational compacity and resolve potential OOM problems for some machines, the whole dataset is splitted to 10 subsets and can be extract by: ``` tar -xzvf MLM.tar.gz ``` Then you can pretrain your own model by running: ``` t5chem train --data_dir MLM/train_0 --output_dir model/pretrain/v1_1-small --task_type pretrain --num_epoch 1 --pretrain google/t5-v1_1-small ```