dp-testcorpus

test corpus for dp work presented as 52 test bags + rawcorpus/ (unbagged files) files came from: Waterloo Image Repository: links.uwaterloo.ca/Repository.html OpenPreserve Foundation (OPF): github.com/openpreserve/format-corpus Canterbury Corpus: corpus.canterbury.ac.nz Calgary Corpus Total # of files: 1340 Process: files extracted, directories removed made a bunch of bag directories: for ((n=1;n<100;n++));do mkdir bag$n; done used shuf to randomly select files for bag directories: for ((n=1;n<100;n++)); do shuf -zen$n ./* | xargs -0 mv -t ../bag$n/; done turned the bag directories into actual bags: do bagit.py --md5 --source-organization UTL --internal-sender-description 'random bag generated for testing' --co$ note: Bags increase in size only in that the number of files in each bag increases by 1