CALM

This contains dataset for a multi-task dataset for measuring language model bias : CALM This dataset is intended towards reliable deployment of language models by measuring issues of gender and race bias in model's prediction. The dataset contains 3 task : question answering, sentiment classification, and natural language inference. The dataset was collected by converting 16 exisiting datasets into templates.