Same Side Stance Classification Adversarial Test Cases

This dataset contains 175 cases. It is manually created to reveal the ability of models to solve different types of adversarial cases for same side stance predictions more systematically. The examples selected here are derived from the dataset used in the Same Side Stance Classification shared task.

We have selected 25 distinct arguments from the "gay marriage" topic that are short and express their stance clearly. For each selected argument, we construct new arguments of four distinct types to obtain two pairs, one with the same stance, and one with an opposing stance:

Negation: a simple negation of the argument.

Paraphrase: alters important words from the argument to synonymous expressions with the same stance.

Argument: uses an argument from the same topic and stance, but semantically completely different regarding the first one.

Citation: repeats or summarizes the first argument and then expresses agreement or rejection (a case frequently occurring in the dataset).

The types Paraphrase, Argument, and Citation are also formulated in a negated version to create additional test instances for the opposite stance.