mPerDisSocial

The dataset consists of multilingual machine-generated personalized social-media texts against and supporting the selected 6 disinformation narratives (health-related and political). It was used to evaluate LLM capabilities for personalized counter-narrative texts generation (e.g., supporting argumentation for intervention) as well as used to evaluate LLM vulnerabilities to being misused for personalized disinformation generation in the selected 10 languages (cs, de, en, et, hr, hu, pl, sk, sl, uk). It consists of 17,278 disinformation articles generated by 16 LLMs of various sizes and architectures (DeepSeek-R1-Distill-Llama-8B, DeepSeek-R1-Distill-Qwen-32B, Gemma-2-27B-it, Gemma-2-2B-it, Gemma-2-9B-it, Gemma-3-27B-it, Gemma-3-4B-it, Llama-3.1-70B-Instruct, Llama-3.1-8B-Instruct, Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Llama-3.3-70B-Instruct, Mistral-Nemo-Instruct-2407, Qwen3-1.7B, Qwen3-32B, Qwen3-4B). The data were generated using prompts targeting 3 social-media platforms (Mastodon, Telegram, and Twitter/X) and 3 target groups (European conservatives, urban residents, and none for baseline). If you use this dataset in any publication, project, tool or in any other form, please, cite the paper. Disclaimer The data contain intentional disinformation, generated by large language models. The dataset has been checked for containment of personally identifiable information (PII) and the samples considered dangerous have been anonymized. However, the used procedure might not be 100% effective, therefore, the people or organizations feeling affected can report the found issues in this regard to dpo[at]kinit.sk. Data The dataset has the following fields: 'prompt_id' - the identifier of the specific combination of target group, target platform, narrative, language and stance 'target_title' - the identifier of the target group used in the prompt ('-', 'Urban population', 'European conservatives') 'platform_name' - the identifier of the target platform used in the prompt ('Mastodon', 'Telegram', 'Twitter') 'narrative_title' - the title of the disinformation narrative used in the prompt 'language' - the intended language of the generated text 'stance' - the intended stance of the generated text towards the narrative ('supporting' or 'against') 'model_name' / 'generator' - the identifier of the LLM that generated the text 'text' - the text of the generated social-media post 'safetyfilter_heuristic' - yes/no whether the heuristics revealed safetyfilter message 'noise_heuristic' - yes/no whether the heuristics revealed noise 'annotation_group-personalization_LLM#' - label of group-personalization quality assigned by a given LLM (# represents the number 1 to 3) 'annotation_platform-personalization_LLM#' - label of platform-personalization quality assigned by a given LLM (# represents the number 1 to 3) 'majority_group-personalization_eval' - label of group-personalization quality resulting from inter-LLM majority voting 'majority_platform-personalization_eval' - label of platform-personalization quality resulting from inter-LLM majority voting 'annotation_X_LLM#' - Yes/No/Partly annotation by a given LLM about additional questions (X represents an identifier of the question, # represents the number 1 to 3) 'majority_X_eval' - inter-LLM majority voted annotation for a given question 'majority_stance_eval' - resulting stance of the generated text towards the narrative (Agree/Disagree) 'gemma-2-9b-it-genai_pred', 'gemma-2-9b-it-multidomain_pred', 'Qwen3-14B-Base-38400-multidomain_pred' - predictions of machine-generated text detectors (1 - machine text, 0 - human text) 'majority_detectability' - inter-detector majority voted prediction of machine-generated text (1 - machine text, 0 - human text) Sample counts: Target group Target platform cs de en et hr hu pl sk sl uk - Mastodon 192 192 192 192 192 192 192 192 192 192 - Telegram 192 192 192 192 192 192 192 192 192 192 - Twitter 192 192 192 192 192 192 192 192 192 192 European conservatives Mastodon 192 192 192 192 192 192 192 192 192 192 European conservatives Telegram 192 192 192 192 192 192 192 192 192 192 European conservatives Twitter 192 192 192 191 192 192 192 192 192 192 Urban population Mastodon 192 192 192 192 192 192 192 192 192 192 Urban population Telegram 192 192 192 192 192 192 192 192 192 192 Urban population Twitter 192 192 192 192 192 192 191 192 192 192