POLITISKY24: U.S. Political Bluesky Dataset with User Stance Labels

POLITISKY24 (Political Stance Analysis on Bluesky for 2024) is a first-of-its-kind dataset for stance detection, focused on the 2024 U.S. presidential election. It designed for target-specific user-level stance detection and contains 16,044 user-target stance pairs centered on two key political figures, Kamala Harris and Donald Trump. In addition, this dataset includes detailed metadata, such as complete user posting histories and engagement graphs (likes, reposts, and quotes). 

Stance labels were generated using a robust and evaluated pipeline that integrates state-of-the-art Information Retrieval (IR) techniques with Large Language Models (LLMs), offering confidence scores, reasoning explanations, and text spans for each label. With an LLM-assisted labeling accuracy of 81%, POLITISKY24 provides a rich resource for the target-specific stance detection task. This dataset enables the exploration of Bluesky platform, paving the way for deeper insights into political opinions and social discourse, and addressing gaps left by traditional datasets constrained by platform policies.

In the uploaded files:

The file user_post_history_dataset.parquet includes the posting history of 8,561 active Bluesky users who have shared content related to American politics.

The file user_post_list_for_stance_detection.parquet contains a list of up to 1,000 recent English-language post IDs per user, intended for use in the stance detection task.

The file user_network_dataset.parquet captures users’ interactions through likes, reposts, and quotes.

The file human_annotated_validation_user_stance_dataset.parquet contains human-annotated stance labels for 445 validation users toward Trump and Harris, resulting in a total of 890 user-target pairs. The labels are divided into three stances: 1 (favor), 2 (against), and 3 (neither).

The file llm_annotated_validation_user_stance_dataset.parquet contains stance labels annotated by an LLM for the same 445 validation users toward Trump and Harris, also totaling 890 user-target pairs. In addition to stance labels, each pair includes an explanation of the reasoning, the source tweets, spans from the source tweets used in the reasoning, and a confidence score.

The file llm_annotated_full_user_stance_dataset.parquet is similar to the above LLM-annotated validation file but covers all dataset users excluding the validation set. It provides stance labels for 8,022 users toward Trump and Harris, totaling 16,044 user-target pairs.

The file human_annotated_validation_stance_relevancy_dataset (post-target entity pairs).parquet contains human-annotated stance labels for 175 validation posts toward Trump and Harris, resulting in 350 post-target pairs. The labels are divided into three stances: 1 (favor), 2 (against), and 3 (neither).

The file human_annotated_validation_stance_relevancy_dataset (query-post stance relevancy pairs).parquet contains 700 query-post stance relevancy pairs derived from the post-target entity pairs.