asta-user-interactions

# ScholarQA and Paper Finder User Interaction Dataset This dataset contains anonymized user interaction data from two AI-powered research tools: **ScholarQA (SQA)** and **Paper Finder (PF)**. The data comes from users who opted in to having their queries and interaction data released. ## Usage ```python from datasets import load_dataset queries = load_dataset("allenai/asta-user-interactions", "optin_queries")["train"] clicks = load_dataset("allenai/asta-user-interactions", "s2_link_clicks")["train"] shown_results = load_dataset("allenai/asta-user-interactions", "pf_shown_results")["train"] ``` Available configs: `optin_queries`, `section_expansions`, `s2_link_clicks`, `report_section_titles`, `report_corpus_ids`, `pf_shown_results`. ## Tools **ScholarQA** is a conversational research assistant that generates comprehensive literature review reports in response to user queries. Reports are organized into sections, with each section containing synthesized information and citations to relevant papers. **Paper Finder** is a semantic paper search tool that returns ranked lists of relevant papers based on user queries. ## Dataset Overview | File | Description | |------|-------------| | `optin_queries_anonymized.parquet` | User queries submitted to both tools | | `section_expansions_anonymized.parquet` | Section expand clicks in SQA reports | | `s2_link_clicks_anonymized.parquet` | Clicks on Semantic Scholar paper links | | `report_section_titles_anonymized.parquet` | Section titles generated in SQA reports | | `report_corpus_ids_anonymized.parquet` | Papers cited in SQA report sections | | `pf_shown_results_anonymized.parquet` | Papers shown in Paper Finder search results | ## Summary Statistics | Metric | SQA | PF | |--------|-----|-----| | Threads (queries) | 127,465 | 131,534 | | Paper link clicks | 24,304 | 181,854 | | Threads with clicks | 5,860 | 47,894 | | Papers referenced | 2,066,393 | 3,858,774 | Additional statistics: - Section expansions: 226,004 events across 33,408 threads - Unique section titles: 500,255 ## Data Collection Period Original collection window: January 1, 2025 to August 26, 2025 Date ranges by dataset: | Dataset | Earliest | Latest | |---------|----------|--------| | optin_queries | 2025-02-25 | 2025-08-27 | | section_expansions | 2025-06-03 | 2025-08-27 | | s2_link_clicks | 2025-03-26 | 2025-08-27 | | pf_shown_results | 2025-03-19 | 2025-08-27 | Note: Actual date ranges are output to `anonymization_stats.json` when the pipeline runs. ## File Descriptions ### optin_queries_anonymized.parquet User queries submitted to ScholarQA and Paper Finder. | Column | Type | Description | |--------|------|-------------| | `query` | string | The user's query text | | `thread_id` | string | Hashed identifier for the conversation thread | | `query_ts` | timestamp | When the query was submitted | | `tool` | string | Which tool was used: `sqa` or `pf` | ### section_expansions_anonymized.parquet Records of users expanding (clicking to view) sections in ScholarQA reports. ScholarQA reports are displayed with sections collapsed by default; this table captures when users click to expand and read a section. | Column | Type | Description | |--------|------|-------------| | `thread_id` | string | Hashed identifier for the conversation thread | | `section_expand_ts` | timestamp | When the section was expanded | | `section_id` | int | Index of the expanded section (0-indexed) | ### s2_link_clicks_anonymized.parquet Clicks on Semantic Scholar paper links within either tool. These represent explicit user interest in viewing a cited or retrieved paper. | Column | Type | Description | |--------|------|-------------| | `thread_id` | string | Hashed identifier for the conversation thread | | `s2_link_click_ts` | timestamp | When the link was clicked | | `corpus_id` | int | Semantic Scholar corpus ID of the clicked paper | | `tool` | string | Which tool the click occurred in: `sqa` or `pf` | ### report_section_titles_anonymized.parquet Section titles from ScholarQA generated reports. Each report contains multiple sections, and this table maps section indices to their titles. | Column | Type | Description | |--------|------|-------------| | `thread_id` | string | Hashed identifier for the conversation thread | | `section_idx` | int | Index of the section within the report (0-indexed) | | `section_title` | string | Title of the section | ### report_corpus_ids_anonymized.parquet Papers cited in ScholarQA report sections. This represents the set of papers that ScholarQA retrieved and cited when generating a report. | Column | Type | Description | |--------|------|-------------| | `thread_id` | string | Hashed identifier for the conversation thread | | `corpus_id` | int | Semantic Scholar corpus ID of the cited paper | ### pf_shown_results_anonymized.parquet Papers shown in Paper Finder search results. Each row represents a paper that was displayed to the user at a specific position in the results list. | Column | Type | Description | |--------|------|-------------| | `thread_id` | string | Hashed identifier for the conversation thread | | `query_ts` | timestamp | When the query was submitted | | `thread_pf_query_num` | int | Query number within the thread (1 = first query) | | `result_position` | int | Position in the results list (0-indexed) | | `corpus_id` | int | Semantic Scholar corpus ID of the shown paper | ## Joining Datasets All datasets can be joined using `thread_id` as the primary key. The hashed thread IDs are consistent across all tables, enabling the same joins as the original data. Common analyses include: - **Clicked sections**: Join `section_expansions_anonymized` with `report_section_titles_anonymized` on `thread_id` and `section_id`/`section_idx` to see which section titles users clicked - **Clicked citations**: Join `s2_link_clicks_anonymized` (filtered to `tool='sqa'`) with `report_corpus_ids_anonymized` on `thread_id` and `corpus_id` to identify which cited papers users clicked - **Paper Finder click-through**: Join `pf_shown_results_anonymized` with `s2_link_clicks_anonymized` (filtered to `tool='pf'`) on `thread_id` and `corpus_id` to compute click-through rates by position ## Citation [TODO: Add citation information] ## License This dataset is licensed under ODC-BY. It is intended for research and educational use in accordance with Ai2's [Responsible Use Guidelines](https://allenai.org/responsible-use).