Reproducibility package for the evaluation of SciSuite

## Abstract This record contains the reproducibility package for the evaluation of SciSuite, a platform for computational reproducibility and research data management with a hybrid conversational and structured user interface. The package includes anonymised user-study data, SUS and NASA-TLX results, task-completion observations, analysis scripts, study materials, and the task datasets used during the evaluation. The scripts reproduce the descriptive statistics, Wilcoxon signed-rank tests, and figures reported for the evaluation. Signed consent forms, personal identifiers, API keys, Zenodo tokens, OpenAI credentials, and platform credentials are not included. ## Package contents The uploaded ZIP file is organised as follows: ### 1. Analysis scripts This folder contains the Python scripts used to analyse the study results. - `analyze_results.py`: computes descriptive statistics and Wilcoxon signed-rank tests for SUS and NASA-TLX scores.- `make_figures.py`: generates the figures used to visualise the SUS and NASA-TLX results.- `requirements.txt`: lists the Python dependencies needed to run the analysis. ### 2. Study data This folder contains the anonymised quantitative data collected during the evaluation. It includes: - participant-level SUS scores;- participant-level NASA-TLX workload scores;- task-completion observations;- approximate task-time observations;- condition labels for SciSuite and the manual Zenodo workflow. In the data files, the label `sciconv` refers to the SciSuite condition. This label was kept because it was the internal name used during data collection and analysis. ### 3. Study materials This folder contains the materials used during the user study, including the consent form template, tutorials, questionnaires, and observation sheet. These materials document the study procedure and allow the evaluation protocol to be inspected or repeated. ### 4. Task datasets This folder contains the datasets used in the evaluation task: - `weather_large.zip`: the weather-observation dataset;- `birds_large.zip`: the biodiversity/bird-observation dataset. Each participant published one dataset with SciSuite and the other dataset manually through Zenodo. The dataset-condition pairing was counterbalanced across participants. ## Running the analysis To reproduce the analysis, first extract the ZIP file, then install the required Python packages: cd 1-analysis-scripts pip install -r requirements.txt Run the statistical analysis: python analyze_results.py Run the figure-generation script: python make_figures.py The scripts reproduce the main quantitative results of the evaluation, including the SUS and NASA-TLX descriptive statistics and the Wilcoxon signed-rank tests. ## Software availability The SciSuite source code is archived separately as a software record and maintained on GitHub. GitHub repository: https://github.com/anex4real/Scisuite Software DOI: https://doi.org/10.5281/zenodo.21284348 Version used for the evaluation: v1.0.0 This Zenodo record contains the study reproducibility package. The full platform source code and installation instructions are provided separately in the software repository. ## Notes on privacy and reproducibility The package contains only anonymised study data. Consent forms signed by participants, personal identifiers, API keys, credentials, and private deployment details are excluded. The task datasets were prepared for the evaluation and do not contain personal or sensitive information.