Artifacts of the paper "Tool Recommendation Using LLMs for Scientific Workflow Development: A Case Study of Galaxy and Nextflow"

This replication package accompanies the paper “Tool Recommendation UsingLLMs for Scientific Workflow Development: A Case Study of Galaxy and Nextflow.” The study investigates whether general-purpose large language models (LLMs)can recommend appropriate platform-specific components for scientific workflowdevelopment. We evaluate GPT-5.5, Gemini 3.6 Flash, and DeepSeek-V3 across tenworkflow tasks: five for Galaxy and five for Nextflow/nf-core. The package contains the materials required to inspect and reproduce the mainanalyses reported in the paper, including: • scientific workflow task specifications;• the structured prompting strategy and prompt template;• prompts provided to the evaluated LLMs;• LLM-generated Galaxy tool and nf-core module recommendations;• expert-curated reference sets containing essential analytical operations, platform-supported components, functional alternatives, optional components, and valid analytical routes;• component-level recommendation classifications;• task-level and model-level precision and completeness results;• recommendation-error classifications;• workflow-level compatibility assessments;• bootstrap confidence-interval analysis;• experienced-developer evaluation materials;• qualitative themes derived from participants’ open-ended responses; and• scripts and processed data used to produce the reported quantitative results. The experienced-developer evaluation involved 28 participants with at leasttwo years of relevant scientific workflow experience: 18 evaluated Galaxyrecommendations and 10 evaluated Nextflow recommendations. Participant-levelmaterials included in this package are anonymised and reported only whereconsistent with the study’s research-ethics and confidentiality requirements. Survey instruments: Galaxy:https://www.surveymonkey.ca/r/7TSCDSL Nextflow:https://www.surveymonkey.ca/r/Z7Y62TB The package is organised by study stage so that researchers can trace theprocess from workflow-task specification and prompting through model output,reference-based evaluation, workflow-level compatibility assessment, andexperienced-developer evaluation.