What Makes Agents Effective for Scientific Design? A Controlled Evaluation with Scientific Foundation Model Tools

Data drop and source code for What Makes Agents Effective for Scientific Design? A Controlled Evaluation with Scientific Foundation Model Tools File Contents analysis.tar.gz Source code for replicating the paper's analysis and figure generation rollouts.tar.gz Agent rollout directories plus supporting data/ files results.tar.gz Generated figures, tables and posterior draws for fitted hurdle models nomad.tar.gz Nomad source code used for this work ursa.tar.gz URSA source code used for this work heat-pli-design.tar.gz The Nomad adaptor for the JANUS HEAT PLI surrogate model. pli_validation.tar.gz AI / NSGA-II Generated PLI QoIs as computed by PAGOSA. Reproducing Results Install Julia v1.12.7, uv and download the above archives. Extract tar -xzf analysis.tar.gz change directories to it's root then: # Extract the archives tar -xzf rollouts.tar.gz tar -xzf pli_validation.tar.gz tar -xzf nomad.tar.gz tar -xzf ursa.tar.gz tar -xzf heat-pli-design.tar.gz tar -xzf results.tar.gz # Setup the environment uv sync uv run julia --project -e 'using Pkg; Pkg.instantiate()' # Run the analysis ./scripts/analysis.jl