PAPERBENCHX: Can AI Reproduce Science Across Disciplines?
- Published
- Source
- arXiv
- Paper number
- 1398
- Field
- Research Agents
- arXiv ID
- 2610.39744
Key points
- PAPERBENCHX is a benchmark of 93 paper-reproduction tasks spanning 12 research areas.
- The tasks run in 10 domain-native scientific computing environments, including Ansys HFSS, Meep, PySCF/GPU4PySCF, AutoDock Vina, and Habitat.
- Every submission is rerun from scratch in a clean environment, and only regenerated evidence is scored against a hidden rubric rather than agent claims.
- Among 10 frontier agent configurations, the best, GPT-6-Astra, reached a 59.08% partial score and a 13.98% full rate.
- 71 tasks were not fully reproduced by any evaluated configuration.
- Average Modeling and Execution scores reached 62.73% and 61.83%, while Validation fell to 42.91%, making scientific validation the weakest stage.
Paper links
External research summaries. These are not HDATF publications or measured product results.