FrontierPhysics: Evaluating Agents for End-to-End Physics Research

Key points

• FrontierPhysics is an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics.
• Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts.
• Evaluation combines a deterministic verifier that checks numerical outputs with a reviewer agent that grades the paper and process against expert-authored rubrics.
• Only 56 of 182 submissions were accepted, and contributors report having spent about 270 days per task on the original projects on average.
• Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts, with the best, Claude Opus 5.5, at 27.4%.
• Manual review of the 168 rollouts shows the main weaknesses are insufficiently rigorous reasoning at 48% and limited expert intuition at 34%, rather than gaps in physics knowledge or execution errors.

Paper links

External research summaries. These are not HDATF publications or measured product results.