ARE-bench: Benchmarking Agents'Research Ability in Real-World Researcher Sessions and Workspaces
- Published
- Source
- arXiv
- Paper number
- 1424
- Field
- Research Agents
- arXiv ID
- 2611.14364
Key points
- ARE-bench is a 75-task benchmark that evaluates agents' research ability on real researcher sessions and in-progress workspaces.
- Each task is anchored at a real user request and the workspace is replayed to that boundary without cleanup, preserving artifacts from earlier turns.
- The benchmark covers seven research stages, 33 requirements, 150 sub-requirements, and 1,173 binary checklist items.
- Among seven frontier models, the strongest passes every checklist item on only 66.2% of tasks.
- Design and Execution rank among the hardest stages for every model.
- Three models pass about 69% of tasks under delivery contracts and general norms alone, but drop to between 28.44% and 40.00% once historical and workspace requirements are also checked.
Paper links
External research summaries. These are not HDATF publications or measured product results.