Scientist World: Grow a Research World for AI-Scientists
- Published
- Source
- arXiv
- Paper number
- 1444
- Field
- Research Agents
- arXiv ID
- 2611.08478
Key points
- A multi-agent pipeline with no human in the loop converts papers into self-contained environments, and the team released the first 1,000 verified environments where each reproduced method serves as the scoring baseline.
- An AI-Scientist is scored by Frontier-Normalized Improvement (FNI) over the paper’s reproduced method, and frontier AI-Scientists surpassed the human researchers 74% of the time.
- The environments preserve 6,400 hours of research traces, recording every idea, experiment, failure, and success.
- Training a small model with only real experimental outcomes as rewards raised its research ideation by 158%, matching the frontier cohort on the median held-out environment.
- The 1,000 papers were selected from 124,816 indexed papers by quality and compute feasibility, and the world keeps absorbing new literature every day.
Paper links
External research summaries. These are not HDATF publications or measured product results.