EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery

Key points

• Problem: Most current AI scientist systems use static, hand-designed pipelines, and their idea-generation and code-generation strategies stay the same regardless of accumulated interaction histories. The authors point out that such systems overlook promising research directions, repeat failed experiments, and spend resources on infeasible ideas.
• Input and output: This is a system paper with an implementation and experiments. The input is a user's research goal sentence; the outputs are a research proposal (background, related work, method, experimental plan, expected results) and an execution report with the code that was run, logs, metrics, and failure diagnoses. After each task, two memories are updated and reused on the next goal.
• Method: The Researcher Agent retrieves related direction knowledge from the ideation memory, builds candidates with an idea tree search that repeats propose, review, and refine, picks the winner of an Elo-rating tournament (pairwise comparisons turned into a ranking score), and expands it into a proposal. The Engineer Agent uses strategies from the experimentation memory to write and run code in four stages (initial implementation, hyperparameter tuning, proposed method, ablation) and fixes failures from the logs. The Evolution Manager Agent writes promising directions from the top 3 ideas, failed directions from failed proposals, and data-processing and training strategies from code search trajectories into the two memories.
• Implementation: Gemini-2.5-Pro generates ideas, Claude-4.5-Haiku generates code, the Semantic Scholar API (a paper search service) finds literature, and memory retrieval uses the mxbai-embed-large embedding run through Ollama (a local model runner). The idea tree search makes at most 21 candidates, and the maximum attempts for the four experiment stages are 20, 12, 12, and 18. The official repository README presents a research agent tool installed from PyPI (the Python package index) and used through a CLI (command-line interface), a TUI (text interface inside the terminal), or a WebUI (browser interface), built from 6 sub-agents (plan, research, code, debug, analyze, write); this setup differs from the three agents in the paper, and the roadmap still lists the benchmark suite as to be released.
• Results: Judged by Gemini-3-flash, EvoScientist beat all 4 open-source and 3 commercial systems, with an average gap over the four criteria (Avg. gap) of +29.17 to +93.34 against open-source systems and +46.00 to +80.83 against commercial ones. With 3 PhD-level human judges it reached an average novelty win rate of 82.50% and a feasibility win rate of 64.17% across four comparisons, and code execution success averaged over four stages rose from 34.39 to 44.56 after experiment strategy evolution (distilling execution strategies from code search trajectories into memory). All 6 papers the system produced were accepted at the ICAIS 2025 AI Scientist Track, which accepted 26 of 82 submissions.
• Limits: The authors state that the evaluation covers computational research tasks whose hypotheses can be tested by simulation and code execution, and that extending to fields that need physical experiments, such as materials science or drug discovery, requires lab workflows and real-world feedback. Execution success in stage 3 (proposed method) rose only from 20.33 to 21.57, and idea quality was judged on 30 queries from researchers. Reviews of the accepted papers repeatedly asked for deeper theoretical formalization, and the manuscripts were written with the paper-writing module of AI Scientist-v2.

Paper links

External research summaries. These are not HDATF publications or measured product results.