AI for Auto-Research: Roadmap & User Guide

Key points

• Problem: AI now produces research artifacts quickly, with The AI Scientist generating a full paper for about $15, but its ability to verify that those artifacts are novel, faithful, and executable lags far behind. The authors argue that single-stage tools have multiplied while a unified analysis of the whole research lifecycle is missing, which makes it hard to tell where AI reliably helps and where it fails.
• Scope: This is a survey of AI for research automation covering developments through June 2026. The authors collected tools, methods, and benchmarks through keyword search on Google Scholar, Semantic Scholar, arXiv, and DBLP, citation tracing, and monitoring of open-source repositories, keeping only work that targets at least one research stage, is publicly accessible, and gives enough detail for analysis. The outputs are a taxonomy, a benchmark table organized by stage, stage-by-stage tool inventories, and cross-stage design principles.
• Taxonomy: The research lifecycle is split into four phases and eight stages. Creation covers idea generation, literature review, coding and experiments, and tables and figures; Writing covers paper writing; Validation covers peer review plus rebuttal and revision; Dissemination covers posters, slides, videos, social media, project pages, and interactive paper agents. Methods are grouped into five families (prompt engineering, retrieval-augmented generation, training-free agentic methods, training-based methods, and hybrids), and a table maps the dominant families to each stage.
• Implementation: What the authors released is a GitHub paper list. The README of the official GitHub repository worldbench/awesome-ai-auto-research tabulates papers by stage with venue, website, and GitHub links, and the top level holds LICENSE, README.md, docs, and index.html. The README states a last update of 2026-10-11 and already includes papers released after the v2 manuscript.
• Evidence: Among the results the authors collect, frontier systems exceed 76% on SWE-bench Verified, while the best model reaches only 37.3% on ResearchCodeBench, a set of 212 novel ML research-code tasks. Standalone LLM reviewers give inflated scores (6.86 vs. 5.70 for humans) and classify 95.8% of rejected papers as acceptable. In a randomized ICLR 2025 study of 22,467 reviews, LLM feedback on human reviews improved review quality in 89% of cases while acceptance rates stayed the same.
• Limits: The authors state that the paper is a snapshot up to their search cutoff and that AI-generated research outputs should be independently verified before scholarly use. The collected work concentrates on the Creation phase, and most systems and benchmarks come from computer science, machine learning, and NLP, leaving generalization to fields such as chemistry, biology, and medicine as an open problem. They also note that systems are evaluated with different base models, prompts, tools, datasets, and compute budgets, which makes reported results hard to compare.

Paper links

External research summaries. These are not HDATF publications or measured product results.