OmniScientist: Toward a Co-evolving Ecosystem of Human and AI Scientists

Key points

• Problem: The authors argue that existing AI Scientists predominantly formulate scientific discovery as a standalone search or optimization problem and leave unmodeled the collaborative mechanisms, contribution attribution, peer review and structured knowledge networks that real science relies on. As a result, current systems struggle to establish a research ecosystem or interact deeply with the human scientific community.
• Input and output: From a user's research topic or query it produces a Markdown literature review, idea proposals with research background, idea and a general implementation approach, experiment code and results, a LaTeX manuscript and review reports whose key judgments carry traceable citations. ScienceArena takes votes on anonymous pairwise comparisons of two models' answers to user-submitted research questions across six tracks and builds a leaderboard from Elo ratings that start at 1000 for every model. This is a system paper combining several modules, a protocol and an evaluation platform, and its evidence comes from per-module case studies and small-scale measurements.
• Method: The literature review is split among Planning, Literature and Writing agents; candidates from embedding-based Elasticsearch search are joined by papers linked through citations and references, and papers that pass relevance verification are expanded layer by layer in a breadth-first search (BFS) until a predefined depth or coverage is reached. Deep Ideation explores, adds and replaces neighboring keywords in a keyword co-occurrence network to refine proposals, logs keywords, proposals and evaluations per round in an Idea Stack, and receives feedback aligned with expert review standards from a critic model fine-tuned on reasoning produced by a prompt that simulates human reviewers' cognitive process. OSP treats humans and AI as the same Participant type, a central Hub records every protocol message, and each research artifact carries a ContributionLedger that lists participants' actions with timestamps in order.
• Implementation: The knowledge base links about 269 million OpenAlex metadata records with their citations, about 2.6 million arXiv PDF full texts, and 102,679 full-text papers from the top ten AI conferences over the past decade with 116,970 referenced baseline models and 68,316 related datasets into a graph of Paper, Author, Concept and Resource nodes, and Diagnose, Search, Normalization, Coding and Review agents keep refining it. The experiment stage selects candidates with a bi-encoder retriever fine-tuned on baseline and dataset self-descriptions combined with summarized citation contexts from other papers and an LLM reranker fine-tuned on citation chains linking papers, baselines and datasets, and then Evolution, Sample, Evaluation and Feedback agents modify and run code. Writing is handled by Outline, Figure, Writing and Refinement agents and review by the five-stage TIMAR (Traceable and Interactive Multi-Agent Review) workflow, and the public addresses given in the paper are omniscientist.ai and sciencearena.ai.
• Results: In a small-batch evaluation (n = 1000), graph refinement raised metadata completeness from 0.965 to 1.000 and correctness from 0.951 to 0.997, and on a benchmark of 100 QA pairs probing inter-paper relationships it raised retrieval accuracy from 0.70 to 0.88. OmniScientist's proposal to replace the Monte Carlo sampler of STDE (Stochastic Taylor Derivative Estimator) with a QMC (Quasi-Monte Carlo) sampler gave lower error on the AllenCahnTwobody equation than both the original STDE and the AlphaEvolve variant at all four settings from 100 to 100000 dimensions, for example 0.000572 against 0.003440 for STDE and 0.002059 for the AlphaEvolve variant at 10000 dimensions. In a study where 10 PhD-level participants each solved 10 computer science and AI questions from HLE, five alone and five with GPT-5, average accuracy was 0.22 with GPT-5 collaboration, 0.1 for humans alone and 0 for GPT-5 alone.
• Limits: The authors state that current capabilities remain constrained to the AI domain, and that the data backbone, primarily arXiv full texts, strongly overrepresents AI and computer science while Nature, Science and other domain-specific journals are not yet integrated. The experiment agent currently supports only computational workflows such as configuring environments, preparing datasets, running training and inference scripts and hyperparameter sweeps, so wet-lab experiments in chemistry or biology and instrument interaction are outside its current scope; the review module is optimized on AI manuscripts and loses quality on other fields, and the modules still require significant compute and processing time. The validation consists of case-level studies such as the STDE improvement on a single equation (AllenCahnTwobody), a collaboration study with 10 PhD-level participants and 10 HLE questions, and a literature review comparison scored by Gemini-2.5-pro, and ScienceArena results are presented as per-track preference patterns and example figures in place of vote counts or Elo values.

Paper links

External research summaries. These are not HDATF publications or measured product results.