Kosmos: An AI Scientist for Autonomous Discovery

Key points

• Problem: Data-driven discovery needs repeated cycles of literature search, hypothesis generation, and data analysis, yet earlier AI research agents could take only a limited number of actions before losing coherence, which limited the depth of their findings. The authors also point out gaps in prior systems. Robin shares limited context between its agents and is primarily tailored to therapeutics development, Sakana's AI Scientist stays within machine learning research, and Google's AI co-scientist generates hypotheses through iterative reasoning but does not perform or analyze experiments.
• Input and output: It takes a research objective and a dataset specified by a scientist. When it judges the objective complete, it synthesizes the key discoveries into three or four scientific reports, in which every statement and figure cites either a paper found by the literature search agent or a Jupyter notebook created by the data analysis agent. This is a system paper on an implemented AI scientist, supported by seven discovery case studies (three reproducing findings from existing manuscripts and four new contributions) and expert evaluations.
• Method: In each cycle it launches several parallel instances of two general-purpose Edison Scientific agents, a data analysis agent and a literature search agent, and assigns each instance a task aligned with the end objective. After up to 10 literature search and analysis tasks in a cycle, it updates a structured world model (a knowledge structure the agents share) with summaries of the task outputs and queries the world model to propose the next cycle's tasks. The authors state that this context management lets Kosmos run eight times as many iterations as existing systems.
• Implementation: In an average run, Kosmos executed 42,000 lines of code across 166 data analysis agent rollouts and read 1,500 full-length papers across 36 literature review agent rollouts, a 9.8x increase in code generation over Robin. The discovery reports are released as PDF supplementary data and on the Edison platform. The README of the official EdisonScientific/kosmos-figures repository describes its contents as the scripts for the plots in figures 2 to 8 and the human validation analysis, and the top level holds one folder per figure.
• Results: When experts checked 102 statements drawn from three representative reports, with the original code and cited papers withheld, 79.4% were accurate, including 85.5% of data analysis statements, 82.1% of literature review statements, and 57.9% of interpretation (synthesis) statements. Collaborating academic groups estimated that the findings of a 20-cycle run would have taken them 6.14 months of research on average (n=7). In the case studies, Kosmos reproduced the log10 fold changes of the top 15 metabolites from a then unpublished metabolomics study with R2 = 0.998, and in a Mendelian randomization (MR) analysis of myocardial fibrosis, 31 of the 32 significant proteins found by Kosmos and by the human analysis overlapped (96.9%).
• Limits: The authors suggest that the low accuracy on interpretation statements is likely due to a propensity to conflate statistically significant results with scientifically valuable ones, and evaluators noted that Kosmos tends to make excessively strong claims. It can only manage datasets up to about 5GB, does not currently excel at raw data such as images or raw sequencing files, and cannot autonomously access public external data for reference or orthogonal validation. Because it is stochastic, independent runs may not consistently converge on the same discoveries, its research directions are sensitive to how the objective is phrased, and the current implementation does not let scientists interact with it in intermediate cycles.

Paper links

External research summaries. These are not HDATF publications or measured product results.