AI scientists produce results without reasoning scientifically

Key points

• Problem: LLM-based scientific agents (AI scientists) increasingly run research on their own, yet benchmarks for them mainly score task completion, such as answering domain questions or finishing a workflow. Such scores show whether an agent succeeded but not how it got there or how it will behave on new problems, so the authors measure the reasoning process itself.
• Input and output: This is an empirical study. The inputs are tasks in 8 scientific domains: workflow tasks (molecular simulation, adsorption surface construction, ML-based property prediction, atomic force microscope experiments), a strategic task (retrosynthetic planning), and hypothesis-driven tasks (spectroscopic structure elucidation, inorganic qualitative analysis, circuit inference). The outputs are scores from more than 25,000 agent runs, diagnostic question scores, and graphs of reasoning traces made of nodes and edges.
• Method: The authors model an agent as a language decision process, a partially observable Markov decision process with text observations and actions, whose policy combines a base-model part and a scaffold part, and they cross 3 models (GPT-4o, Claude Sonnet 4.5, GPT-OSS-120B) with 2 scaffolds (ReAct and structured tool calling) on the same environments and tools. They fit item response theory (IRT) to per-domain knowledge and reasoning questions to estimate abilities, then split the variance in success with M7, chosen by PSIS-LOO (an approximation of leave-one-out cross-validation) among Bayesian models M1 to M8. Each trace step is labeled as hypothesis, evidence, test, judgment, update, or commitment, and the share of traces showing productive motifs and reasoning breakdowns is counted on the resulting graphs.
• Implementation: The authors built Corral, an evaluation framework that separates environments from agents through an HTTP interface, and its environments rest on LAMMPS simulations, a live atomic force microscope, a custom thermodynamic equilibrium engine, and manually curated spectra, covering more than 15 environment scopes and more than 90 tools. The paper states that tasks, run reports, traces, and question data are public in a Hugging Face collection. The official repository README describes a corral command-line interface (CLI) with run and bench commands and a Python API, the afm, corral_md, resistor_network, retrosynthesis, spectra_elucidation, and wetlab environments plus a lightweight math environment called samplemath, and agents such as Reflexion, LLMPlanner, and AI Scientist in addition to the ReAct and tool calling agents used in the paper.
• Results: In the selected M7 model, reasoning ability accounts for 41.4% of the explained variance and environment with scope for 30.1%, while the scaffold accounts for 1.5% and tool description verbosity for 0.1%. Gathered evidence goes unused in 68% of traces, untested claims appear in 53%, and refutation-driven belief revision appears in 26%. In spectroscopic structure elucidation, inorganic qualitative analysis, and retrosynthetic planning, injecting early successful steps does not raise performance, and gains appear only with near-complete successful trajectories.
• Limits: The authors treat each task as an independent episode, test two simple scaffolds (ReAct and tool calling), and leave advanced orchestration such as plan-then-execute pipelines, hierarchical planners, and multi-agent setups out of scope. They use 3 models, and the main evaluation fixes a budget of 20 to 40 LLM calls per task and temperature 0.0, and the log-probability analysis covers only GPT-OSS-120B. GPT-OSS-120B produced malformed responses in 41% of traces, and the authors measured them without fixes such as constrained decoding.

Paper links

External research summaries. These are not HDATF publications or measured product results.