Stateless Language Agents: Scaling Long-Horizon Automated Research

Key points

• Problem: Automated research (AutoResearch) systems run LLM agents over long horizons, but agents replay growing histories, duplicate one another's work, or stop experimenting while token consumption continues. In one run of CORAL, a framework whose parallel agents select their own experiments and share discoveries through persistent memory, on a GPU kernel task, the last improvement came at 231M tokens, after which 98% of the final 1,500 sessions made no tool calls. Most evaluations use short budgets or benchmarks that saturate early and leave these failure modes untested, and the authors trace the failures to two choices: where research state lives and who decides what to try next.
• Input and output: SLA takes one task (a task adapter that specifies the editable files, permitted resources, and evaluation procedure, a seed solution, and a protected evaluator) and a cumulative token budget. Until the budget is exhausted, an independent evaluator scores every candidate the Workers produce, and the harness keeps the global best solution, each Worker's local candidate, the measured outcomes, and checkpoints as research state. This is a system paper that implements the framework and compares it with three recent frameworks on software engineering, kernel optimization, and algorithm design tasks.
• Method: The principle is stateful search with stateless agents: every agent starts a new session at each invocation, while the harness owns the research state (candidate solutions and measured outcomes) and rebuilds a fresh, role-specific context for every invocation. Each epoch, the Advisor receives the task specification, the current global best code and score, and a harness-built evidence summary that groups outcomes by search direction and separates measured non-improvements from failures in implementation, runtime, or correctness. Each Worker trial sees only the task specification, its assignment, its retained candidate, and compact local feedback, evidence from other directions reaches it only through the Advisor's assignments, and a redirected Worker restarts from the global best.
• Implementation: The Advisor and Workers use the same coding agent, in two configurations, OpenAI Codex CLI v0.152.1 with GPT-5.5 and Claude Code v2.1.258 with Claude Opus 4.8; in the main experiments they also use the same LLM, and Appendix C.1 separately reports a GPT-5.5 Advisor paired with GPT-5.4 mini Workers. The defaults are 3 Workers and three local trials per Worker per Advisor epoch, Workers run as unprivileged processes under Linux Landlock filesystem restrictions, and the harness clears saved sessions between trials and scores candidates outside Worker environments with protected evaluator code. The baselines are CORAL and SwarmResearch, whose agents keep long-running sessions, and EvoX adapted to launch a fresh coding agent for each generation; all three run from pinned upstream releases with unmodified search logic, and the same external cumulative-token counter stops every method.
• Results: On Anthropic kernel optimization with Codex (1B tokens, three runs per method), SLA averaged 1112.0 cycles against 1275.7 cycles for SwarmResearch, the strongest baseline by mean, and reached SwarmResearch's final performance with 67.9M tokens, compared with the 986.3M tokens SwarmResearch needed, a 93.1% reduction. On the four FrontierSWE tasks, run once per method with Codex, SLA scored 4.95 points higher on average than the strongest per-task baseline, trailing on all four at 175M tokens (25% of the budget) but leading on all four at the full 700M budget. In ablations that ran three 100M-token continuations from shared checkpoints on Anthropic kernel and FrontierSWE libexpat, removing any one of Advisor context reconstruction, Worker context isolation, or Advisor assignments reduced mean progress in every comparison, and in the two runs analyzed for token usage (Anthropic kernel and FrontierSWE libexpat) the Advisor used 0.24% to 0.51% of all tokens.
• Limits: Because long runs are costly, most configurations were run once, and only Anthropic kernel optimization with Codex and the checkpoint ablations were replicated. Comparisons are matched on tokens, and the cost estimates exclude the compute for running and evaluating experiments. All tasks have executable evaluators, so goals that are vague or slow to evaluate remain untested.

Paper links

External research summaries. These are not HDATF publications or measured product results.