Discovering Cooperative Pipelines: Autoresearch for Sequential Social Dilemmas

Key points

• Problem: Iterative LLM policy synthesis, where an LLM writes a Python policy, evaluates it in self-play and refines it, works on Sequential Social Dilemmas (multi-agent games where individually rational play hurts the group), but its pipeline of system prompt, feedback, helper functions and refinement steps was tuned by hand. The author asks whether an AI agent can design that pipeline.
• Input and output: This is a method paper that proposes and tests a two-level autoresearch framework. The outer researcher agent (Claude Opus 4.6 run as a coding agent) reads the inner pipeline's source code, the experiment history and the read-only environment code, and outputs a new configuration that edits the system prompt, feedback function, helper library and iteration logic. The inner policy-synthesis LLM then produces one Python policy run by all agents, and that policy's welfare score on held-out seeds goes back to the outer loop.
• Method: In each outer iteration the researcher edits the running-best configuration and runs the inner loop (synthesis, AST safety check and smoke test, self-play evaluation, feedback construction), keeping the change only if the score beats the best so far and reverting it with git otherwise. The validation pipeline, environment simulator and evaluation stay frozen, so the researcher can change only the prompt, feedback, helpers and iteration logic, a design meant to prevent reward hacking. The welfare objective is either utilitarian efficiency U or Rawlsian maximin (the return of the worst-off agent), and the author reads the setup as information design, where the researcher chooses what to show the synthesizer.
• Implementation: The experiments are 12 autonomous researcher runs on two gridworlds, Cleanup (public goods, 10 agents) and Gathering (common pool resource, 4 agents), with Gemini 3.1 Pro and Claude Sonnet 4.6 as policy synthesizers. The 12 runs took 62.2 hours in total, of which the researcher agent used 25.6 hours (41%). According to the official repository README, the release includes the inner-loop code (llm_self_play.py, run_inner_loop.py), the researcher-editable pipeline/ folder, the autoresearch/ folder with the outer-loop scripts (measure.sh, run_experiment.sh and the researcher instructions program.md), and a GEPA baseline runner.
• Results: On Cleanup the researcher raised efficiency U from baselines of 1.93 (Gemini) and 0.86 (Sonnet) to about 3.1 to 3.2, and on Gathering all four runs, starting from baselines between 0.03 and 2.42, converged to 2.47 to 2.52. At matched environment queries it beat prompt-only GEPA by 2 to 3 times on Cleanup and by 20% on Gathering. All 4 maximin-optimized Cleanup runs inserted an explicit fairness mechanism such as time-based role rotation, and none of the 4 efficiency-optimized runs did.
• Limits: The author names the single researcher model (Claude Opus 4.6) as the main limit and calls an ablation across other coding agents the most important follow-up. The validation pipeline, helper-library skeleton and orchestrator API are engineered artifacts, and generalization to settings without such a harness is stated as unevaluated. The environments are fully observed 2D gridworlds with discrete actions and 1000-step episodes, so whether the discovered mechanisms (BFS-Voronoi partitioning, time-rotated roles) transfer to continuous or partially observable settings remains open.

Paper links

External research summaries. These are not HDATF publications or measured product results.