ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration

Key points

• Problem: The authors see plausible unsupported success as the central failure in long-horizon research workflows, where results may be misreported or claims may outrun their evidence. Among existing systems such as AI Scientist, AI Scientist v2, and Agent Laboratory, many use the same or closely related model families for execution and review, which can leave correlated errors uncaught, their workflows are tightly coupled end to end, which makes stages hard to replace or resume, and few provide explicit system-level checks on experimental integrity and manuscript quality.
• Input and output: The full pipeline takes a research direction and moves through a ranked idea report (IDEA_REPORT.md), an experiment log (EXPERIMENT_LOG.md), and a narrative report to a compiled paper PDF, and the rebuttal workflow takes a paper with its reviews and returns a paste-ready rebuttal. Skills pass results to each other through such plain-text files, so work can resume from saved artifacts. This is a technical report on the design and early deployment of an implemented open-source system, and a controlled comparative evaluation is left for future work.
• Method: The executor produces an artifact such as code, a manuscript section, or an experiment design, a reviewer that the recommended configuration draws from a different model family returns a rubric score and action items, and the loop ends when the score exceeds a threshold (default 6/10) and all critical items are resolved, or when it reaches the maximum number of rounds (default 4). The reviewer receives file paths instead of the executor's summary and reads the artifacts directly, with a configurable access scope (document only, plus result files, or the whole repository) and context policy (a fresh thread each round or memory across rounds). Auditing runs in three stages: an integrity audit in which a cross-model reviewer examines the evaluation code and outputs (model-derived reference labels, self-normalized scores, numbers that differ from the actual result files, and more), a claim ledger that rates results as supported, partial, or invalidated, and a fresh zero-context reviewer that checks the manuscript's numbers against the raw result files.
• Implementation: As of v0.4 (April 2026), ARIS (Autonomous Research via Adversarial Multi-Agent Collaboration) has more than 65 Markdown SKILL.md skills, 5 workflows, and 6 MCP (Model Context Protocol) bridges (Codex, Oracle, Claude, Gemini, MiniMax, and llm-chat), it was tested on 3 executors (Claude Code, Codex CLI, and Cursor), and its research wiki builds a lightweight knowledge graph from four entity types (papers, ideas, experiments, and claims) with 8 relationship types. The default pairing is a Claude-family executor with a GPT-family reviewer, or the reverse, with reviews in the current implementation routed to GPT-5.4 through Codex MCP, and the system includes effort presets (lite, balanced, max, beast), a FigureSpec renderer that turns JSON into SVG (Scalable Vector Graphics) figures, a standalone Rust command-line interface (CLI) called ARIS-Code, and a prototype meta-optimize loop that proposes SKILL.md patches from usage logs and shows the user only those the reviewer scores at least 7/10. Per its README, the official GitHub repository wanshuiyin/Auto-claude-code-research-in-sleep contains skills, mcp-servers, tools, and templates folders, installer scripts, and Claude Code and Codex CLI plugins, and the README describes a later state that adds Workflows 5 to 7 and new reviewer bridges.
• Results: Instead of a controlled comparison, the authors document one overnight run in which, over about eight hours, the system completed 4 review and revision rounds, launched more than 20 GPU experiments, and removed unsupported claims while the internal reviewer score rose from 5.0 to 7.5/10. For the deployment footprint, they report growth from 21 core skills at release to more than 65, with 4 GPU backends (local, SSH (Secure Shell) remote servers, Vast.ai, Modal), 9 venue template families, and 30 or more community-contributed skills. The authors state that the reported outcomes are observational and cannot be causally attributed to ARIS alone, and that this single trajectory should not be read as evidence that cross-family review beats same-family review.
• Limits: The authors state that ARIS cannot guarantee that any output is correct, novel, or scientifically sound, and that the three-stage audit can catch common integrity failures but is an advisory safety net, distinct from a formal verification system that would detect every error, inconsistency, or fabrication. The review loop can amplify reviewer biases by overfitting to the reviewer model's preferences, and over-iteration past diminishing returns can degrade paper quality. Repository-level review may send source code to external LLM APIs, so the authors advise against enabling it on repositories with sensitive code or secrets unless an approved local-only review path is available, and local-only reviewer routing is still planned.

Paper links

External research summaries. These are not HDATF publications or measured product results.