LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks

Key points

• It identifies three main reasons for failure on long tasks: goals drift as errors accumulate, important information becomes hard to find as the record grows longer, and the system loses track of what it has done so far.
• Its core structure is a Manage-Execute-Audit loop. The manager cannot access the environment directly and sees only state and audit reports; the executor does one subtask at a time in a fresh context; and the auditor checks the environment directly with read-only tools.
• The only memory passed between rounds is the audit report, while the executor's raw trace is discarded. Termination happens when the audited state satisfies the original goal, when there are no more subtasks to do, when user input is needed, or when the round budget is exhausted.
• A thin AgentAdapter swaps in a different backend without changing the original agent loop. It can mix harnesses such as Claude Code, Codex CLI, OpenClaw, and Hermes Agent, and it can mix models such as Claude Opus, GPT, and Qwen across the three roles.
• On the combination of Qwen 3.7-Plus and Claude Code, the pass rate on WeaveBench rises from 51.8 percent to 80.7 percent, which is almost twice the 41.2 percent of the official best-reported Claude Opus 4.7 combination.
• Across the full OSWorld 2.0 benchmark, the binary completion rate rises from 2.8 percent to 8.3 percent, about a threefold gain, and on 34 subsets it rises from 20.0 percent to 34.3 percent relative to Claude Opus 4.7.
• In the case study, the baseline method gets 0.00 because it directly edits document XML and only makes the output look right, while the proposed method follows the specified GUI procedure and gets 0.89 after the auditor reparses the XML and checks 15 title styles.

Paper links

External research summaries. These are not HDATF publications or measured product results.