AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks

Key points

• Problem: The loop of trying, measuring and revising is mostly implemented in the scaffold, so the model is asked mainly for one attempt at a time. Agent training data is typically built by keeping only passing solutions, which discards intermediate attempts, their feedback and the revisions that followed, so hours-long improvement processes are almost absent from the data.
• Input and output: This is a method paper that presents a recipe for building training data and a model trained on it. In an environment with a task and a scoring function, the model submits a solution each round, receives feedback such as the score, logs and errors, and decides its next revision, and the best score so far is the result. Training trajectories come from machine learning tasks in GitHub repositories and algorithmic problems from online judges.
• Method: A teacher model turns each repository or problem into an environment with a task, a scoring script on hidden data and a sandbox, keeping only environments where the reference solution gets a score and a simple baseline scores far below it. A teacher agent, told its budget of hours and hundreds of tool calls, acquires operational knowledge by reading code and documentation or running small experiments, and rounds that lower the score stay in the trajectory. Trajectories are selected as a whole by final score and process rules, and the loss is applied only to decisions that move the solution forward.
• Implementation: The selected trajectories are mixed with the unchanged deep-research data of the earlier AREX recipe to fine-tune Qwen3.8-27B. Non-coding benchmarks cap inner turns at 300 and total turns at 1500, Frontier-CS uses 188 tasks with 5 hours each, and MLE-bench Lite uses 12 hours per task with skill documents in the context. The official repository README says it contains evaluation runners and dataset definitions for the six benchmarks and links a Hugging Face model.
• Results: AREX-2 scored an Any Medal rate of 81.8 on MLE-bench Lite, the highest among the compared models, and 70.7 on Frontier-CS, the highest among open-weight models, and in deep research it reached 84.0 on BrowseComp, 52.6 on text-only HLE, 92.2 on GAIA and 93.8 on DeepSearchQA, above both earlier AREX models (4B and 122B). On Frontier-CS its score kept rising, 54.4 after 1 hour, 65.9 after 2 hours and 70.7 after 5 hours, and on BrowseComp without correctness feedback it rose from 64.8 at an average of 47 turns per question to 84.0 at 143 turns. In the stage-wise ablation the base model scored 28.8, skills and more rounds raised it to 68.2, and AREX-2 under the same conditions reached 81.8.
• Limits: The authors state that a gap remains to the strongest BrowseComp and HLE results. The MLE-bench Lite score is measured with skill documents in the context, AREX-2 used the authors' optimized harness in the Frontier-CS time comparison, and its HLE score is on the text-only subset. Training covers two domains, machine learning engineering and algorithmic programming, and the authors name wider domains and longer horizons as next steps.

Paper links

External research summaries. These are not HDATF publications or measured product results.