Aligning Autoresearch: Co-Improvement between Agents and Humans

Key points

In AutoBenchmark, benchmarks that the research agent built without human feedback were easy enough for solvers to score above 80. With detailed human direction, solver scores dropped by half.
Fine-grained human feedback lowered solver scores by 28 to 46.5 points in every one of the six benchmark-solver pairs, and this difficulty transferred to held-out solvers that were never used in the loop.
HackBench was built from 169 real trajectories, where the median number of commands per trajectory is 264 while the median number of hacked commands is only 1. The best monitor, Claude Opus 5, reached only 64.88 F1 at hack localization.
AutoInteract builds multi-turn interaction data from user models learned on real histories and trains with reinforcement learning, raising average Pass@1 on LiveCodeBench-Pro from 29.9 to 43.0 at the 4B scale and from 49.1 to 55.7 at 27B, with multi-turn gains from 23.0 to 43.2.
In the RL-XAR experiments, standard LLM judges preferred model writing over expert human writing in 63.5% to 84.6% of cases. Meta-optimizing rubrics on expert writing reversed this, and training lifted story writing from 2.8 to 8.2.
As a limitation, on Wikipedia section writing RL-XAR moved the score only from 2.7 to 4.0, far below humans and frontier models, and the human evaluation was a small preliminary study by the authors.

External research summaries. These are not HDATF publications or measured product results.