RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

Key points

• The paper formally identifies that automatically improving an agent harness leads to 'overfitting': scores rise only on the benchmarks used during evolution and the gains disappear on new benchmarks.
• The proposal side is regularized: the number of edits bundled at once is reduced over time (only smaller and smaller edits are allowed), and when progress stalls the search is nudged toward unexplored parts of the harness.
• The selection side is regularized: a critic filters out benchmark-specific logic, and a pruner removes changes that are too trivial, too expensive, or no longer useful.
• Across eight benchmarks spanning coding, agentic workspace tasks, and engineering design, scores on previously unseen benchmarks rose by up to 4.7 points, beating the average of four prior automatic evolution methods by up to 22.9%.
• Thanks to regularization, the final harness uses 30% fewer tokens; it consumes 58% fewer tokens than the most expensive prior method (AHE) while scoring higher.
• The improvements held on both Gemini and Claude models, and on a smaller model never used during evolution, showing the harness learned reusable principles rather than memorized tricks.

Paper links

External research summaries. These are not HDATF publications or measured product results.