Self-Assessment Reinforcement Learning for Search-State Awareness in Retrieval-Augmented Large Language Models

Key points

• When a search agent fails to recognize what information it has obtained and what is still missing, two failure modes appear: hallucinated reasoning and prematurely terminated search.
• SARL appends a self-assessment phase after the think-search-answer loop, so the model re-examines its own trajectory in an isolated context and judges whether it is complete.
• The model is trained with a dual reward that rewards assessments by whether they corrected the answer and adds an exponential penalty for repeated reliance on assessment.
• Across ten benchmarks SARL scored 0.429 on average, a 24.3% relative improvement over the strongest baseline (ASearcher 0.345), with average gains of 35.9% on multi-hop reasoning and 27.9% on deep research.
• It recorded peak improvements of 58.1% on Musique and 48.3% on FRAMES, showing transfer to tasks left out of training.
• Under extended search horizons of up to 32 turns, existing methods degraded during training while SARL maintained stability.

Paper links

External research summaries. These are not HDATF publications or measured product results.