A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents

Key points

• Problem: LLMs reason well in closed settings such as math and code benchmarks. Techniques that make intermediate reasoning explicit, such as Chain-of-Thought prompting, decomposition and program-aided solving, often assume static contexts and short-horizon reasoning, and conventional LLMs lack mechanisms to act, adapt or improve in open environments where information changes. Earlier surveys mostly cover reasoning inside the model or agent architecture, so the authors reorganize the field around reasoning as the principle that links planning, action and learning.
• Scope: This paper is a survey. It covers work up to 2025 on planning, tool use, search, feedback, memory and self-improvement in single-agent and multi-agent settings, and gives separate chapters to applications and benchmarks in math and coding, scientific discovery, embodied agents, healthcare, and web and research agents. The official repository (Awesome-Agentic-Reasoning) publishes a README with paper tables in the same taxonomy, a figs folder with figures and a materials folder with talk slides.
• Taxonomy: To characterize environmental dynamics, the survey uses three layers: a foundational layer (single-agent planning, tool use and search), a self-evolving layer (feedback, memory and adaptation) and a collective layer (role division, collaboration and co-evolution among multiple agents). Each layer is split into in-context methods, which keep parameters frozen and scale inference-time compute and orchestration, and post-training methods, which move successful reasoning patterns into the weights with RL and fine-tuning. The formalization factors the policy on a POMDP (partially observable Markov decision process) into internal reasoning and external action.
• Synthesis: Self-evolution is split by what changes: verbal (textual reflections and guidelines, Reflexion), procedural (a library of executable tools and skills, Voyager) or structural (the agent's own source code and architecture, AlphaEvolve). Feedback is split into reflective feedback (self-critique during inference), parametric adaptation (training feedback into parameters) and validator-driven feedback (resampling until a validator such as a unit test passes); the survey notes that the last kind is non-diagnostic, so it corrects individual outputs and leaves the reasoning unchanged.
• Applications and benchmarks: For research agents, the survey covers systems such as The AI Scientist, which chains idea generation, code writing, experiment execution and result visualization, and CycleResearcher, where a reviewer agent's automated peer-review feedback and an iterative preference-training loop help the research agent refine its next drafts. Benchmarks are grouped into mechanism-level suites (tool use, search, memory and planning, multi-agent) and application-level suites, with examples such as ToolLLM-ToolBench, which collects 16,464 real-world APIs across 49 categories.
• Limits: The authors list open problems in personalization, credit assignment across tokens, tool calls, skills and memory updates over long interactions, training and evaluating world models in non-stationary environments, multi-agent collaborative training, auditable latent reasoning, and governance for real deployment. They state that existing benchmarks and guardrails mostly test short-horizon behavior and that post-training of multi-agent memory remains largely unexplored.

Paper links

External research summaries. These are not HDATF publications or measured product results.