Recursive Language Models
- Published
- Source
- arXiv
- Paper number
- 100
- Field
- Architecture
- arXiv ID
- 2512.24601
Key points
- Large language models (LLMs) are constrained by finite context windows, which makes them difficult to apply to many real long-horizon tasks that can involve millions of tokens.
- LLMs exhibit context rot, meaning performance degrades as input length grows even within a fixed context window, especially on tasks that require dense information access.
- Existing approaches to long-context management, such as compression, summarization, or task-specific agent systems, are often inadequate because they lose information, generalize poorly, or still run into the core context-window limit of LLMs.
- Instead of feeding an arbitrarily long user prompt directly into the LLM context window, the method treats it as an external object inside a persistent REPL (Read-Eval-Print Loop) programming environment.
- It lets the LLM generate code inside the REPL to inspect, transform, decompose, and build intermediate values from the external prompt programmatically.
- By enabling symbolic recursion in code, the method can call sub-RLMs on programmatically constructed prompt transformations, bypassing output-length limits and enabling complex iterative processing.
Paper links
External research summaries. These are not HDATF publications or measured product results.