ParamMem: Augmenting Language Agents with Parametric Reflective Memory

Key points

• Existing self-reflection mechanisms in language agents often generate repetitive, inaccurate, or unhelpful outputs, undermining their ability to explore diverse solutions and diagnose errors.
• Existing retrieval-based approaches for improving reflective diversity, such as DoT-bank, are limited in their ability to capture compositional patterns and are vulnerable to embedding collapse, which reduces retrieval effectiveness.
• Empirical evidence shows a strong positive correlation between reflective diversity and task success in language agents, with a mean Pearson coefficient of 0.76.
• ParamMem builds a parametric memory module by fine-tuning a lightweight LLM, such as LLaMA-3.1-8B with LoRA, on an auxiliary dataset of input samples paired with expert-generated global reflections.
• The fine-tuned ParamMem module generates diverse reflections through temperature-controlled sampling and directly encodes cross-sample patterns into its parameters.
• ParamMem is integrated into agent frameworks, including ParamAgent and ParamAgent-plus, to combine this parametric memory with existing episodic memory and, in the case of ParamAgent-plus, with cross-sample memory such as DoT-bank to condition the actor LLM.

Paper links

External research summaries. These are not HDATF publications or measured product results.