When Does Experience Memory Help Closed-Loop LLM Driving Agents? A Controlled Evaluation with Resampled Warmup Pools, Token-Matched Fillers, and Audited Baselines
- Published
- Source
- arXiv
- Paper number
- 1358
- Field
- Agents / Memory
- arXiv ID
- 2610.27714
Key points
- The authors froze the planner, encoder, and protocol, resampled the warmup pool as ten disjoint draws, and added token-matched controls to build a controlled evaluation framework.
- Each retrieval memory exceeded No-Memory in exactly one pool of ten (exact sign test p = 0.011), and the pool-averaged effect of adding memory was negative at −0.20.
- A token-matched irrelevant-filler control scored 0.800 on the merge task, significantly above both memories, and a single 15-token caution instruction reached 0.980.
- The temporal-graph memory TG-Mem was indistinguishable from flat text memory (p=1.0) while using 1,038 prompt tokens per decision against 210 to 274.
- Baseline port quality silently inverted conclusions: a subtly crippled DiLu port scored 0.375 against 0.670 for the audited port under the same conditions.
- As a limitation, every decisive contrast came from HighwayEnv with gpt-4o-mini and the 1 Hz decision cadence, a weak-planner regime.
Paper links
External research summaries. These are not HDATF publications or measured product results.