Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
- Published
- Source
- arXiv
- Paper number
- 210
- Field
- AI / General
- arXiv ID
- 2605.22791
Key points
- On multi-key retrieval tasks, where several similar associations exist, Gated DeltaNet-2 showed a substantial advantage, confirming that separated gates help the model distinguish competing pieces of information within a fixed-size state.
- By selectively erasing specific channels, the model avoids overwriting or blurring existing stored data that may still be relevant when new information arrives.
- SWA handles local, fine-grained interactions within a small fixed window, such as the most recent 2,000 tokens.
Paper links
External research summaries. These are not HDATF publications or measured product results.