TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
- Published
- Source
- arXiv
- Paper number
- 325
- Field
- Machine Learning
- arXiv ID
- 2606.06494
Key points
- It designs low-rank updates by using the singular-value basis, U and V, of the pretrained weights as a fixed reference frame.
- A soft spectral penalty suppresses updates along principal-component directions to reduce interference between tasks.
- Fine-grained adaptation is routed into the long-tail spectral coordinates that have higher flexibility.
- It improves continual-learning performance for PEFT methods based on spectral decomposition.
Paper links
External research summaries. These are not HDATF publications or measured product results.