Activation-Based Active Learning for In-Context Learning: Challenges and Insights
- Published
- Source
- arXiv
- Paper number
- 319
- Field
- LLMs / NLP
- arXiv ID
- 2606.05134
Key points
- However, this paper reports a negative result: MLP outputs, viewed through the lens of large activations or the first four moments, do not correlate with example quality or task performance.
- Specifically, across all tasks and models tested, the absolute Spearman correlation coefficient reached only 0.33 at most, showing that activation-based sampling should not be used for in-context learning.
- The authors hypothesize that this may be due to superposition, the phenomenon where models represent more features than they have dimensions, which suggests methods such as Sparse Autoencoders (SAEs) may be a promising direction.
Paper links
External research summaries. These are not HDATF publications or measured product results.