SkillSmith: Learning to Compose Parametric Skills and Textual Knowledge
- Published
- Source
- arXiv
- Paper number
- 789
- Field
- LLMs / NLP
- arXiv ID
- 2607.27497
Key points
- It treats model weights, or prefix cache, as a modality on par with text so that the LLM can read and reason over both together.
- SkillSmith takes source-task weights, text metadata, and a target description as input, and directly synthesizes new prefix weights tailored to the target.
- It builds a new synthetic dataset called Composite SNI to verify the effectiveness of combining text and weights on new tasks that require composing skills from two tasks.
- With 4B Gemma 3, it beats all baselines that use only text or only weights, and on MMLU-ProX it even beats methods based on downstream fine-tuning.
- It shows that retrieval-based methods can synthesize weights effectively in noisy environments even without exact source mapping.
Paper links
External research summaries. These are not HDATF publications or measured product results.