A Mechanistic Interpretability Study of an Astronomy Foundation Model
- Published
- Source
- arXiv
- Paper number
- 1488
- Field
- Interpretability
- arXiv ID
- 2611.15653
Key points
- We present, to our knowledge, the first causal mechanistic interpretability study of an astronomy foundation model, AION-1, focusing on its multimodal advantage in redshift prediction, where combining imaging with photometry outperforms photometry alone.
- We find that this advantage is largest for spatially extended and dust-attenuated galaxies, and that most of it is localized to the first of the encoder’s twelve blocks.
- Spatial extent is causally used, and one attention head dominates linear probing for it, yet ablating that head has negligible effect on redshift prediction: the information is distributed across heads and is decodable again one layer after we erase it.
- Dust attenuation shows the opposite pattern: it is only weakly decodable anywhere in the encoder, yet a single head is necessary for most of the multimodal advantage, particularly for the most dust-attenuated galaxies
- None of the astrophysically motivated operations we tested explains what this head computes.
Paper links
External research summaries. These are not HDATF publications or measured product results.