A Mechanistic Interpretability Study of an Astronomy Foundation Model

Published
Source
arXiv
Paper number
1488
Field
Interpretability
arXiv ID
2611.15653

Key points

  • We present, to our knowledge, the first causal mechanistic interpretability study of an astronomy foundation model, AION-1, focusing on its multimodal advantage in redshift prediction, where combining imaging with photometry outperforms photometry alone.
  • We find that this advantage is largest for spatially extended and dust-attenuated galaxies, and that most of it is localized to the first of the encoder’s twelve blocks.
  • Spatial extent is causally used, and one attention head dominates linear probing for it, yet ablating that head has negligible effect on redshift prediction: the information is distributed across heads and is decodable again one layer after we erase it.
  • Dust attenuation shows the opposite pattern: it is only weakly decodable anywhere in the encoder, yet a single head is necessary for most of the multimodal advantage, particularly for the most dust-attenuated galaxies
  • None of the astrophysically motivated operations we tested explains what this head computes.

Paper links

External research summaries. These are not HDATF publications or measured product results.

Read original (opens in a new tab)