SciExam for ENSO: Can AI Agents Build Climate Models?
- Published
- Source
- arXiv
- Paper number
- 1194
- Field
- Agents
- arXiv ID
- 2610.10513
Key points
- Built the first family of benchmarks that grade results solely with hidden scientific criteria (statistics, variable dependence, forecasting skill), with no answer key, rubric, or LLM judge.
- Agents must freeze their own diagnostic metrics in advance, so they cannot tweak the grading criteria to fit their results.
- Six of twelve agents scored higher than the published reference model, with strengths in variable reconstruction and forecasting.
- The top agents' model structures were each compatible with one of the two competing hypotheses explaining ENSO asymmetry, producing results directly relevant to an open scientific debate.
- Scores held up even when calendar years were hidden, confirming the agents were not memorizing the observational record.
Paper links
External research summaries. These are not HDATF publications or measured product results.