The Critic Can Hurt: On the Safety of Critics in LLM-Guided AutoML Search
- Published
- Source
- arXiv
- Paper number
- 1476
- Field
- AI / Agents
- arXiv ID
- 2611.19037
Key points
- The authors built a four-arm audit comparing no critic, random feedback, real critic, and oracle under the same budget and frontier.
- A naive 8B critic scored below the no-critic floor on 7 of 8 binary tabular datasets.
- The worst case on ADULT reached a loss of 0.245 AUC.
- Fault injection showed that random errors are nearly costless while systematically directional errors consume effective search rounds.
- An idealized model yields the necessary condition q(m−1) > β for a critic to help.
- A validation gate and a learned detector reduced the average loss, but neither guaranteed the no-critic baseline on every dataset.
Paper links
External research summaries. These are not HDATF publications or measured product results.