Large Language Models as a Universal Searcher for Open-World Automated Machine Learning
- Published
- Source
- arXiv
- Paper number
- 1441
- Field
- Machine Learning
- arXiv ID
- 2611.12791
Key points
- The authors encode model architectures, training hyperparameters, and data ratios into unified discrete numerical sequences called NCode, jointly optimizing all three design spaces.
- The LLM searcher is reframed as a ranking task that picks superior candidates from a pool, performing open-world optimization without task-specific fine-tuning.
- Trained solely on trajectories from the single vision domain of NAS-Bench-301, it achieves state-of-the-art performance across image classification, image segmentation, speaker verification and text-to-image benchmarks.
- It scaled to the large-scale training and joint optimization of a billion-parameter Diffusion Transformer, requiring 896 GPU-days and attaining a state-of-the-art GenEval score of 0.91.
- GRPO-trained Chain-of-Thought reasoning revealed an optimization strategy shifting from early-stage diversity-driven exploration to late-stage anchor exploitation.
Paper links
External research summaries. These are not HDATF publications or measured product results.