The FM Agent

Key points

• Problem: The authors describe industrial problems such as combinatorial optimization, time-series forecasting, and high-performance kernel tuning as sharing a structure where evaluating candidate solutions is relatively straightforward but identifying truly effective ones is exceptionally difficult, and progress has relied on specialized engineers who design and iteratively refine algorithms. They see the primary focus of most prior multi-agent work on code generation as a limitation and aim for a general agent that discovers state-of-the-art solutions through large-scale search across diverse, complex problem domains.
• Input and output: This is a system paper on FM Agent, a general-purpose multi-agent system built by the FM Agent Team at Baidu AI Cloud. Its input is the problem to solve, which for machine learning tasks means a dataset and task objectives, and candidate solutions are scored by the domain-specific evaluators built into FM Agent. Its output is a program refined by evolutionary search, such as an ML pipeline, a combinatorial optimization heuristic, a CUDA kernel, or construction code for a math problem.
• Method: In the Cold Start Stage, several generative agents with different strategies build a diverse pool of initial algorithms, which are partitioned into clusters assigned to multiple islands, populations that evolve independently most of the time. In the Evolve Stage, each island evolves its population through mutation and crossover with periodic interaction between islands to cross-pollinate ideas, while a cluster-based sampling strategy that adjusts selective pressure to real-time population diversity and a curated elite pool balance exploration and exploitation. The evaluators give a single fitness score and LLM judge feedback in general, and for complex cases such as machine learning or kernel generation they use domain-specific multi-dimensional metrics such as balancing accuracy and latency.
• Implementation: The system runs on the Ray framework and splits program generation and program evaluation into separate parallel worker pools scheduled asynchronously, which supports concurrent execution of thousands of evolutionary processes. In the three benchmark experiments FM Agent uses Gemini-2.5-pro and optimization is performed exclusively by the LLM without human intervention, while a Human-Interactive Feedback Module that lets experts steer evolution through natural language or code and retrieve an expert knowledge base with RAG (retrieval-augmented generation) is optional but recommended by the authors. The GitHub addresses in the paper belong to the compared systems OpenEvolve and CUDA-L1, and the paper gives no code address for FM Agent itself.
• Results: On MLE-Bench, made of 75 Kaggle competitions, FM Agent (averaged over 3 runs with different seeds) reaches a 43.56% medal rate, above Operand ensemble at 39.56% and InternAgent at 36.44% as taken from the official report. On the 10 problems of ALE-Bench lite, where every agent uses Gemini 2.5 Pro, its overall average score of 1976.3 beats ALE-Agent, a purpose-built agent with domain-specific knowledge and a diversity-oriented search, at 1879.3 by 5.2% and the Iterative-Refinement baseline, which repeatedly refines solutions with a Self-Refine methodology, at 1201.3 by 64.6%. On a workload set clustered from KernelBench Level 3 problems, the most challenging kernels, under a tightened numerical tolerance, it achieves 2.08x to 20.77x speedups over torch.compile, and on packing 26 circles it reaches a radius sum of 2.6359740012 against AlphaEvolve's 2.6358627564.
• Limits: The MLE-Bench baseline numbers come from the official report and were produced with different base models such as gpt-4o, o1-preview, deepseek-r1, and gpt-5, and on ALE-Bench lite short contests ALE-Agent averages 2284.8 against FM Agent's 2250.8. The ablations come from a single ALE-Bench task (ahc016) with five runs per setting, and on the uncertainty inequality task the authors state that search performance is highly sensitive to problem abstractions derived from expert domain knowledge. The claim that FM Agent matches or surpasses real-world algorithmic practice in various industrial scenarios within and beyond Baidu appears without numbers.

Paper links

External research summaries. These are not HDATF publications or measured product results.