MineralExp: Benchmarking LLM Agents for Experimental Design in Real-World Mineral Exploration
- Published
- Source
- arXiv
- Paper number
- 1469
- Field
- Research Agents
- arXiv ID
- 2611.17931
Key points
- The authors organized geochemical experimental design in mineral exploration into 19 experimental types across five mineral-system processes: Source, Transport, Deposition, Modification, and Preservation, turning it into a multi-label problem.
- The benchmark was built from published ore-deposit literature, yielding 277 expert-reviewed cases and 1,572 reference experimental plans.
- Across 28 configurations of 4 agent harnesses and 7 LLMs, overall recall ranged from 70.76% to 91.56%, yet the highest overall F1 was only 63.84%.
- While reference cases involve 3.49 experiments on average, DeepSeek-V4.1-Flash predicts nearly 10 per case, dropping its Precision to approximately 32%.
- In LLM-as-a-Judge evaluation, partial coverage ranged from 83.59% to 93.51%, but strict coverage stayed between 22.65% and 36.83%.
- As a limitation, the domain harness Biomni consumed approximately 67k input tokens per case, more than 12× the input token cost of OpenCode, yet scored lower.
Paper links
External research summaries. These are not HDATF publications or measured product results.