MineralExp: Benchmarking LLM Agents for Experimental Design in Real-World Mineral Exploration

Published
Source
arXiv
Paper number
1469
Field
Research Agents
arXiv ID
2611.17931

Key points

  • The authors organized geochemical experimental design in mineral exploration into 19 experimental types across five mineral-system processes: Source, Transport, Deposition, Modification, and Preservation, turning it into a multi-label problem.
  • The benchmark was built from published ore-deposit literature, yielding 277 expert-reviewed cases and 1,572 reference experimental plans.
  • Across 28 configurations of 4 agent harnesses and 7 LLMs, overall recall ranged from 70.76% to 91.56%, yet the highest overall F1 was only 63.84%.
  • While reference cases involve 3.49 experiments on average, DeepSeek-V4.1-Flash predicts nearly 10 per case, dropping its Precision to approximately 32%.
  • In LLM-as-a-Judge evaluation, partial coverage ranged from 83.59% to 93.51%, but strict coverage stayed between 22.65% and 36.83%.
  • As a limitation, the domain harness Biomni consumed approximately 67k input tokens per case, more than 12× the input token cost of OpenCode, yet scored lower.

Paper links

External research summaries. These are not HDATF publications or measured product results.

Read original (opens in a new tab)