ToolUniverse: An open platform for democratizing AI scientists

Key points

• Problem: AI scientists, agents built on conversational LLMs that call scientific tools, query datasets, and carry out complex analyses, have progressed rapidly, but most are bespoke and difficult to reuse because they lack standardized interfaces for tools and data. Scientific tools differ in purpose, inputs, parameters, and runtime environments, run locally or through APIs and laboratory systems, and overlap in function, which complicates selecting and executing the right tool for a goal stated in natural language.
• Input and output: When the user's chosen LLM, reasoning model, or agent receives a natural language request, it finds relevant tools with Tool Finder and issues a call naming the tool and its arguments, and Tool Caller validates the inputs, executes the tool, and returns structured results or actionable error messages. Access is through a Python API, HTTP, MCP (Model Context Protocol), and direct function calls, and AI scientists are built without additional training or fine-tuning. This is a system paper that implements and releases the platform and reports benchmarks and case studies.
• Method: Every tool declares its purpose in natural language, a schema that names and types all inputs and outputs with their constraints, and a backend-agnostic invocation format, so the same call can execute a Python library, an ML model, a dataset endpoint, or a laboratory instrument. Tool Finder uses TF-IDF (term frequency and inverse document frequency) keyword search, LLM in-context search, and embedding search with a fine-tuned GTE-Qwen2-1.5B model, and Tool Composer chains tools into sequential, parallel, or feedback driven workflows. Tool Discoverer generates definitions and code for new tools from natural language descriptions and refines them until each quality dimension reaches at least 9 out of 10 in deployment-oriented settings or an iteration limit is reached, Tool Optimizer refines tool descriptions against test case execution results, and generated tools pass human review before registration.
• Implementation: A direct count of the open-source repository main branch on 7 July 2026 found 2,777 unique tools, including 2,421 local and remote dataset tools, 179 scientific package tools, and 89 ML model tools. The version described in the manuscript is v1.3.1, it installs with pip install tooluniverse, and it can also be called from agent frameworks such as Gemini CLI or Claude Code. The README of the official mims-harvard/ToolUniverse repository gives install paths through an MCP configuration, a Claude Code plugin, and the Python SDK, plus the tu CLI for finding and running tools, and the repository top level holds src, skills, benchmarks, examples, and tests folders.
• Results: On two LAB-Bench subtasks, DbQA (60 biomedical database questions) and SeqQA (70 nucleotide and protein sequence questions), the authors reused the items and scoring released by Biomni and compared identical agents and base models that differed only in whether ToolUniverse tools were available. The Claude Code agent with Claude Opus 4.8 rose from 56.7 to 78.3 accuracy on DbQA and from 94.3 to 99.5 on SeqQA, and the Codex agent with GPT-5.5 (mean of 3 runs) rose from 81.1 to 92.8 and from 91.4 to 96.2. Biomni, a purpose-built biomedical agent, scored 74.4 and 81.9, and both ToolUniverse-augmented agents exceeded it on both subtasks.
• Limits: The authors state that ToolUniverse orchestrates existing tools, so any conclusion is bounded by the underlying databases and models, that machine learning outputs such as an ADMET-AI (absorption, distribution, metabolism, excretion, and toxicity prediction) score and association evidence are estimates to confirm experimentally, and that the scientist remains in the loop because the platform does not by itself certify scientific correctness. The quantitative evaluation covers two LAB-Bench subtasks, and the Claude scores count questions the base model refused on safety grounds as wrong answers. For the clinical trial case, the authors write that the finding is best read as a reproducible safety signal from a secondary analysis rather than a definitive effect.

Paper links

External research summaries. These are not HDATF publications or measured product results.