Paper2Agent: Reimagining Research Papers As Interactive and Reliable AI Agents

Key points

• Problem: Significant technical barriers often remain before a paper's new computational method can be used on new data, and a reader might need to locate the code repository, install dependencies, configure environments and interpret the expected inputs and outputs. The authors argue that even a powerful tool like AlphaGenome is hard for biologists to use because of the technical barrier of handling API keys, client objects and variant objects. They note that executable papers increased reproducibility but still required technical familiarity, and that Papers with Code improved discoverability while the barrier of installing and executing the code remained.
• Input and output: This is an implemented multi-agent system shown through case studies. The input is a paper with its code repository link, supplements and data, and the output is an MCP (Model Context Protocol) server Python file with tools, resources (manuscript, code, data) and prompts (multi-step workflow instructions), plus a configured environment. Connecting the server to a chat agent such as Claude Code yields a paper agent that runs the paper's methods from natural language queries.
• Method: An orchestrator agent built in Claude Code invokes four sub-agents as needed at different stages: the Environment-manager (sets up an isolated environment), the Tutorial-scanner (finds tutorials in the repository), the Tutorial-tool-extractor-implementor (implements tutorials as reusable tool functions) and the Test-verifier-improver (writes, runs and fixes tests). The Test-verifier-improver repeats a loop of generating, executing and fixing tests using only the tutorial's own examples to check that the original numbers and figures come back, and functions that keep failing lose their MCP decorators and are dropped from the server. Verified tools are locked for reproducibility, and every tool keeps a link to the original GitHub source code.
• Implementation: The 22 AlphaGenome MCP tools were built in around 3 hours on a personal laptop without human intervention, the TISSUE server has 6 tools, and 7 Scanpy preprocessing and clustering tools were built in around 45 minutes on a personal laptop. All three agents in the AlphaGenome benchmark used claude-sonnet-4-20250514 as the LLM and ran on a MacBook Air (M2), and the AlphaGenome, Scanpy and TISSUE MCP servers are hosted on Hugging Face Spaces. The GitHub README explains installing the repository's skills/paper2agent folder as a skill in Claude Code or Codex and connecting to the MCP servers hosted on Hugging Face Spaces, and says the skill launches parallel specialist agents, checks the work with fresh verifier agents and delivers a ZIP file with usage instructions (USAGE.md).
• Results: On 15 AlphaGenome tutorial queries the AlphaGenome agent reached 100.0% accuracy, above Claude + Repo (Claude Code given the repository) at 60.0% and Biomni at 40.0%, and on 15 newly curated queries it again reached 100.0% against 80.0% and 60.0%. The TISSUE agent's outputs on mouse somatosensory cortex spatial transcriptomics data and the Scanpy agent's preprocessing and clustering of three 10x Genomics PBMC (peripheral blood mononuclear cell) datasets matched the results human researchers obtained on the same data. An AI co-scientist that connected the AlphaGenome and ADHD GWAS data MCPs executed a hypothesis chosen by the user, picked rs1626703 out of 209 candidate variants, predicted that it alters splicing of the MPHOSPH9 (M-phase phosphoprotein 9) gene in glutamatergic neurons, and then finished the analysis of all 39 loci within two hours.
• Limits: The authors state that if the original codebase is incomplete, poorly documented or contains unresolved errors, Paper2Agent cannot reliably expose it as a functioning tool. The accuracy benchmark depends on expert knowledge of the paper, manually implemented ground truth and manual grading, and the work so far focuses on methodological papers. The ADHD result is a computational prediction made with AlphaGenome, and the authors note that these hypotheses benefit from subsequent human evaluation.

Paper links

External research summaries. These are not HDATF publications or measured product results.