TinyScientist: An Interactive, Extensible, and Controllable Framework for Building Research Agents

Key points

• Problem: Existing automatic research systems often design and use overly complex agent frameworks that are hard to use or extend without significant technical expertise. The authors trace this to weak interactivity (complex research intents and unclear communication interfaces make it hard for researchers to engage with the agent's progress), limited extensibility (representative frameworks rely on rigid, tool-specific designs that make new tools hard to add), and insufficient controllability (many systems offer weak supervision of safety, ethics, and cost budget).
• Input and output: This is a framework paper that releases a framework, a Python package, and a web demo for building research agents. Inputs and outputs can be conference-style PDFs such as ACL (Association for Computational Linguistics) or ICLR (International Conference on Learning Representations), structured JSON, or plain text, and a typical run takes a plain-text research intent and returns a fully formatted conference paper PDF. The run also yields the research idea, experiment code, and peer-review results.
• Method: A base TinyScientist class orchestrates four workflow stages (Thinker, Coder, Writer, Reviewer), and each stage refines its own output up to a preset iteration cap. The Thinker samples n ideas and refines each for k rounds, the Coder generates and runs experiment code through an external coding agent such as Aider, the Writer inserts citations through the Semantic Scholar API, and the Reviewer writes simulated reviews and a meta-review. Feature components add a Formatter that puts results into tables, an MCPClient that attaches tools through MCP, and a Checker that enforces cost and safety limits.
• Implementation: The released Python package produces a research idea, experiment code, a conference-format PDF, and a review with the seven lines of code in Algorithm 1 (calls to think, code, write, and review), and the web UI (user interface) has five pages: configuration, intent input, idea view, code view, and paper view. According to the README, the GitHub repository holds the core, tools, and formatters modules and frontend and backend demo code, and the package is published on PyPI (Python Package Index) as tiny-scientist. The README states that the think, code, write, and review stages now run on the Claude Agent SDK (software development kit) by default, with the OpenAI Agents SDK as an option.
• Results: With gpt-4o-mini as the backbone for both frameworks, the authors report that TinyScientist's papers scored better than or comparable to Agent Laboratory under both GPT-4o judging and human judging on a 5-point scale, with the largest improvement of 0.23 points in biology writing quality. On ML-related tasks, adding tools raised the GPT-4o writing score from 3.93 to 4.06 and the idea score from 3.78 to 3.82. Of 20 potentially unsafe biology tasks, the Checker blocked 18 at the Thinker stage and flagged the remaining 2 with warnings at later stages.
• Limits: The authors report that inconsistent LaTeX formatting occasionally breaks compilation, causing problems such as missing references or misaligned lines. The diagrams generated for key sections still have limited visual quality and informativeness. Evaluation used 20 ML tasks and 20 out-of-distribution biology tasks, and on the biology tasks both frameworks scored lower, which the authors say is likely because the prompts are tailored primarily to the ML domain.

Paper links

External research summaries. These are not HDATF publications or measured product results.