NanoResearch: Co-Evolving Skills, Memory, and Policy for Personalized Research Automation

Key points

• Problem: Automated research systems such as The AI Scientist and EvoScientist can traverse the full research lifecycle, but the authors view them as uniform pipelines that produce near-identical outputs regardless of each user's resources, methodological preferences, and target formats. The authors name three missing capabilities. Each run starts from scratch and re-encounters the same debugging patterns, past hypotheses, validated configurations, and inferred resource constraints are discarded when a session ends, and implicit preferences such as preferring simpler methods are too diffuse to write as rules and fade when compressed into memory entries.
• Input and output: This is a system paper with an implementation and experiments. NanoResearch takes a user-specified research topic and a user profile built through interactive queries (preferences, compute budget, target venue style, and so on), runs three stages, idea generation and planning, experimental validation, and paper writing and review, and outputs a LaTeX paper. The user gives natural-language feedback at the end of each stage, and the Orchestrator, the agent that coordinates the system, retrieves relevant entries from the Skill Bank and the Memory Module before each task and updates both stores afterward.
• Method: Stage 1 retrieves papers from academic databases (for example, arXiv and Semantic Scholar) and parses performance scores from them, proposes hypotheses with a ReAct loop, turns the hypothesis kept after a novelty check into a JSON experiment blueprint with datasets, baselines, metrics, and ablations, and revises the blueprint with critiques from an internal LLM reviewer. Stage 2 writes code, deploys it to an environment such as a SLURM cluster, debugs it automatically until execution succeeds or the retry budget runs out, and analyzes the results, and Stage 3 drafts the manuscript section by section and has a Review agent that works without skill or memory retrieval revise it until predefined quality thresholds are met. After each stage the Orchestrator adds general rules to the Skill Bank and project-specific experience to the Memory Module, and user feedback updates the planner model's parameters through SDPO, which uses the feedback-conditioned model as its own teacher to create a token-level learning signal without a reward model.
• Implementation: Literature retrieval uses the OpenAlex API, and the only trainable component is the Orchestrator's planner (Qwen3-8B). The other agents use DeepSeek-V3.2 for ideation, planning, and setup and execution, GPT-5.3-Codex for coding and debugging, Claude Sonnet 4.6 for writing and figure prompts and code, Gemini 3.1 Flash for figure images, Gemini 3.1 Flash Lite for review, and Gemini 3 Pro for revision. Per its README, the official repository (OpenRaiser/NanoResearch) describes a 9-stage pipeline from ideation to review, local GPU or SLURM execution, an evo pipeline that updates the user profile, skills, and memory, a ram-train command that trains an SDPO LoRA adapter on (x, y, feedback) triples collected from evo runs, and Claude Code and Codex integration, and its roadmap lists a benchmark evaluation suite as a remaining item.
• Results: On 20 Claude-generated tasks in 7 domains with 20 simulated researchers, NanoResearch in Round 1 scored Compliance 8.163, E2E (share of runs that finish the whole pipeline) 1.000, and Perf. (average task accuracy of the produced method) 0.6844, higher than the four baselines (AI-Researcher, DeepScientist, EvoScientist, AI Scientist-v2) on every metric, while the baselines' E2E ranged from 0.500 to 0.900. By Round 3 Compliance rose to 8.963, Innovation (originality of the idea relative to prior work) to 5.645, and Perf. to 0.7548, and the total cost per topic was $1.430, lower than $6.039 for AI Scientist-v2 and $2.944 for EvoScientist. In an evaluation by three PhD researchers on their own three tasks, Perf. rose from 0.6466 in Round 1 to 0.8603 in Round 3.
• Limits: The evaluation centers on AI and ML tasks whose outputs can be completed in code and text, and the authors leave extension to biology, chemistry, or physics, which involve physical experiments and instrument control, to future work. The 20 simulated tasks and the user-role feedback were all produced by Claude, subjective scores such as Compliance, Innovation, and Expression (writing quality of the paper) were rated by an LLM judge, and the human evaluation covers three PhD researchers and three tasks. In the human evaluation, Round 1 Perf. of 0.6466 was below EvoScientist's 0.6537, and Innovation and Expression dipped slightly from Round 2 to Round 3.

Paper links

External research summaries. These are not HDATF publications or measured product results.