AI Can Learn Scientific Taste

Key points

• Problem: The ability to judge and propose research ideas with long-term impact, which the authors call scientific taste, is largely concentrated among highly experienced researchers. Current AI Scientist training mainly targets literature search and experiment execution, which address how to carry out research, so choosing which research directions are worth pursuing remains open. RLVR, which relies on rewards verifiable by ground-truth answers or unit tests, is hard to apply to such open-ended tasks, and RLHF depends on costly human annotations and struggles to capture community-level preferences from individual preferences alone.
• Input and output: Scientific Judge takes the titles, abstracts, and publication dates of two papers, writes a reasoning trace, and answers A or B for the paper with the higher citation count. Scientific Thinker takes the title and abstract of a seed paper and returns a follow-up research idea as a title and an abstract. This is a method paper that proposes the RLCF training paradigm and tests it with the SciJudgeBench preference-pair dataset and trained models.
• Method: RLCF pairs papers from the same field and year and treats the one with clearly more citations as preferred, keeping a training pair only when the citation gap is at least 8 and at least 30% of the higher count. Scientific Judge is trained on these pairs with GRPO (Group Relative Policy Optimization) and receives a reward of 1 for a correct prediction and 0 otherwise. Scientific Thinker samples several ideas per seed paper and is trained with Comparison-Based GRPO, where each idea's reward is its win rate in a round-robin tournament judged by Scientific Judge.
• Implementation: The authors built 720,341 preference pairs across computer science, mathematics, physics, and other fields from 2.1M arXiv papers published through 2024. Judge models were trained with the MS-SWIFT framework on five Qwen2.5-Instruct sizes from 1.5B to 32B, Qwen3-4B-Instruct-2507, Qwen3-30B-A3B-Instruct-2507, and Llama-3.1-8B-Instruct, and Thinker models were trained from Qwen3-30B-A3B-Thinking-2507 and Qwen3-4B-Thinking-2507 on 4,000 high-citation seed papers published from January to July 2025. The top level of the official GitHub repository holds the README (English and Chinese), the paper PDF, and an assets folder of figures, and the README gives a results summary along with links to a Hugging Face collection of models and datasets and to an online demo.
• Results: On the 1,000 in-domain test pairs, SciJudge-Qwen3-30B reached 82.7% accuracy, where a pair counts as correct only if the answer holds under both A and B orderings, above GPT-5.4 Thinking (81.6%), the highest of the four strong LLM baselines in Table 3. On 904 pairs of papers published in 2025, after the training period, Qwen3-4B rose from 64.7% before training to 80.9% after training and Qwen3-30B-A3B from 71.7% to 83.1%. Under majority voting by GPT-5.2-high, GLM-5, and Gemini 3 Pro, SciThinker-30B trained with SciJudge-Qwen3-4B as the reward model beat its base policy (the policy model before training) with an in-domain win rate of 81.5%, and its average in-domain win rate against ideas from GPT-5.2, GLM-5, and Gemini 3 Pro was 54.2%, compared with 30.3% for the base policy.
• Limits: The authors state that citations are an imperfect form of community feedback, since some high-potential papers may receive few citations at first and become influential later, and that the controlled evaluations reduce simple author, institution, and topic explanations but cannot remove all biases in citation-based feedback. The ideation evaluation mainly relies on strong LLM evaluators, and implementing a subset of the proposed ideas is left for future work. Scientific Judge is trained mainly on titles and abstracts, and the authors note that richer paper context such as related work sections may improve its judgement.

Paper links

External research summaries. These are not HDATF publications or measured product results.