Reimagining computational quantum research with AI scientists
- Published
- Source
- arXiv
- Paper number
- 1410
- Field
- Research Agents
- arXiv ID
- 2611.02833
Key points
- In OmniQ, an expert-audited virtual research environment, the authors converted 59 papers into reproduction tasks and evaluated five agentic systems powered by large language models and six third-party agent harnesses.
- The reproduction experiments identified reproducibility issues in 18 of the 60 studies, where the paper description, released code, and reported results were inconsistent.
- On the cleaned tasks, Claude Code with Opus 5.5 achieved the highest reproduction rate of approximately 93%, while Meta-Harness with DeepSeek performed worst at 19%.
- Residual Agents distilled verified workflow repairs into reusable experience, raising the reproduction success of DeepSeek V4.1 Flash from 68% to approximately 85%.
- Existing third-party agent harnesses did not necessarily improve reproduction and could even degrade performance.
- All reproduced results were integrated into an extensible platform built on the model context protocol, making verified research workflows directly accessible to AI scientists.
Paper links
External research summaries. These are not HDATF publications or measured product results.