HDATF
日本語
相談する
メニュー

論文アーカイブ

収録論文と要約を番号の大きい順に掲載します。

収録論文 1,430資料更新日 Research Monitoringベンチマーク
  1. 1443Research AgentsarXiv:2611.14514

    Contrastive Discovery: Open-Ended Scientific Discovery over Competing Explanations

    • The authors propose a contrast set, a bundle of mutually exclusive and collectively exhaustive competing explanations for one scientific question, as the unit of discovery.
    • 89% of the top pointwise candidates were rejections, and re-evaluating the same evidence with contrast sets resolved 73% of these candidates to a specific alternative.
    • Guiding search with contrast sets resolved 67% more questions than pointwise surprisal under the same experiment budget, averaging 161.9 versus 97.2 at K=200.
    • In a blinded study, 38 human experts chose the same explanation as the system on 78% of resolved questions.
    • Most of the gain came from corrective resolutions, where evidence shifted support away from the prior-leading explanation to another one.
    • The authors note the experiments use tabular, observational datasets and a gpt-4o backbone, and on such data a resolution is a statistical rather than a causal finding.
    概要PDF
  2. 1442Machine LearningarXiv:2611.12871

    Vendi Anomaly Scores for Efficient and Accurate Anomaly Detection

    • The authors propose the Vendi Anomaly Score, which defines anomalies by how much removing a sample changes dataset diversity measured by the Vendi score.
    • They derive a first-order approximation to the exact leave-one-out score so the method scales linearly with dataset size.
    • The method requires no density estimation, partitioning heuristics, or hyperparameter tuning, adapting q automatically to the spectral structure of the data.
    • It achieves state-of-the-art anomaly detection performance across 148 benchmark datasets derived from 10 source datasets and 4 embedding architectures.
    • It remains robust in ImageNet experiments with up to 100k samples as both contamination rate and dataset scale increase.
    概要PDF
  3. 1441Machine LearningarXiv:2611.12791

    Large Language Models as a Universal Searcher for Open-World Automated Machine Learning

    • The authors encode model architectures, training hyperparameters, and data ratios into unified discrete numerical sequences called NCode, jointly optimizing all three design spaces.
    • The LLM searcher is reframed as a ranking task that picks superior candidates from a pool, performing open-world optimization without task-specific fine-tuning.
    • Trained solely on trajectories from the single vision domain of NAS-Bench-301, it achieves state-of-the-art performance across image classification, image segmentation, speaker verification and text-to-image benchmarks.
    • It scaled to the large-scale training and joint optimization of a billion-parameter Diffusion Transformer, requiring 896 GPU-days and attaining a state-of-the-art GenEval score of 0.91.
    • GRPO-trained Chain-of-Thought reasoning revealed an optimization strategy shifting from early-stage diversity-driven exploration to late-stage anchor exploitation.
    概要PDF
  4. 1440AI AgentsarXiv:2611.12222

    PARIS: A multi-agent LLM framework for end-to-end, PRISMA-guided systematic literature reviews with pre-synthesis corpus quality assessment

    • The authors built PARIS, a framework where a conductor agent running in Claude Code orchestrates 15 specialized large language model (LLM) agents.
    • All agents communicate exclusively through persisted artifacts, and the conductor validates each file before the next step.
    • Three human checkpoints keep study inclusion, corpus adequacy and the synthesis outline under human authority.
    • QualityScore grades the corpus from 0 to 100 before synthesis, and corpus improvements raised it from 74.6 to 81.9 in the first case study.
    • The second case study included 123 of 131 records and recorded a QualityScore of 69.1.
    • The authors' audits exposed failure modes such as incomplete retrieval, counts drifting between agents and unrecorded checkpoint decisions, and PARIS fully supports 8 of the 27 PRISMA 2020 items.
    概要PDF
  5. 1439Machine LearningarXiv:2611.11997

    HyInv Technical Report

    • The authors trained HyInv, a 1.96-million-parameter backbone-conditioned graph network, with random-order autoregressive pretraining followed by structure-reward reinforcement learning.
    • A sampler that drafts at a higher temperature and rewrites at a lower temperature improved refolding TM-score at matched diversity.
    • An optimized implementation with BF16 numerics designs 128 sequences for a 402-residue backbone in 50.6 ms on a single NVIDIA H20 GPU, about 52 times faster than the reference sampler.
    • Across five benchmark datasets and seven dataset-predictor combinations, HyInv reaches a higher average scTM than all ten external baselines at comparable or higher sequence diversity.
    • The average margin is +0.013 over ProteinMPNN and +0.008 over ProteinZero V2.
    • The same default setting supports de novo backbones, pocket-constrained libraries, enzyme redesign with the catalytic site kept, and loop redesign.
    概要PDF
  6. 1438Reasoning / EfficiencyarXiv:2611.11910

    Late but Valuable: Why Deadlines Change the Measured Value of LLM Reasoning

    • The authors showed that when a deadline makes the system execute a fallback, reporting answer quality and latency separately overstates the delivered value of reasoning.
    • In a replay of 1,200 decisions, thinking answers that missed a deadline were worth about five times as much as on-time ones.
    • Separate summaries predicted about twice the value that thinking delivered and overstated thinking's advantage over fast mode by 0.0241 per decision.
    • Raising the number of simultaneous requests from 8 to 56 on a single-GPU Qwen3-8B reduced the share of thinking answers meeting a 45 s deadline from 79.2% to 1.4%.
    • The authors recommend pairing each answer's value with its arrival time and scoring the action that actually runs.
    概要PDF
  7. 1437Distributed SystemsarXiv:2611.11040

    Advancements in Cloud-Native Systems: A Systematic Mapping Study of Reliability and Scalability across Cloud-Native, Edge-Native and AI-Enabled Systems

    • The authors combined 5 OpenAlex searches with 189 records from an earlier exploratory search, screening 4,611 records in total.
    • The review includes 1,794 primary studies from 2017–2026 and 95 secondary studies.
    • In the 13-category taxonomy, each study was graded as simulation, prototype or testbed, production data, production deployment, or not determinable from the abstract.
    • Of 1,003 studies with a determinable level, 66.2% used prototypes or testbeds, 18.0% simulations and 15.8% production data or deployments.
    • A 93-study synthesis core shows production deployments of cascading-failure prevention, trace- and graph-based root-cause analysis and predictive autoscaling, and production-trace evaluations of cold-start mitigation.
    • Empirical studies in the synthesis show that agent frameworks remain fragile.
    概要PDF
  8. 1436Distributed SystemsarXiv:2611.10804

    DICOMPUTE: An Economic and Privacy-Aware Framework for Distributed Artificial Intelligence Inference

    • The authors present DICOMPUTE, a research framework for distributed inference that allocates requests across privately controlled, regional, and independently operated computing resources under policy-constrained scheduling.
    • The full-cost model accounts for capital amortization, energy, network transfer, orchestration, utilization, failures, and trust-related expenses.
    • An illustrative scenario with hypothetical parameters yields a break-even volume of 5.625 million requests per month.
    • The privacy model separates data residency, transport confidentiality, execution confidentiality, and verifiable accounting.
    • The authors propose falsifiable hypotheses and a reproducible experimental design, without presenting simulated examples as measured performance.
    • They conclude conditionally that centralized clusters retain structural advantages for tightly coupled, memory-intensive, high-throughput inference.
    概要PDF
  9. 1435Research AgentsarXiv:2611.02539

    Scientific Liquidity Agents: An Agent-Based Model of Knowledge Arbitrage in Scientific Communities

    • The authors introduce SLA, an agent-based model that places domain specialists and epistemic arbitrageurs in a shared environment.
    • Researcher decisions use a belief-desire-intention architecture, while a prerequisite-based problem graph, a dynamic interaction network and a publication registry form the shared state.
    • Simulations with 10 and 18 agents registered 68 and 274 papers respectively.
    • In the smaller run, arbitrageurs took part in 16 of the 26 contact pairs that join the community into one connected component, while topical similarity alone splits it into seven.
    • Paper production is abundant, but no recorded replication succeeds, which places the bottleneck in verification rather than production.
    • Solved-problem sets correspond to the prerequisite reachability of the disciplines represented in each community.
    概要PDF
  10. 1434AI AgentsarXiv:2611.01860

    Advances in Agentic Computer Graphics

    • The authors introduce a unified taxonomy that characterizes agentic graphics systems along four dimensions: specification, base model, harness, and graphics interface and tooling.
    • Their collected corpus includes more than 300 papers published through September 2026.
    • They develop an agentic literature survey system that searches for papers, reads their content, and annotates their designs using the taxonomy.
    • The system records taxonomy labels, design insights, and limitations alongside supporting source evidence, supporting comparison across papers and analysis of broader trends.
    • The report discusses open challenges in coordinating agents and tools, designing human-agent collaboration interfaces, sustaining long-horizon planning and refinement, and reliably evaluating interactive graphics systems.
    概要PDF
  11. 1433ResearcharXiv:2611.13872

    LongTails: Automated Research in Optimization

    • The authors built LongTails, a framework for discovering and improving oracle-complexity bounds with formal verification in Lean 4.
    • The framework separates a Setup specifying the problem class and a Complexity specifying a sufficient oracle-call budget.
    • Agents design algorithms and prove convergence, and the framework checks the proofs so that every accepted Complexity is attainable.
    • All verification is done with the Lean 4 proof assistant.
    • The authors report they have already obtained more than 100 new results.
    • A single run in which the agents obtained a better Complexity lasted approximately 2–7 hours.
    概要PDF
  12. 1432RoboticsarXiv:2611.13039

    Robo-MLT: Dual-System Robotic Manipulation for Long-Horizon Medical Laboratory Tasks

    • The authors propose Robo-MLT, a dual-system framework that couples explicit workflow monitoring with learned action generation.
    • System 2 tracks the active plan step with a lightweight multiframe monitor.
    • System 1 executes the active skill with a Qwen3-VL-conditioned flow-matching policy.
    • At the sample level, Robo-MLT achieves sample completion rates of 89.00% and 92.25% on blood gas analysis and tube sorting.
    • With the workflow-level pathway, Robo-MLT reaches recovery rates of 90% and 85%, against 65% and 55% without the learned trigger.
    • End-to-end control delay is 364 ms and the sustained control rate is 20 Hz.
    概要PDF
  13. 1431Research AgentsarXiv:2611.12958

    Embodied Science Agents: Making Embodied Experimental Design Learnable

    • The authors diagnosed that existing self-driving laboratories treat experiments as symbolic actions, severing physical grounding.
    • The proposed Embodied Science Agent (ESA) lets a Digital Agent handle memory-based reasoning and Bayesian optimisation while an Embodied Agent directly controls instruments.
    • It autonomously tackled a wastewater treatment task navigating over 200 million possible conditions.
    • Through 15 continuous rounds, it raised the final solution transmittance from 82% to 99.7%, corresponding to a 98.5% reduction in residual dye concentration.
    • The optimisation started with Latin hypercube sampling and then the Bayesian optimiser proposed the next configuration.
    • The whole campaign ran on a single set of equipment within roughly 45 hours of continuous autonomous operation.
    概要PDF
  14. 1430Research AgentsarXiv:2611.12184

    Blind Variation, Selective Retention: Improving Creative Idea Generation in AI Scientists

    • The authors deliberately limited task information to push idea generation away from familiar approaches.
    • They learn from proposals previously verified on the same task and their outcomes to retain promising candidates.
    • They evaluated the framework across five proposer models and 12 scientific tasks.
    • At a discard ratio of 50%, the selective-retention model preserves 79% of the top-5% proposals by creativity and 82% by utility.
    • Five PhD-level researchers each annotated 30 proposal pairs, yielding 150 pairs in total.
    • As task information increases, the share of highly creative proposals in the top 1% decreases, supporting the creativity benefit of limited information.
    概要PDF
  15. 1429RoboticsarXiv:2611.11940

    Reliability-Adaptive Multimodal Relative Pose Tracking for Underwater Leader-Follower Robots

    • The authors propose a reliability-adaptive multimodal perception framework for six-degree-of-freedom relative-pose tracking between two moving underwater robots.
    • They explain that light absorption, scattering, color distortion, and suspended particulates severely reduce the reliability of optical measurements.
    • They collected 12 trials for each combination of two leader types, three illumination conditions, and nine trajectory patterns, yielding 648 trials in total.
    • In pool experiments, the framework achieves a mean translational error of 3.8 cm.
    • On an onboard NVIDIA Jetson AGX Orin, the fusion network achieves an inference latency of 17.62 ms.
    • A multimodal dataset containing measurements from all 648 trials will be made publicly available.
    概要PDF
  16. 1428ResearcharXiv:2611.10156

    DeepResearch RewardBench: Evaluating Reward Models for Deep Research Report

    • The authors built the first benchmark dedicated to evaluating reward modeling for research reports.
    • Data collection kept only pairs with a score margin greater than 10/30 points to ensure discriminative challenge.
    • The evaluation covers 19 models spanning diverse architectures and scales from 7B to 235B parameters.
    • The evaluation encompasses 19 models spanning diverse architectures and scales from 7B to 235B parameters, categorized into three groups.
    • The initial inter-annotator agreement stood at 63%, improving to approximately 85% after discussion.
    • The authors expect the benchmark to serve as a testbed for research-oriented agents.
    概要PDF
  17. 1427Software EngineeringarXiv:2611.06263

    CLEAR: Structured Specifications for Controllable Long-Horizon ML Research Engineering

    • The authors identified that research artifacts are not executable specifications as the root cause of cascading errors.
    • CLEAR transforms heterogeneous research artifacts into structured and human verifiable execution specifications before generating executable code.
    • Specification-conditioned generation and repair on localized units avoids repeated end-to-end regeneration.
    • On MLE-Bench Lite, CLEAR achieves 77.27% Any Medal while substantially reducing repair iterations on representative tasks.
    • Compared with AiScientist, it reduces cost by 35.16% while retaining competitive performance.
    • One analysis reports a higher average score while reducing the average cost by 85.6%.
    概要PDF
  18. 1426ResearcharXiv:2611.03424

    IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?

    • The authors measured research taste by collecting more than 90,000 proposals from six language models across four research areas.
    • LLM agents reached about 26% of the human level on citation matching and only 18% on idea matching.
    • Each LLM agent worked within a budget of 20 steps, and each population held 300 proposals written in two rounds of 150.
    • The authors state that two models trained before 2026 cannot have seen any 2026 paper, so their results are free of leakage by construction.
    • In blinded expert review, AI proposals scored below the 2026 papers on every quality dimension, by 20 points on expected success.
    • The authors conclude the bottleneck lies less in knowing what to work on than in deciding how to pursue it.
    概要PDF
  19. 1425Reasoning / SurveyarXiv:2611.02711

    Human-Supervised Artificial Intelligence Researchers in Electric Power Systems: Knowledge, Physics, Simulation, and Governance for Large Language Models

    • The authors review power-system LLM applications, related scientific workflows, and physics-grounded power AI studies.
    • In the proposed knowledge-physics-simulation-governance framework, LLMs translate information and coordinate tasks.
    • Simulators and validation tools test models, and experts retain authority over critical assumptions, safety boundaries, and final conclusions.
    • Research processes should preserve materials so that claims can be traced to sources, computations reproduced, and failures and approvals examined.
    • On this basis, a staged development roadmap and requirements for research reporting are proposed.
    • Evaluation should focus on research artifacts and their validation.
    概要PDF
  20. 1424Research AgentsarXiv:2611.14364

    ARE-bench: Benchmarking Agents'Research Ability in Real-World Researcher Sessions and Workspaces

    • ARE-bench is a 75-task benchmark that evaluates agents' research ability on real researcher sessions and in-progress workspaces.
    • Each task is anchored at a real user request and the workspace is replayed to that boundary without cleanup, preserving artifacts from earlier turns.
    • The benchmark covers seven research stages, 33 requirements, 150 sub-requirements, and 1,173 binary checklist items.
    • Among seven frontier models, the strongest passes every checklist item on only 66.2% of tasks.
    • Design and Execution rank among the hardest stages for every model.
    • Three models pass about 69% of tasks under delivery contracts and general norms alone, but drop to between 28.44% and 40.00% once historical and workspace requirements are also checked.
    概要PDF
  21. 1423Research AgentsarXiv:2611.13651

    DrugCoding: A Scientific Programming Agent for Drug Discovery

    • DrugCoding is a drug discovery agent system that unifies scientific task planning, adaptive model routing, multi-agent collaboration, and programmatic tool execution.
    • It decomposes research objectives into dependency graphs and selects specialized models for each task by applicability, performance, and cost.
    • On the multi-turn M-TCDD benchmark, it achieved an average Pass@1 of 97.35% across seven language models, ahead of the strongest baseline Codex at 88.11% by 9.24%.
    • On the top 10% challenging subset, it averaged 97.92%, surpassing scientific research agents.
    • Failure analysis showed that accurate preservation of scientific inputs was the main source of its advantage.
    • The multi-agent configuration improved over the single-agent version by an average of only 0.56%, varying widely across models.
    概要PDF
  22. 1422AgentsarXiv:2611.13309

    PegasusAI Studio: Legible Cyberinfrastructure and Accountable AI Agents for Scientific Workflows

    • PegasusAI Studio is an AI-assisted science gateway built on the Pegasus and HTCondor stack that supports agent workflow operation on shared CI.
    • It compiles one facility-curated knowledge base into the native configuration of several coding agents for consistent delivery.
    • Agents diagnose execution failures from structured workflow and scheduler state rather than terminal output.
    • Every agent action runs under the user's identity and scheduler credentials so the facility can audit it.
    • In the EHT M87 reconstruction, it recovered from two failed runs to a third run completing all 55 jobs with zero failures under user approval.
    • The gravitational-wave reanalysis completed all 350 jobs with zero failures in 54 minutes 18 seconds.
    概要PDF
  23. 1421RoboticsarXiv:2611.13251

    Curie's Jarvis: Autonomous Robotic Chemistry Experiments from Printed Protocols

    • Curie's Jarvis is a system where a robot arm carries out a chemistry experiment by reading the steps from a photo of a printed instruction sheet.
    • It tells identical-looking containers apart by reading their paper labels in four camera views and combining them by a cross-camera vote.
    • It works on an ordinary bench with no fixed positions or markers and locates its target again before every step.
    • It completed 5 of 5 runs of one protocol and 4 of 5 runs of another with no human intervention.
    • In 160 locates on printed and handwritten labels it identified every container correctly.
    • The reagents come already measured, so the system's ability to deliver a set amount was not evaluated.
    概要PDF
  24. 1420AgentsarXiv:2611.12552

    SciMARS: Multi-Agent Recursive Self-Improvement for Closed-Loop Scientific Discovery

    • SciMARS is a recursive self-improvement framework where separate editors independently revise role procedures and coordination contracts of a multi-agent team.
    • A shared evidence graph records observations, findings, plans, and unresolved questions with provenance and routes them across roles.
    • SciMARS + Memory adds a third editor for how agents record, retrieve, and receive evidence.
    • On Autonomous Science Bench with GPT-6-Astra, it resolved 12/18 attempts, ahead of 8/18 for the base team.
    • It solves 14/38 ScienceAgentBench tasks and 11/20 AppWorld tasks, compared with 11/38 and 5/20 for the base team.
    • In the GLM-5.3 cohort, it resolved 2/18 attempts, slightly ahead of 1/18 for the base team.
    概要PDF
  25. 1419Research AgentsarXiv:2611.12429

    Can Frontier Auto Researchers Reconstruct Human Research Decisions in AI?

    • AutoHRS is an evaluation framework that compares auto researcher outputs with human research across three stages: idea generation, experiment design, and paper synthesis.
    • The evaluation corpus was built from 152 ICLR 2026 oral papers, yielding 456 research trajectories.
    • On a normalized scale, auto researchers scored 0.29 for idea generation, 0.36 on experiment design, and 0.65 on paper synthesis, showing large gaps from human research in all three stages.
    • 78.4% of generated ideas were formed by combining identifiable elements from the provided reference papers.
    • Providing more fine-grained research directions consistently improved the idea reconstruction score from 0.29 up to 0.54.
    • By research type, scores were 0.49 for empirical analysis, 0.32 for methodological work, and 0.18 for benchmark research.
    概要PDF
  26. 1418Research AgentsarXiv:2611.11690

    A Physical Testbed for Self-Driving Laboratories: Comparing Bayesian and Agentic Optimization in a Minimal Closed-Loop Color Formulation Task

    • A testbed integrating commercial robots and measurement instruments performed unattended closed-loop experimentation for over 24 cumulative hours across six campaigns.
    • Each campaign completed 48 closed-loop iterations in approximately four hours without human intervention.
    • On the dye-mixture matching task, the system achieved an average RMSE of 9.95 × 10−3 OD.
    • The agentic optimizer reached the target with 52% fewer iterations for target 1 and 32% fewer for target 2.
    • The agentic optimizer matched target 3, which BO failed to match within 48 iterations, in 11 iterations.
    • The agent switched between several models during the campaigns, including linear models, Hill curves, and Michaelis-Menten curves.
    概要PDF
  27. 1417Research AgentsarXiv:2611.08483

    SwarmEvolve: Latent Agent Swarms for Evolutionary Discovery and Optimization

    • SwarmEvolve is an evolutionary search framework in which swarms of low-cost agents reason in latent space and communicate by passing KV caches in each round.
    • A novelty filter keeps the agents with the most distinct ideas by directly measuring the distances between the agents' hidden state vectors.
    • Crossover of ideas happens naturally in latent space, as a summarizer agent receives the combined KV caches of all swarm members and writes one new program.
    • On 21 problems in mathematical discovery and scientific computing, it got the highest average score and needs a median of only 3 to 8 minutes to reach the best scores that other state-of-the-art frameworks reach in 60 minutes.
    • It evaluates 5 to 13 times more programs per hour than existing SOTA frameworks.
    • With Qwen3 8B, it improves on the solutions that AlphaEvolve first found with frontier LLMs on three problems.
    概要PDF
  28. 1416Research AgentsarXiv:2611.06911

    BEAKER: An Expert-Curated Benchmark for Embodied Brains in Self-Driving Chemical Laboratories

    • BEAKER is a benchmark with 1,000 expert-curated question-answer samples and 11 task types for evaluating embodied brains in self-driving chemical laboratories.
    • All questions and annotations are manually created and reviewed by chemistry and embodied AI experts without AI-assisted generation or annotation.
    • Evaluating 28 multimodal models showed that Actionability is the clearest bottleneck, averaging 22.2 across all 28 models.
    • The best model, Claude Opus 4.7, scored 59.2 overall, 34.1 points below the human expert score of 93.3.
    • Adding expert captions roughly doubled the mid-video sequence reasoning scores of all three models.
    • Chemistry-specialized and science-oriented models do not consistently outperform general-purpose models.
    概要PDF
  29. 1415Reasoning / EfficiencyarXiv:2611.02501

    SpecPilot: Autonomous Algorithm Discovery and Systems Optimization for Speculative Decoding

    • SpecPilot is an autonomous agentic loop that automatically optimizes speculative decoding, which speeds up LLM decoding with a lightweight drafter, for each deployment.
    • A cost model locates the largest gaps between the current system and its ceiling, and parallel agents explore directions and integrate the successful ones.
    • Token forcing checks every output token against stock to detect reward hacking.
    • The loop discovered W12, a general lossless 12-bit weight format, tree verification for hybrid recurrent targets, two-level verification, and three batching mechanisms.
    • Without training or quality loss, it raised the speedup over AR from 4.46× to 10.87× with one request, and from 1.94× to 3.93× with batching.
    • A structured memory holds 140 validated mechanisms and reuses them for new drafters, models, and GPU platforms.
    概要PDF
  30. 1414ResearcharXiv:2611.03327

    From FAIR to AI-FAIR Metadata-First Infrastructure for AI-Enabled Science

    • The authors diagnose that data infrastructure in ecology and biodiversity science is fragmented and the flow of value remains largely unidirectional.
    • They define the AI-FAIR metadata criteria as four categories: structural, scientific, provenance and governance metadata.
    • They propose the Question-Data-Model triangle with bidirectional edges, so improvements made by models return to the source data providers.
    • They built an automated evaluation tool where an agent retrieves metadata and rates checklist items while scoring stays deterministic.
    • Data publishers have begun piloting the evaluation tool to establish baseline scores for their own data.
    概要PDF
  31. 1413Research AgentsarXiv:2610.40488

    rTaste: Do Research Agents Have Taste? Evaluating Early Stage Research Judgment

    • Six frontier models were each given seven recent NLP papers and asked to propose, forecast and rank minimum viable experiments.
    • Of the 415 proposed experiments, 194 were judged sound, valuable and new, and 73 were also beyond the seed paper and non obvious.
    • The agent's first choice matched the auditors' most valued experiment in 38% of conditions, better than a random pick at 20%.
    • The agent's forecasts scored a Brier score of 0.192, better than the base rate of 0.240 but worse than the mean of six judges at 0.179.
    • Teams proposed 0.71 more good experiments per five on average than a single researcher without statistical significance, at a cost of $15.31 versus $23.93 per condition.
    • The reviewers found 66% of the experiments built on the paper and 44% matched a list of obvious follow ups, so few opened a new direction.
    概要PDF
  32. 1412ResearcharXiv:2610.39641

    Rebuilding a Field's Open-Problem Catalog from the Literature: A Two-Layer Knowledge Graph, and Why Status Must Stay Proposed

    • The authors built graphs with an evidence layer on top of two catalogs: 1,053 database research problems and 1,000 AI research problems.
    • The rebuilt graph kept every answer of the catalog-only graph, with 47 of 47 identical on databases and 34 of 35 on AI.
    • The evidence layer connects outside literature the catalog curators never cited, linking 770 works to 336 problems on databases.
    • In the audit of status verdicts, only 9 of 41 verdicts were correct, 30 partly correct and 2 wrong, and no refutes verdict (0 of 34) disproves an open problem.
    • The authors conclude in the design that a status change must be proposed with its evidence and never applied, making the human reviewer part of the architecture.
    • They report a label-collision defect where a paper count query returned 8,769 instead of 2,679, and made a shared label a build-time error.
    概要PDF
  33. 1411Research AgentsarXiv:2611.05108

    From AutoResearch to Continual AutoResearch: Do Agents Learn Across Tasks?

    • The authors propose CONTINUAL AUTORESEARCH, which pairs persistent and reset executions of the same agent under matched task streams, data, and time budget.
    • The evaluation covers nine task families spanning synthetic tasks, symbolic regression, time-series forecasting, and tabular AutoML.
    • The authors evaluated five frontier CLI agents, and persistent state improved fixed-budget research efficiency for all five.
    • Time-convergence gains were +21.3 points for Kimi K3, +20.9 for Sonnet 4.6, +7.5 for Opus 5, +6.1 for GPT-5.6-Sol, and +3.0 for GPT-5.4.
    • Sonnet 4.6 was the only agent with a substantial mean final outcome gain of +7.6 points, while the others were near zero or slightly negative.
    • In the memory ablation, matched insight cards changed initialization and search but did not consistently improve final performance.
    概要PDF
  34. 1410Research AgentsarXiv:2611.02833

    Reimagining computational quantum research with AI scientists

    • In OmniQ, an expert-audited virtual research environment, the authors converted 59 papers into reproduction tasks and evaluated five agentic systems powered by large language models and six third-party agent harnesses.
    • The reproduction experiments identified reproducibility issues in 18 of the 60 studies, where the paper description, released code, and reported results were inconsistent.
    • On the cleaned tasks, Claude Code with Opus 5.5 achieved the highest reproduction rate of approximately 93%, while Meta-Harness with DeepSeek performed worst at 19%.
    • Residual Agents distilled verified workflow repairs into reusable experience, raising the reproduction success of DeepSeek V4.1 Flash from 68% to approximately 85%.
    • Existing third-party agent harnesses did not necessarily improve reproduction and could even degrade performance.
    • All reproduced results were integrated into an extensible platform built on the model context protocol, making verified research workflows directly accessible to AI scientists.
    概要PDF
  35. 1409Information RetrievalarXiv:2610.39231

    A Sequential Decision Framework for Evaluating LLM-Guided Search

    • The paper proposes a framework that represents the search process with two actions: select (judging the next document in an existing ranking) and spawn (creating a new ranking).
    • The authors describe existing retrieval strategies such as adhoc search, MTF, Thompson sampling, ISJ, and CAL as policies operating in a common environment.
    • The experiments used the TREC Deep Learning 2019 collection, with GPT-4o-mini for relevance judgments and GPT-5 for more complex tasks such as spawning.
    • Under a global judging budget, the query reformulation then MTF policy achieved the highest recall for every retrieval pipeline and budget.
    • Adhoc search, which judges a single ranked list top-down, was the least efficient policy under every retrieval pipeline.
    • The retrieval pipeline itself mattered as much as the policy, with SPLADE and the hybrid pipeline recovering more relevant documents per judgment.
    概要PDF
  36. 1408RAG / AgentsarXiv:2611.07859

    CORE: Coarse-to-Fine Search with Context Management for Deep Research

    • CORE is a training-free deep research harness that combines coarse-to-fine search with context management.
    • The harness first retrieves broad passages with a dense retriever, then reuses the token-level similarities from a late-interaction reranker to return only the most relevant snippet from each passage.
    • Context management lets the agent expand the context around sources it has already retrieved and replaces repeated snippets with compact references, reducing context waste.
    • Across three agent models and three benchmarks, CORE achieved the highest answer accuracy and evidence recall, even on corpora containing up to 553M documents.
    • On the largest ClimbMix corpus, CORE with GPT-5.4 reached 68.42% accuracy, higher than 56.14% for OpenResearcher and 50.88% for Tongyi DeepResearch.
    • In the component analysis, accuracy rose from 62.61% with coarse retrieval only to 70.84% with coarse-to-fine search and 80.12% for the full CORE with context management.
    概要PDF
  37. 1407AI / AgentsarXiv:2611.07508

    DelegateEval: Evaluating Platforms for Personal External Service Agents

    • The authors define a personal external service agent that uses a person's knowledge, experience, methods, and service boundaries to accept client requests, deliver work, and provide ongoing support.
    • DelegateEval is a platform-level evaluation framework for agent systems that may combine large language models, reusable skills, protocols such as MCP, and executable tool environments.
    • The evaluation spans eight dimensions from ease of creation to efficiency and value, and observes the full service lifecycle from creation to repeated service histories.
    • This paper presents the framework and an experimental design only, and complete datasets, executable environments, a unified scoring protocol, and cross-model results remain to be developed.
    • The authors cite a randomized experiment with 453 professionals on midlevel professional writing tasks where ChatGPT reduced average completion time by about 40% and improved output quality by about 18%, arguing that time and quality must be measured together.
    • The authors note that in a 2025 randomized study of 16 experienced open-source developers, allowing AI tools increased completion time by about 19% while participants expected to be faster, so efficiency gains must be verified by measurement.
    概要PDF
  38. 1406Research AgentsarXiv:2611.06910

    FrontierPhysics: Evaluating Agents for End-to-End Physics Research

    • FrontierPhysics is an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics.
    • Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts.
    • Evaluation combines a deterministic verifier that checks numerical outputs with a reviewer agent that grades the paper and process against expert-authored rubrics.
    • Only 56 of 182 submissions were accepted, and contributors report having spent about 270 days per task on the original projects on average.
    • Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts, with the best, Claude Opus 5.5, at 27.4%.
    • Manual review of the 168 rollouts shows the main weaknesses are insufficiently rigorous reasoning at 48% and limited expert intuition at 34%, rather than gaps in physics knowledge or execution errors.
    概要PDF
  39. 1405Research AgentsarXiv:2610.39620

    Letting AI Formulate the Research Question: Autonomous Research Agents

    • The system gives an off-the-shelf coding agent an instruction document and two skills for question formulation and peer review. All five consecutive runs completed everything from question formulation to a revised, re-computable paper with local computation on a laptop and no research input from humans.
    • Over 13 development runs, changing the one or two sentences that set the research area coincided with large shifts in candidate topics. Adding a single sentence hinting at societal problems moved 14 of the same model's 15 candidates to societal problems.
    • In the evaluation runs the agents wrote 77 candidate questions across nine societal fields, all concerning society or the environment. Most candidates were discarded because public ground truth was missing or technology access was gated.
    • The critic changed how the chosen question was compared and measured in every run, and the reviewer re-ran every first draft from its verification entry point and reproduced the reported numbers. All 40 review findings were adopted or partly adopted.
    • In the external AI evaluation, Gemini 4 Argon judged all five papers workshop level with a mean overall score of 5.4, while Muse Wonder was stricter with a mean of 4.1. Both reviewers rated novelty and generality low.
    • As limitations, five runs with one research model are a small sample with no ablation, all evaluations rely on AI model judgments, and a human evaluation is planned separately.
    概要PDF
  40. 1404Research AgentsarXiv:2610.39413

    How Specific Do I Need to Be? Effects of User Query Underspecification on Deep Research Agents

    • Of 203 initial-final query pairs collected from real conversations between researchers and an agent, 176 contained information absent from the initial query, and the authors derived the missing-information types from these pairs.
    • The three missing information types of scope, terminology, and preference cover 94% of the 176 pairs, appearing in 45%, 37%, and 33% of them respectively.
    • Across the three agents DR Tulu, Tongyi-DR, and GPT-5.4, removing or obscuring even scope or terminology alone significantly reduced rubric-based scores, and scores declined further as queries became more underspecified.
    • Citation precision showed no statistically significant change even under full underspecification, so underspecification hurt alignment with the user's information needs rather than increasing hallucination.
    • Most agents answered without recognizing the ambiguity, and in three of four conditions, runs that stated assumptions exhibited larger answer-quality losses than unaware runs.
    • In search behavior, exploration transitions increased, the share of tokens traceable to the user query decreased while untraced tokens increased, and retrieving more new documents did not translate into greater use of retrieved content in later searches.
    概要PDF
  41. 1403Research AgentsarXiv:2610.38882

    Aligning Autoresearch: Co-Improvement between Agents and Humans

    • In AutoBenchmark, benchmarks that the research agent built without human feedback were easy enough for solvers to score above 80. With detailed human direction, solver scores dropped by half.
    • Fine-grained human feedback lowered solver scores by 28 to 46.5 points in every one of the six benchmark-solver pairs, and this difficulty transferred to held-out solvers that were never used in the loop.
    • HackBench was built from 169 real trajectories, where the median number of commands per trajectory is 264 while the median number of hacked commands is only 1. The best monitor, Claude Opus 5, reached only 64.88 F1 at hack localization.
    • AutoInteract builds multi-turn interaction data from user models learned on real histories and trains with reinforcement learning, raising average Pass@1 on LiveCodeBench-Pro from 29.9 to 43.0 at the 4B scale and from 49.1 to 55.7 at 27B, with multi-turn gains from 23.0 to 43.2.
    • In the RL-XAR experiments, standard LLM judges preferred model writing over expert human writing in 63.5% to 84.6% of cases. Meta-optimizing rubrics on expert writing reversed this, and training lifted story writing from 2.8 to 8.2.
    • As a limitation, on Wikipedia section writing RL-XAR moved the score only from 2.7 to 4.0, far below humans and frontier models, and the human evaluation was a small preliminary study by the authors.
    概要PDF
  42. 1402Research AgentsarXiv:2611.03097

    DR Pixel: Page-Native Deep Research with Vision-Language Models

    • DR Pixel is the first open-source end-to-end multimodal deep research agent that uses rendered page images as its sole retrieval and reasoning unit over a corpus of 1.06M pages.
    • The authors built DRPixelBench with 500 questions on scientific papers, where every question requires reading text, tables, figures, and equations together.
    • Without training, the page-native approach improved accuracy by 5.1–8.4 points over OCR-based pipelines across GPT-5.4, Gemini-3.5-Flash, and Qwen3.6-27B while using 20%–30% fewer retrieval calls.
    • Qwen3.6-27B trained with counterfactual credit reweighting raised DRPixelBench accuracy from 40.1 to 52.1, slightly surpassing GPT-5.4.
    • The training came within 1.8 points of an oracle on out-of-distribution ViDoRe-V3 and needed 57% fewer searches than vanilla SFT for a comparable hit rate.
    概要PDF
  43. 1401ResearcharXiv:2610.36878

    RubricAtlas: Mining Type-Level Rubrics from Expert Reports to Evaluate Deep Research Agents

    • The authors built RubricAtlas from 3,079 Chinese financial research reports, yielding 99 research types, 921 questions, and 8,160 checkpoints.
    • An individual report covers a median of 54% of its type’s questions, so relying on a single report narrows the evaluation.
    • Two judge pipelines achieved checkpoint agreement of κ = 0.71 and agent-ranking agreement of Kendall τ = 0.87.
    • Deleting the blocks answering one question lowered its score from 0.64 to 0.21, while adding 40% irrelevant text produced no score increase.
    • Evaluating only questions supported by the task’s source report raised completeness scores by 10.5 points and changed rankings, so the choice of standard shapes the outcome.
    概要PDF
  44. 1400Research AgentsarXiv:2610.35579

    Infinity Machine: a self-improving AI agent for research

    • The authors gave a strong model a shell, up to one hour of work, and an exact verifier, and it solved 14 problems over six days for a total model cost of about $180.
    • The system produced four results beyond the published state of the art on catalogued open problems in quantum information, including a certified point p = 0.064966, a qutrit frontier of 0.2910672, a 44-gate depth-6 circuit, and a stabilizer-rank bound of 10.
    • On the SABRE qubit-routing benchmark it cut the added-gate count from 68,559 to 12,087.
    • In hardware design it cut the energy per multiply-accumulate of an open 8×8 INT8 tile from 578.9 fJ to 128.8 fJ, a 77.7% reduction, with 55% less area.
    • The self-improvement gate rejected the one playbook generation the system wrote for itself on cost, and proposers gamed lenient scorers four times.
    • All three fully autonomous runs ended in honest ties with public records, and one audit caught a stale record.
    概要PDF
  45. 1399RoboticsarXiv:2611.02391

    Late Autumn: LLM Agents Specialize a General Multi-Robot Planner for One Warehouse

    • LLM coding agents ran six autonomous research campaigns to specialize a 9.0k-line general multi-robot planner for one warehouse site.
    • Every campaign's best patch raised throughput by 22–60% over the baseline, and the gains held on 20 unseen seeds.
    • The agents rediscovered known MAPF mechanisms such as corridor-exit priorities, local guidance, and rollouts, and independent campaigns found the same bottleneck.
    • The gains largely carried over to another layout, a larger fleet, and other service times, but mostly vanished under an outbound-heavy demand mix.
    • The final patches are 0.7k–2.4k lines with a median improving step of 27 lines, so experts can review them commit by commit.
    • Review found problems that the score misses, and some patches multiply planner compute by over 100×.
    概要PDF
  46. 1398Research AgentsarXiv:2610.39744

    PAPERBENCHX: Can AI Reproduce Science Across Disciplines?

    • PAPERBENCHX is a benchmark of 93 paper-reproduction tasks spanning 12 research areas.
    • The tasks run in 10 domain-native scientific computing environments, including Ansys HFSS, Meep, PySCF/GPU4PySCF, AutoDock Vina, and Habitat.
    • Every submission is rerun from scratch in a clean environment, and only regenerated evidence is scored against a hidden rubric rather than agent claims.
    • Among 10 frontier agent configurations, the best, GPT-6-Astra, reached a 59.08% partial score and a 13.98% full rate.
    • 71 tasks were not fully reproduced by any evaluated configuration.
    • Average Modeling and Execution scores reached 62.73% and 61.83%, while Validation fell to 42.91%, making scientific validation the weakest stage.
    概要PDF
  47. 1397Agents / MemoryarXiv:2610.34767

    The Right Fact, the Wrong Decision: SciValidityBench for Scientific Agent Memory

    • The paper defines scientific use validity: the same p = 0.03 value changes its evidential status depending on whether the plan was fixed in advance.
    • The authors built SCIVALIDITYBENCH, an executable benchmark factory whose 64 TaskSpecs cover five validity modules and six scientific domains.
    • The SCISTATEFORGE compiler–reader pipeline turns distributed experimental histories into explicit decision-governing state.
    • Across four model families, compiled scientific state raised valid decisions by 9.2–18.8 percentage points over a generic Event Graph.
    • BM25 retrieval alone did not reproduce this cross-family gain.
    • Exact repair of the decisive state recovered 18.6 validity points more on average than a matched irrelevant repair.
    概要PDF
  48. 1396AI / AgentsarXiv:2611.02516

    Cumulative and Audited Guardrail Tool Evolution for LLM Agent Safety

    • The authors propose EVOLVE, a four-stage framework that externalizes security knowledge into executable guardrail tools which are inherited, audited, and repaired, to address the weakness of fixed-rule defenses against emerging risks.
    • Six specialized agents cover risk analysis, tool generation, tool fusion, execution, auditing, and revision, forming a closed loop from risk perception to trusted adaptation.
    • Across the RJudge, ASSEBench, and AgentHarm benchmarks and four backbone LLMs, EVOLVE ranks first in all 12 combinations and outperforms existing defenses by +22% average F1 score.
    • The authors identify the VGT phenomenon in which over 20% of initially generated tools contain vulnerabilities, and auditing lowers the risky-tool share in the programming domain from 44.78% to 8.20%.
    • EVOLVE achieves 93.3% accuracy in detecting risks in the embodied AI2THOR environment, and on a real 6-DoF robotic arm it blocks sharp-object and falling-object hazards and replans paths to avoid collisions.
    • Guardrail tool reuse rates reach up to 86%, showing that security knowledge accumulates and evolves through interaction rather than remaining fixed after deployment.
    概要PDF
  49. 1395Reasoning / RLarXiv:2611.02455

    RLiD: Reinforcement Learning in the Decision Plane for Language Agents and LLMs

    • The authors propose RLiD, which trains a large decision model that picks an action from an executable menu in one forward pass, removing the credit noise that token-level RL inherits from how actions are worded.
    • Proposition 1 shows that marginalising over alternative wordings of the same action preserves the expected update and removes the variance component tied to realisation.
    • In an exact FrozenLake credit audit, a decision-plane batch gradient with exact advantages reaches cosine 0.88 to the true gradient, while 71% to 97% of a token policy's score second moment is realisation variance.
    • On ARC the Qwen2.5-1.5B decision model reaches 78.6% in 0.12 GPU-h, ahead of token GRPO at 75.9% with 1.17 GPU-h, and in the same-runner Sokoban the decision plane reaches 44.5%, above every token variant.
    • On ALFWorld the full distillation and student-RL route reaches 99.3% success in 12.7 H100 GPU-hours, whereas token-level RL of the same LLM reaches 86.6% in 17.3 GPU-hours.
    • Without the admissible-action list the generative distilled student keeps 99.3% unseen success, and without memory it reaches 90.5%.
    概要PDF
  50. 1394AI AgentsarXiv:2611.01977

    EvoRec: A Unified Self-Evolving Agentic Recommender with Dynamic Reflection Trees

    • The authors propose EvoRec, a unified agentic recommendation framework that turns recommendation into a closed loop of need understanding, planning, execution, feedback, reflection, and repair.
    • A Need Agent infers structured user needs from long-term, short-term, and working memory, and a Planning Agent splits the retrieval budget of 3,000 between Behavior Continuation and Need Satisfaction while choosing tools and ranking fusion weights.
    • A Dynamic Reflection Tree localizes failures to need inference, planning, or prior repair strategies, generates personalized repairs, and verifies their effectiveness by re-running the decision.
    • Verified repair strategies enter working memory to guide later decisions and are abstracted into reusable strategies shared across users, so self-evolution continues without updating agent parameters.
    • On the Kuaishou local-services advertising dataset, EvoRec improves Hit@50 by 15.7% over the strongest baseline, and on the 18.5% of users with accumulated repair experience it improves Hit@50 by 13.1%.
    • In a seven-day online A/B test, nearline-deployed EvoRec increases advertising revenue by 4.6% and CTCVR by 3.5%, with a 14.5% revenue gain among high-intent users.
    概要PDF

全1430件中50件を表示