HDATF
English
Start a conversation
Menu

Paper Archive

Every archived paper with its summary, highest number first.

Papers 1,430Archive updated Research MonitoringBenchmarksRSS feed
  1. 031AgentsarXiv:2502.04180

    Multi-agent Architecture Search via Agentic Supernet

    • Traditional multi-agent system (MAS) design is labor-intensive and requires extensive manual prompt engineering and complex agent-to-agent communication definitions.
    • Current automated MAS design methods usually search for a single, uniform optimal system, which leads to resource inefficiency and high cost on simpler queries.
    • A single optimal MAS often lacks adaptability across different task domains, so it can degrade or run inefficiently when transferred without re-optimization.
    • MaAS introduces an agent supernet, a stochastic L-layer directed acyclic graph (DAG) built from agent operators, from which customized multi-agent structures can be sampled.
    • The controller network uses MoE-style networks and early-exit operators to manage computation depth according to query complexity and dynamically sample query-specific structures from the supernet.
    • The system uses Monte Carlo methods to optimize the operator distribution in the supernet and refines individual agent operators through agent-based text gradients, balancing performance and token cost.
    AbstractPDF
  2. 030Agents / EnsemblingarXiv:2502.00674

    Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?

    • Traditional mixture-of-agents approaches assume that combining different LLMs leads to better performance.
    • There is still limited understanding of the fundamental tradeoff between diversity and quality in LLM ensembles.
    • The paper introduces Self-MoA, which generates multiple outputs from a single high-performing model.
    • It also develops Self-MoA-Seq to handle more outputs under context-length constraints.
    • It conducts a comprehensive analysis of the quality-diversity relationship across more than 200 experiments.
    • Ensemble performance is more sensitive to output quality than to diversity.
    AbstractPDF
  3. 029ReasoningarXiv:2502.03373

    Demystifying Long Chain-of-Thought Reasoning in LLMs

    • Large language models still struggle with complex reasoning domains such as advanced math, science QA, and software engineering, even when Chain-of-Thought prompting is used.
    • In particular, our understanding of how long CoT capabilities emerge in LLMs remains limited in the context of reinforcement learning training.
    • RL training for CoT often runs into unstable growth in CoT length, outputs that exceed context-window limits, and reward hacking unless the design choices are appropriate.
    • The researchers systematically studied long CoT reasoning by combining supervised fine-tuning (SFT) and reinforcement learning (RL) with models such as Llama-3.1-8B and Qwen2.5-7B-Math.
    • They explored distilling long CoT patterns from a strong emergent model, QwQ-32B-Preview, to curate high-quality SFT data and filtering noisy web-extracted data to expand verifiable reward signals.
    • For RL, they introduced a new reward function called Cosine Length-Scaling Reward with an N-gram repetition penalty to stabilize CoT length and control generation behavior during training.
    AbstractPDF
  4. 028ReasoningarXiv:2502.02390

    CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoning

    • Fast-thinking LLMs often struggle with complex, multi-step, knowledge-intensive tasks that require repeated deliberation.
    • Existing slow-thinking methods such as Chain-of-Thought (CoT) and basic Monte Carlo Tree Search (MCTS) variants are limited to initial inputs or static knowledge retrieval, so they lack dynamic adaptation to changing information.
    • Existing RAG methods usually integrate external knowledge only at the initial input stage, which can lead to irrelevant information overload or missing key details later in the reasoning process.
    • CoAT introduces a new dynamic associative memory (AM) mechanism that actively generates and integrates fresh, relevant information at each step of reasoning in real time.
    • It adopts an optimized Monte Carlo Tree Search (MCTS) framework that adds a new Association step and explicitly accounts for the quality of both generated content and associative memory.
    • The node value in MCTS is computed from the value of generated content and the value of associative content, weighted by coefficient β, ensuring that the quality and relevance of dynamic knowledge are prioritized during search.
    AbstractPDF
  5. 027Reasoning / Data EfficiencyarXiv:2502.03387

    LIMO: Less is More for Reasoning

    • Conventional wisdom says that complex reasoning in LLMs requires vast amounts of training data.
    • Collecting and training on large datasets is expensive in compute and resources.
    • Models trained on large, uncurated reasoning datasets have limited generalization and may memorize.
    • LIMO is a data-efficient supervised fine-tuning (SFT) approach that uses a minimal dataset.
    • It builds a highly curated dataset of 800 math reasoning problems through multi-stage filtering for problem difficulty and detailed rule-based scoring of reasoning-chain quality.
    • The training process allows the base model to use pretrained knowledge and integrate meta-reasoning tasks into coherent chains.
    AbstractPDF
  6. 026Video GenerationarXiv:2502.01061

    OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models

    • End-to-end human animation models are hard to scale to large datasets because strict data filtering requirements discard more than 90% of the raw video data.
    • Weak correlation between major driving signals, such as audio for lip sync, and general body motion restricts existing models to specialized scenarios and blocks full-body animation and diverse interactions.
    • Current models are limited to narrow use cases such as front-facing portraits with static backgrounds because small, highly curated datasets limit generalization.
    • OmniHuman is proposed as an end-to-end multimodal conditional human video generation framework built on a more advanced Diffusion Transformer (DiT) backbone.
    • It adopts a new Omni-conditions training strategy with a progressive three-stage mixed-conditioning scheme that expands data with weak conditions while carefully balancing strong versus weak conditioning during training.
    • It efficiently integrates diverse driving conditions, such as audio, pose, and text, and appearance conditions, such as reference images and motion frames, with minimal additional parameters, and it reuses the DiT backbone for appearance conditioning.
    AbstractPDF
  7. 025Reasoning / EfficiencyarXiv:2501.19393

    s1: Simple test-time scaling

    • Although test-time scaling has emerged in closed models such as OpenAI's o1, there has been a lack of open-source models that exhibit clear and accessible test-time scaling behavior.
    • Traditional large language model performance improvements have depended mainly on compute-intensive training-time scaling, which has made frontier research inaccessible to many.
    • Previous open-source attempts to reproduce test-time scaling were often either complex, such as Monte Carlo tree search or multi-agent systems, or still resource-intensive, such as millions of RL samples.
    • We curated s1K, a high-quality reasoning dataset of 1,000 samples carefully selected from more than 59,000 examples by quality, difficulty, and diversity.
    • We fine-tuned the Qwen2.5-32B-Instruct base model on s1K with supervised fine-tuning, focusing the loss only on reasoning traces and solutions.
    • We developed Budget Forcing, a new and simple decoding-time intervention that directly controls thought tokens by forcing generation to stop or by suppressing the end-of-thought token and appending Wait to induce more reasoning.
    AbstractPDF
  8. 024RAG / AgentsarXiv:2501.09136

    Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG

    • Large language models (LLMs) often generate stale or factually incorrect content because they depend on static training data.
    • Traditional retrieval-augmented generation (RAG) systems use rigid, predefined workflows, which limits complex multi-step reasoning and iterative response refinement.
    • The emerging field of Agentic RAG lacks a unified taxonomy, revealing fragmented system designs and inconsistent terminology.
    • This survey establishes a comprehensive taxonomy for Agentic RAG architectures based on agent cardinality, control structure, autonomy, and knowledge representation.
    • It systematically analyzes the evolution of the RAG paradigm from Naïve RAG through modular and graph-based approaches, contextualizing the emergence of agentic systems.
    • It provides practical guidance for system designers and identifies key open research questions for unifying and advancing Agentic RAG.
    AbstractPDF
  9. 023Alignment / InterpretabilityarXiv:2501.11120

    Tell me about yourself: LLMs are aware of their learned behaviors

    • Understanding the internal states and emergent, implicit behaviors of large language models (LLMs) remains difficult, especially when such behaviors are not explicitly described in the training data.
    • Detecting hidden backdoor behaviors in LLMs that activate only under specific and often obscure trigger conditions is a major AI safety challenge.
    • Ensuring LLM transparency and alignment requires methods that proactively identify and understand undesirable tendencies, including those arising from training-data bias or malicious contamination.
    • This study fine-tuned LLMs, GPT-4o and Llama-3.1-70B, on diverse datasets that implicitly exhibit specific behaviors such as economic decision making, strategic goal pursuit in games, and vulnerable code generation.
    • To evaluate the models' ability to explain the policies they learned, we probed them with out-of-distribution questions in free-form, numeric, multiple-choice, and two-step reasoning formats.
    • For backdoor detection, this study introduced reverse learning, which explicitly learns the reverse mapping needed to elicit a trigger by augmenting the training data with swapped user and assistant messages.
    AbstractPDF
  10. 022Reasoning / RLarXiv:2501.12948

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    • Existing LLM reasoning methods rely heavily on labor-intensive human-annotated data, creating scalability problems and introducing human cognitive biases.
    • Training models to imitate human reasoning paths can cap performance and block the discovery of potentially better non-human strategies.
    • Without initial supervised fine-tuning on reasoning tasks, the paper applies DeepSeek-R1-Zero, a pure outcome-based reinforcement learning approach, directly to the strong base LLM DeepSeek-V3-Base.
    • It implements a multi-stage training framework, DeepSeek-R1, that combines an initial readability-oriented SFT, continued RL with a language-consistency reward, and a second SFT stage using both reasoning and non-reasoning data.
    • It uses knowledge distillation to transfer DeepSeek-R1's advanced reasoning ability to smaller open-source models, improving accessibility.
    • The work shows that sophisticated reasoning can emerge in LLMs through outcome-based reinforcement learning without explicit human-annotated reasoning traces.
    AbstractPDF
  11. 021Scientific AI / ChemistryarXiv:2501.06590

    ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning

    • LLMs struggle with complex chemical reasoning tasks that require precise computation.
    • Existing models lack the ability to learn from and reuse past problem-solving experience.
    • Current approaches rely heavily on human-curated knowledge or fixed workflows.
    • Even small mistakes can cascade into failure in chemical reasoning.
    • We developed ChemAgent, a framework with a dynamic, self-updating library system.
    • It implements three types of memory: Planning Memory, Execution Memory, and Knowledge Memory.
    AbstractPDF
  12. 020Long-form GenerationarXiv:2501.09751

    OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking

    • It proposes a method that expands reflection before generation by leveraging graphs of related concepts.
    • This enables the model to produce longer, more interesting, and more informative responses.
    • Machine writing with large language models often relies on retrieval-augmented generation.
    • However, such approaches keep the model within a preselected boundary, which limits the generation of richly informative content.
    • More specifically, simply retrieved information tends to lack depth and novelty and often suffers from redundancy, which harms generated text quality and leads to shallow, unoriginal, and repetitive output.
    • To address these issues, it proposes OmniThink, a slow-thinking machine writing framework that mimics the human process of iterative expansion and reflection.
    AbstractPDF
  13. 019Architecture / MemoryarXiv:2501.00663

    Titans: Learning to Memorize at Test Time

    • Existing models struggle to process and remember very long sequences effectively.
    • Transformers have quadratic complexity, which limits the context window.
    • Current approaches lack an effective way to combine different memory types, such as short-term, long-term, and persistent memory.
    • Models face challenges in generalization and length extrapolation.
    • It introduces a neural long-term memory module that learns to remember at test time based on a surprise signal.
    • It presents three architectural variants for integrating memory effectively: Memory as Context, Memory as Gate, and Memory as Layer.
    AbstractPDF
  14. 018RAG / VideoarXiv:2501.05874

    VideoRAG: Retrieval-Augmented Generation over Video Corpus

    • Large language models and large vision-language models often generate inaccurate or stale information because they rely on parametric knowledge.
    • Conventional retrieval-augmented generation, or RAG, focuses mainly on text data and includes static images only in a limited way, while video is often converted to text and loses rich multimodal information such as temporal dynamics and visual context.
    • Long videos challenge LVLMs because of computational infeasibility and limited context windows, and many videos do not have associated text such as subtitles.
    • VideoRAG embeds both the query and the video, using frames and subtitles, with an LVLM and selects the top-k relevant videos by cosine similarity to enable dynamic video retrieval from a large corpus.
    • For response generation, the framework connects the selected video frames and associated text, either subtitles or generated auxiliary text, with the user query and feeds this comprehensive multimodal input into the LVLM.
    • An adaptive frame selection mechanism with k-means++ clustering identifies the most useful and least redundant video frames, and when subtitles are unavailable it uses ASR to generate auxiliary text from the audio track.
    AbstractPDF
  15. 017LLMs / Long ContextarXiv:2501.08313

    MiniMax-01: Scaling Foundation Models with Lightning Attention

    • Existing language models are limited by short context windows of 32K to 256K tokens.
    • Linear attention is theoretically promising, but it has not been successfully scaled in practice.
    • It uses a hybrid architecture that combines lightning attention and softmax attention at a 7:1 ratio.
    • It uses a MoE model with 32 experts and a carefully engineered parallel computation strategy.
    • It follows a multi-stage training process optimized for extended context lengths.
    • Linear attention can scale successfully when it is implemented and optimized properly.
    AbstractPDF
  16. 016Architecture / AdaptationarXiv:2501.06252

    Transformer²: Self-adaptive LLMs

    • SVF replaces additive adapters with learned scaling on the singular values of each weight matrix, changing task behavior while preserving the base singular vectors.
    • Transformer² uses dispatch-and-apply inference, and experts can be selected directly, by a learned classifier expert, or mixed through few-shot CEM optimization.
    • The paper reports that SVF and Transformer² outperform LoRA with less than 10% of LoRA's parameter count, including gains on math, code, reasoning, and vision-language benchmarks.
    • The strongest evidence is not just higher scores, but expert composability and transfer across models, which points to reusable skill vectors rather than one-off fine-tuning.
    AbstractPDF
  17. 015Math ReasoningarXiv:2501.04519

    rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking

    • This method uses MCTS both at inference time and during data generation, with a math policy SLM proposing steps and an SLM-based Process Preference Model guiding the search.
    • The core idea is to replace naive step-level reward annotations with preference pairs derived from MCTS Q-values, making the process reward model more robust than direct Q-value regression or outcome-only rewards.
    • In four rounds of self-evolution, the system starts from a large DeepSeek-Coder bootstrap and progressively improves SLM-r1 through SLM-r4 and PPM-r1 through PPM-r4 using millions of verified solutions for 747K problems.
    • With Qwen2.5-Math-7B, it reaches 90.0% on MATH and 53.3% on AIME, but the approach is compute-intensive and best suited to tasks where intermediate or final answers can be checked programmatically.
    AbstractPDF
  18. 014ReasoningarXiv:2501.04682

    Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought

    • Meta-CoT is defined as p(y|x)=∫p(y|x,z)p(z|x)dz, where z denotes a latent reasoning and search trajectory rather than a directly supervised final explanation.
    • It targets the limits of standard CoT: one-shot generation, weak self-verification, poor backtracking, and weak exploratory reasoning on hard math, logic, and planning tasks.
    • The paper proposes a training pipeline that internalizes search-like behavior using process supervision, process reward models, synthetic search traces from algorithms such as MCTS or A*, instruction tuning, and reinforcement-learning post-training.
    • It reports qualitative and empirical signs that larger frontier models can approach more complex solutions with more test-time reasoning compute, while highlighting unresolved scaling, verifier, and consistency issues.
    AbstractPDF
  19. 013Reasoning / SearcharXiv:2501.05366

    Search-o1: Agentic Search-Enhanced Large Reasoning Models

    • As a method, the model pauses generation when it produces the special tokens 'search query start' and 'search query end', retrieves the top k web documents, inserts refined knowledge between the 'search results start' and 'search results end' tokens, and then resumes.
    • The key idea is that search is not a one-time preface as in standard RAG, but a recurrent model-controlled action at uncertain points in the reasoning trajectory.
    • The Reason-in-Documents stage analyzes retrieved pages in light of the current reasoning context and emits concise final information, preventing verbose or irrelevant pages from derailing the chain of thought.
    • In terms of results and implications, the strongest gains appear in knowledge-intensive multi-step settings such as GPQA and multi-hop QA, suggesting that retrieval helps most when the reasoning model can decide both when to search and how to fold evidence back into the proof path.
    AbstractPDF
  20. 012RAG / Long ContextarXiv:2501.01880

    Long Context vs. RAG for LLMs: An Evaluation and Revisits

    • The method uses GPT-4o and strict exact-match checking to filter out questions that can be solved without external context, then evaluates only cases where the provided documents are actually needed.
    • The retrieval study compares BM25, Contriever, OpenAI text-embedding-3-small, LlamaIndex-style indexed retrieval, and RAPTOR. RAPTOR is the strongest RAG setup at about 38.5% accuracy.
    • As a result, LC reaches about 56.3% exact-match accuracy and outperforms RAG's 49.0% overall, with a clear advantage on factual questions from Wikipedia and stories and on who, where, and which-type questions.
    • In tradeoff terms, RAG is still relevant because it works well on fragmented, naturally segmented, dialog-like context and uniquely solves about 10% of evaluation questions that LC fails.
    • The evaluation lesson is that synthetic long context assembled from irrelevant passages can bias comparisons because it resembles RAG preprocessing, so context relevance should be treated as a benchmark variable.
    AbstractPDF
  21. 011Research AgentsarXiv:2501.04227

    Agent Laboratory: Using LLM Agents as Research Assistants

    • In pipeline terms, human research ideas are transformed into a curated arXiv literature review, an experimental plan, executable ML code, and a final research report and code repository.
    • A method-specialized LLM agent team collaborates across PhD student, postdoc, ML engineer, and professor roles. mle-solver uses replacement and edit operations, execution feedback, LLM grading, self-reflection, and a pool of high-scoring programs.
    • As a result, o1-preview scored highest overall, co-pilot mode improved human-evaluated quality from 3.8 to 4.38 out of 10, and gpt-4o execution was reported at about $2.33 per paper, showing an 84% cost reduction versus prior autonomous research methods.
    • Notably, automatic NeurIPS-style reviews averaged 6.1 out of 10, while human reviewers averaged 3.8, so the system is better viewed as a research assistant than a reliable autonomous scientist.
    AbstractPDF
  22. 010RAG / EfficiencyarXiv:2412.15605

    Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks

    • The core idea is to replace online RAG pipelines with a scheme that preloads all relevant documents offline into a long-context LLM and reuses the precomputed KV cache at inference time.
    • The method has three stages: knowledge preloading and KV encoding, query-time generation conditioned on the cached state, and cache reset and truncation that removes query-specific tokens while preserving the knowledge cache.
    • As a result, on SQuAD 1.0 and HotPotQA in small, medium, and large settings, CAG is reported to match or outperform BM25 and dense-index RAG baselines while eliminating retrieval time. For HotPotQA-Large, generation is reported to take 2.26 seconds instead of 92.08 seconds for dynamic in-context processing.
    • In scope and tradeoff, the approach removes the embedding model, vector store, and retriever, which simplifies deployment, but it depends on long-context capacity and is best suited to bounded, relatively stable knowledge bases.
    AbstractPDF
  23. 009Medical AI / ReasoningarXiv:2412.18925

    HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs

    • The method converts difficult closed-book medical exam questions into 40K verifiable questions with objective answers, enabling automatic feedback on both answer accuracy and reasoning quality.
    • The core idea is to use a medical verifier as an active learning component. Failed initial chain-of-thought attempts are improved through backtracking, exploring new paths, verification, and revision until a correct trajectory is found.
    • The training first fine-tunes on complex reasoning trajectories approved by the verifier, then applies PPO reinforcement learning with verifier-based rewards to strengthen independent medical reasoning.
    • In terms of results and limits, the 8B model shows an 8.5-point benchmark gain and the 70B model leads open-source medical LLMs, but verifier reliability and transfer from exam-style questions to real clinical ambiguity remain open challenges.
    AbstractPDF
  24. 008ArchitecturearXiv:2412.09764

    Memory Layers at Scale

    • Each memory layer acts as a low-cost learned lookup table that retrieves the top-k trainable keys for an input query and combines their values, rather than using attention over sequence activations.
    • Multiplicative keys reduce retrieval cost, values are sharded across GPUs, and custom CUDA kernels reach near-H100 memory bandwidth, enabling up to 128B-parameter memory in the reported scaling study.
    • As a result, Memory+ outperforms dense models with more than twice the compute budget and also beats MoE and PEER when compute and parameter counts are matched, with the clearest gains on factual memory benchmarks.
    • The implication is that the best design balances dense reasoning capacity with sparse memory capacity, which suggests that future LLMs can separate compute-intensive transformation from efficient knowledge storage.
    AbstractPDF
  25. 007AgentsarXiv:2412.21154

    Aviary

    • From a framework perspective, Aviary provides a gym-style environment for language agents that use reasoning, tools, code, and iterative action-observation loops.
    • In formal terms, the agent is treated as a policy in a language decision process, which is a language-based partially observable Markov decision process formulation.
    • The tasks include molecular cloning, literature-based research QA, and protein stability engineering, all of which require multi-step scientific reasoning.
    • As a result, online learning and test-time scaling let open-source non-frontier LLM agents compete with frontier agents and human experts at much lower cost.
    AbstractPDF
  26. 006Image GenerationarXiv:2412.18653

    1.58-bit FLUX

    • The method quantizes FLUX.1-dev into ternary -1, 0, and +1 weights, reaching a 1.58-bit representation for a large text-to-image model.
    • The quantization method is data-free, relying on self-supervision from the original model instead of an external image dataset.
    • Reported efficiency gains include 7.7x less storage, 5.1x less inference memory, and lower latency through custom low-bit kernels.
    • The benchmarks suggest that image quality at 1024×1024 remains similar despite the extreme compression.
    AbstractPDF
  27. 005Medical AIarXiv:2412.19260

    MEDEC

    • Instead of asking medical QA questions, MEDEC asks a model to verify the accuracy and consistency of existing or generated clinical records.
    • The dataset contains 3,848 clinical texts covering diagnosis, management, treatment, pharmacotherapy, and etiology-related errors.
    • The evaluation compares 17 shared-task systems and frontier LLMs, with physicians used as the human reference point.
    • In conclusion, LLMs show useful error-detection ability but still fall short of physician-level performance, underscoring the need for careful deployment in clinical documentation.
    AbstractPDF
  28. 004Reasoning / EfficiencyarXiv:2412.21187

    On the Overthinking of LLMs

    • The problem is that o1-family models often spend long reasoning traces on simple tasks, which brings only limited accuracy gains while increasing latency and cost.
    • In evaluation, the paper proposes outcome-level and process-level efficiency metrics to measure whether reasoning compute is being used sensibly.
    • The method trains the model with a self-learning strategy to simplify reasoning while preserving answer quality across tasks of different difficulty.
    • The implication is that future reasoning systems need adaptive test-time compute policies, not a habit of using verbose chains of thought for every query.
    AbstractPDF
  29. 003LLMs / Open ModelsarXiv:2501.00656

    OLMo 2

    • The release includes weights, data, code, recipes, logs, and intermediate checkpoints, making it useful for reproducible research and downstream adaptation.
    • In the training recipe, Dolmino Mix 1124 and late-stage curriculum learning improve downstream capability without relying only on more compute.
    • The instruction model OLMo 2-Instruct builds on the Tülu 3 recipe and extends it with reinforcement learning that uses verifiable rewards in the final stage.
    • As a result, the base model remains more transparent while staying competitive with open-weight systems in the Llama 3.1, Qwen 2.5, and Gemma 2 families.
    AbstractPDF
  30. 002AgentsarXiv:2412.16241

    Agents Are Not Enough

    • The core claim is that stronger generative models alone are not enough to make agents effective or sustainable in real user workflows.
    • From a historical perspective, earlier agent systems stalled or failed because autonomy, context, trust, coordination, and user representation were harder than the agent metaphor suggests.
    • Structurally, the proposed ecosystem separates executing agents, user-representing Sims, and user-facing assistants instead of forcing a single model component to handle everything.
    • In industry terms, useful agent systems need not only autonomous action loops but also persistent user context, workflow coordination, and human-facing control interfaces.
    AbstractPDF