HDATF
English
Start a conversation
Menu

Paper Archive

Every archived paper with its summary, highest number first.

Papers 1,193Archive updated BenchmarksRSS feed
  1. 044RL / AgentsarXiv:2502.19328

    Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems

    • Existing reward models rely too heavily on human preferences, which can introduce bias and accuracy problems.
    • Current approaches lack systematic verification of factual accuracy and instruction following.
    • Traditional reward models often fail to capture multiple important evaluation criteria at the same time.
    • The authors developed a multi-agent reward system that combines specialized verification agents with human preference signals.
    • They built a router component that intelligently decides which verification agents to call.
    • They also implemented a unified mechanism that combines scores from multiple evaluation sources.
    AbstractPDF
  2. 043Math ReasoningarXiv:2503.00735

    LADDER: Self-Improving LLMs Through Recursive Problem Decomposition

    • Reinforcement learning for large language models (LLMs) requires a progressive curriculum of difficulty in the training tasks, but that is often unavailable and leads to training stagnation or catastrophic performance collapse.
    • Complex reasoning domains such as mathematics pose a substantial challenge because there is a large gap between problems the model can solve and those it cannot solve yet.
    • Existing LLM training and improvement methods often rely heavily on large human-curated datasets or continuous human supervision and feedback.
    • The LADDER framework enables recursive problem decomposition, allowing the LLM to autonomously generate a tree of progressively simpler variants of a complex problem it cannot currently solve, using transformation libraries and induced prompting.
    • A robust numerical verification framework evaluates answer correctness and provides a clear, objective reward signal for reinforcement learning.
    • Test-Time Reinforcement Learning (TTRL) extends LADDER by generating problem-specific focused variants at inference time for previously unsolved test problems, enabling targeted and dynamic learning on individual hard questions.
    AbstractPDF
  3. 042Alignment / SafetyarXiv:2502.16797

    Forecasting Rare Language Model Behaviors

    • Standard LLM evaluations cover thousands of queries, but deployed models process billions, leaving a significant blind spot for rare but potentially harmful behaviors.
    • Behavior that appears safe in limited tests can create substantial risk when exposed to far more queries in deployment.
    • Traditional evaluation methods do not account for the fundamental scale gap or the stochastic nature of language models, which can generate different answers to the same query.
    • Researchers from Anthropic, Mila, and UC Berkeley developed a Gumbel-tail method to predict the risk of undesirable LLM behavior at deployment scale using limited evaluation data.
    • The method estimates each query's trigger probability by repeatedly sampling the model and measuring the fraction of responses that exhibit the target behavior.
    • It uses extreme value theory, specifically the Gumbel distribution, to model and extrapolate the tail behavior of these trigger probabilities from a small set of evaluation queries.
    AbstractPDF
  4. 041Reasoning / RLarXiv:2503.01307

    Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs

    • Some large language models achieve substantial self-improvement through reinforcement learning on verifiable reasoning tasks, while others quickly plateau under the same training.
    • It remained unclear which intrinsic properties or early capabilities enable LLMs to use extra compute effectively for self-improvement.
    • There was limited understanding of the foundational traits that determine success or failure in RL-based self-improvement for LLM reasoning.
    • The team developed a framework that uses a GPT-4o-mini classifier to identify and quantify four cognitive behaviors in LLM reasoning traces: verification, backtracking, subgoal setting, and reverse chaining.
    • They ran comparative RL experiments on models with different self-improvement abilities, Qwen-2.5-3B and Llama-3.2-3B, in the Countdown game.
    • They induced self-improvement with targeted interventions, such as fine-tuning models on synthetic data rich in specific cognitive behaviors and curating pretraining data to amplify those behaviors.
    AbstractPDF
  5. 040Reasoning / SurveyarXiv:2502.21321

    LLM Post-Training: A Deep Dive into Reasoning Large Language Models

    • Pretrained large language models (LLMs) often struggle with factual accuracy and produce hallucinated or logically inconsistent outputs.
    • LLM reasoning is primarily based on statistical patterns rather than explicit logical inference, which limits performance on complex reasoning tasks.
    • Despite massive pretraining, LLMs may still be misaligned with human intent, preferences, and ethical considerations, so additional refinement is needed for trustworthy deployment.
    • This paper systematically surveys and categorizes existing post-training methodologies for LLMs, with a focus on their role in improving reasoning ability.
    • It organizes these techniques into three connected stages: fine-tuning (for example, instruction, CoT, PEFT), reinforcement learning (for example, RLHF, DPO, RLAIF), and test-time scaling (for example, CoT prompting, Tree-of-Thoughts, MCTS).
    • The survey highlights the specific points at which each technique helps refine LLM behavior, improve robustness, and ensure adaptability across applications.
    AbstractPDF
  6. 039Training EfficiencyarXiv:2503.02875

    The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models

    • Existing methods for improving LLM reasoning require either massive labeled data or computationally expensive sampling.
    • Traditional fine-tuning approaches struggle with efficiency and scalability on complex reasoning tasks.
    • The method exploits the Prefix Self-Consistency phenomenon, where different solution paths share a common early reasoning prefix.
    • We develop UPFT, a method that fine-tunes the model on extracted prefix substrings while preserving the reasoning structure.
    • It integrates structure tuning to prevent catastrophic forgetting of reasoning ability.
    • The initial tokens in a reasoning process carry key information and remain highly consistent across different solutions.
    AbstractPDF
  7. 038Architecture / ReasoningarXiv:2502.13842

    Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking

    • Small large language models, or LLMs, face an inherent performance bottleneck when handling complex reasoning tasks because of fixed compute budgets and limited parameter counts.
    • The traditional approach of continually scaling LLM parameters to improve performance runs into diminishing returns and rapidly rising compute and deployment costs, raising sustainability concerns.
    • Empirical analysis shows that "hard" tokens induce persistent gradient oscillations and sharp spikes across Transformer layers, indicating structural burden and insufficient depth for processing complex information.
    • It introduces the concept of an Inner Thinking Step, which breaks the generation of a single token into a series of internal reasoning steps and repeatedly refines its representation.
    • It implements Residual Thinking Connections, or RTC, which accumulate the output of each thinking step to enable gradual representation refinement and break through performance bottlenecks.
    • It applies Adaptive Token Routing, or ATR, to dynamically assign extra thinking steps only to the most important tokens, avoiding unnecessary work on easy tokens and improving compute efficiency, while Thinking Step Encoding, or TSE, provides contextual information at each step.
    AbstractPDF
  8. 037Reasoning / AgentsarXiv:2502.08235

    The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks

    • LRMs tend to prefer extended internal reasoning over interaction with the environment, which leads to inefficient performance.
    • There is no systematic framework for analyzing and quantifying overthinking behavior in AI systems.
    • Understanding the balance between reasoning and action in interactive environments remains limited.
    • This paper develops a comprehensive framework for studying and measuring excessive deliberation in LRMs.
    • Using SWE Bench Verified, the authors analyzed 4,018 trajectories to identify patterns.
    • They introduced a scoring system that was validated against expert human evaluation.
    AbstractPDF
  9. 036Efficiency / Multi-modelarXiv:2502.14815

    Optimizing Model Selection for Compound AI Systems

    • Compound AI systems usually use the same LLM for every module, which limits overall performance.
    • There has been no systematic approach to optimizing model selection across different system components.
    • We developed LLMSelector, a framework that exploits monotonic performance relationships.
    • It efficiently assigns models to modules using LLM-based performance estimation.
    • It implements an iterative optimization approach through sequential module nomination.
    • End-to-end performance is often monotonic with respect to module-level performance.
    AbstractPDF
  10. 035Benchmarks / Code AgentsarXiv:2502.12115

    SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?

    • Existing LLM coding benchmarks rely on unit tests, which makes them vulnerable to grader hacking and insufficient for validating complex full-stack software engineering tasks.
    • Earlier evaluations lack direct economic metrics, so they cannot quantify the real monetary value of LLM outputs in realistic software development scenarios.
    • Current benchmarks are often too narrow, focusing on isolated programming problems or specific repositories rather than the diverse, ambiguous, and end-to-end nature of commercial freelance software engineering work.
    • We developed SWE-Lancer, a benchmark built from more than 1,400 real freelance software engineering tasks at Expensify with up to $1 million in actual monetary rewards.
    • It includes two task types: individual-contributor (IC) SWE tasks, which require code patch generation, and SWE manager tasks, where the model must choose the best solution among multiple proposals.
    • We adopted a rigorous five-stage pipeline that uses end-to-end (E2E) tests written in Playwright and triple-validated by expert engineers to ensure robust, hack-resistant evaluation.
    AbstractPDF
  11. 034LLMs / DiffusionarXiv:2502.09992

    Large Language Diffusion Models

    • The common belief is that the capabilities of large language models, such as scalability and in-context learning, come only from autoregressive architectures.
    • Autoregressive models have limitations, including the curse of reversal, where they struggle with tasks that require bidirectional or reverse reasoning.
    • There has been no large language diffusion model that can match the performance of billion-parameter autoregressive models.
    • We developed LLaDA, a masked diffusion model for discrete token sequences, using a Transformer backbone without causal masking.
    • We optimized a principled cross-entropy loss that upper-bounds the negative log-likelihood and trained an 8B-parameter LLaDA model from scratch on a 2.3-trillion-token dataset.
    • We implemented a flexible iterative sampling strategy at inference time, including low-confidence remasking to balance generation quality and speed.
    AbstractPDF
  12. 033Architecture / EfficiencyarXiv:2502.11089

    Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

    • Traditional attention in LLMs has quadratic complexity in sequence length, making long-context processing extremely expensive during training and inference.
    • Existing sparse attention methods often fail to turn theoretical efficiency gains into real latency reductions because of inefficient memory access patterns and incompatibility with modern hardware structures such as MQA/GQA.
    • Most sparse attention approaches are inference-only and require a pretrained full-attention backbone, which can hurt performance or make end-to-end training impractical or inefficient.
    • Native Sparse Attention (NSA) introduces a hierarchical sparse attention mechanism that combines three parallel branches: compressed attention for global context, selected attention for finely important tokens, and sliding-window attention for local context.
    • A learnable gating mechanism dynamically aggregates the outputs of these branches, allowing coarse and fine information to be flexibly combined while maintaining high sparsity.
    • NSA uses dedicated Triton-based kernels designed for hardware alignment, focusing on group-centered data loading and shared KV fetching to optimize memory access and maximize GPU tensor-core utilization in both training and inference.
    AbstractPDF
  13. 032Reasoning / SurveyarXiv:2502.03671

    Advancing Reasoning in Large Language Models: Promising Methods and Approaches

    • Large language models, or LLMs, often lack robust and systematic reasoning ability, which makes them struggle with complex tasks such as logical deduction, mathematical problem solving, and multi-step reasoning.
    • These shortcomings lead to serious problems such as factual inaccuracy, inconsistency, and hallucination, which limit the reliability of LLMs in critical applications.
    • Integrating the data-driven paradigm of modern LLMs with the structured reasoning of classical AI remains an ongoing and difficult challenge.
    • This paper conducts a comprehensive literature review that systematically collects and analyzes prior work on improving LLM reasoning.
    • It classifies the many reasoning improvement techniques into three major groups: prompting strategies, structural innovations, and learning-based approaches.
    • The survey also reviews the benchmarks and metrics used to evaluate LLM reasoning and identifies open problems and future research directions.
    AbstractPDF
  14. 031AgentsarXiv:2502.04180

    Multi-agent Architecture Search via Agentic Supernet

    • Traditional multi-agent system (MAS) design is labor-intensive and requires extensive manual prompt engineering and complex agent-to-agent communication definitions.
    • Current automated MAS design methods usually search for a single, uniform optimal system, which leads to resource inefficiency and high cost on simpler queries.
    • A single optimal MAS often lacks adaptability across different task domains, so it can degrade or run inefficiently when transferred without re-optimization.
    • MaAS introduces an agent supernet, a stochastic L-layer directed acyclic graph (DAG) built from agent operators, from which customized multi-agent structures can be sampled.
    • The controller network uses MoE-style networks and early-exit operators to manage computation depth according to query complexity and dynamically sample query-specific structures from the supernet.
    • The system uses Monte Carlo methods to optimize the operator distribution in the supernet and refines individual agent operators through agent-based text gradients, balancing performance and token cost.
    AbstractPDF
  15. 030Agents / EnsemblingarXiv:2502.00674

    Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?

    • Traditional mixture-of-agents approaches assume that combining different LLMs leads to better performance.
    • There is still limited understanding of the fundamental tradeoff between diversity and quality in LLM ensembles.
    • The paper introduces Self-MoA, which generates multiple outputs from a single high-performing model.
    • It also develops Self-MoA-Seq to handle more outputs under context-length constraints.
    • It conducts a comprehensive analysis of the quality-diversity relationship across more than 200 experiments.
    • Ensemble performance is more sensitive to output quality than to diversity.
    AbstractPDF
  16. 029ReasoningarXiv:2502.03373

    Demystifying Long Chain-of-Thought Reasoning in LLMs

    • Large language models still struggle with complex reasoning domains such as advanced math, science QA, and software engineering, even when Chain-of-Thought prompting is used.
    • In particular, our understanding of how long CoT capabilities emerge in LLMs remains limited in the context of reinforcement learning training.
    • RL training for CoT often runs into unstable growth in CoT length, outputs that exceed context-window limits, and reward hacking unless the design choices are appropriate.
    • The researchers systematically studied long CoT reasoning by combining supervised fine-tuning (SFT) and reinforcement learning (RL) with models such as Llama-3.1-8B and Qwen2.5-7B-Math.
    • They explored distilling long CoT patterns from a strong emergent model, QwQ-32B-Preview, to curate high-quality SFT data and filtering noisy web-extracted data to expand verifiable reward signals.
    • For RL, they introduced a new reward function called Cosine Length-Scaling Reward with an N-gram repetition penalty to stabilize CoT length and control generation behavior during training.
    AbstractPDF
  17. 028ReasoningarXiv:2502.02390

    CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoning

    • Fast-thinking LLMs often struggle with complex, multi-step, knowledge-intensive tasks that require repeated deliberation.
    • Existing slow-thinking methods such as Chain-of-Thought (CoT) and basic Monte Carlo Tree Search (MCTS) variants are limited to initial inputs or static knowledge retrieval, so they lack dynamic adaptation to changing information.
    • Existing RAG methods usually integrate external knowledge only at the initial input stage, which can lead to irrelevant information overload or missing key details later in the reasoning process.
    • CoAT introduces a new dynamic associative memory (AM) mechanism that actively generates and integrates fresh, relevant information at each step of reasoning in real time.
    • It adopts an optimized Monte Carlo Tree Search (MCTS) framework that adds a new Association step and explicitly accounts for the quality of both generated content and associative memory.
    • The node value in MCTS is computed from the value of generated content and the value of associative content, weighted by coefficient β, ensuring that the quality and relevance of dynamic knowledge are prioritized during search.
    AbstractPDF
  18. 027Reasoning / Data EfficiencyarXiv:2502.03387

    LIMO: Less is More for Reasoning

    • Conventional wisdom says that complex reasoning in LLMs requires vast amounts of training data.
    • Collecting and training on large datasets is expensive in compute and resources.
    • Models trained on large, uncurated reasoning datasets have limited generalization and may memorize.
    • LIMO is a data-efficient supervised fine-tuning (SFT) approach that uses a minimal dataset.
    • It builds a highly curated dataset of 800 math reasoning problems through multi-stage filtering for problem difficulty and detailed rule-based scoring of reasoning-chain quality.
    • The training process allows the base model to use pretrained knowledge and integrate meta-reasoning tasks into coherent chains.
    AbstractPDF
  19. 026Video GenerationarXiv:2502.01061

    OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models

    • End-to-end human animation models are hard to scale to large datasets because strict data filtering requirements discard more than 90% of the raw video data.
    • Weak correlation between major driving signals, such as audio for lip sync, and general body motion restricts existing models to specialized scenarios and blocks full-body animation and diverse interactions.
    • Current models are limited to narrow use cases such as front-facing portraits with static backgrounds because small, highly curated datasets limit generalization.
    • OmniHuman is proposed as an end-to-end multimodal conditional human video generation framework built on a more advanced Diffusion Transformer (DiT) backbone.
    • It adopts a new Omni-conditions training strategy with a progressive three-stage mixed-conditioning scheme that expands data with weak conditions while carefully balancing strong versus weak conditioning during training.
    • It efficiently integrates diverse driving conditions, such as audio, pose, and text, and appearance conditions, such as reference images and motion frames, with minimal additional parameters, and it reuses the DiT backbone for appearance conditioning.
    AbstractPDF
  20. 025Reasoning / EfficiencyarXiv:2501.19393

    s1: Simple test-time scaling

    • Although test-time scaling has emerged in closed models such as OpenAI's o1, there has been a lack of open-source models that exhibit clear and accessible test-time scaling behavior.
    • Traditional large language model performance improvements have depended mainly on compute-intensive training-time scaling, which has made frontier research inaccessible to many.
    • Previous open-source attempts to reproduce test-time scaling were often either complex, such as Monte Carlo tree search or multi-agent systems, or still resource-intensive, such as millions of RL samples.
    • We curated s1K, a high-quality reasoning dataset of 1,000 samples carefully selected from more than 59,000 examples by quality, difficulty, and diversity.
    • We fine-tuned the Qwen2.5-32B-Instruct base model on s1K with supervised fine-tuning, focusing the loss only on reasoning traces and solutions.
    • We developed Budget Forcing, a new and simple decoding-time intervention that directly controls thought tokens by forcing generation to stop or by suppressing the end-of-thought token and appending Wait to induce more reasoning.
    AbstractPDF
  21. 024RAG / AgentsarXiv:2501.09136

    Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG

    • Large language models (LLMs) often generate stale or factually incorrect content because they depend on static training data.
    • Traditional retrieval-augmented generation (RAG) systems use rigid, predefined workflows, which limits complex multi-step reasoning and iterative response refinement.
    • The emerging field of Agentic RAG lacks a unified taxonomy, revealing fragmented system designs and inconsistent terminology.
    • This survey establishes a comprehensive taxonomy for Agentic RAG architectures based on agent cardinality, control structure, autonomy, and knowledge representation.
    • It systematically analyzes the evolution of the RAG paradigm from Naïve RAG through modular and graph-based approaches, contextualizing the emergence of agentic systems.
    • It provides practical guidance for system designers and identifies key open research questions for unifying and advancing Agentic RAG.
    AbstractPDF
  22. 023Alignment / InterpretabilityarXiv:2501.11120

    Tell me about yourself: LLMs are aware of their learned behaviors

    • Understanding the internal states and emergent, implicit behaviors of large language models (LLMs) remains difficult, especially when such behaviors are not explicitly described in the training data.
    • Detecting hidden backdoor behaviors in LLMs that activate only under specific and often obscure trigger conditions is a major AI safety challenge.
    • Ensuring LLM transparency and alignment requires methods that proactively identify and understand undesirable tendencies, including those arising from training-data bias or malicious contamination.
    • This study fine-tuned LLMs, GPT-4o and Llama-3.1-70B, on diverse datasets that implicitly exhibit specific behaviors such as economic decision making, strategic goal pursuit in games, and vulnerable code generation.
    • To evaluate the models' ability to explain the policies they learned, we probed them with out-of-distribution questions in free-form, numeric, multiple-choice, and two-step reasoning formats.
    • For backdoor detection, this study introduced reverse learning, which explicitly learns the reverse mapping needed to elicit a trigger by augmenting the training data with swapped user and assistant messages.
    AbstractPDF
  23. 022Reasoning / RLarXiv:2501.12948

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    • Existing LLM reasoning methods rely heavily on labor-intensive human-annotated data, creating scalability problems and introducing human cognitive biases.
    • Training models to imitate human reasoning paths can cap performance and block the discovery of potentially better non-human strategies.
    • Without initial supervised fine-tuning on reasoning tasks, the paper applies DeepSeek-R1-Zero, a pure outcome-based reinforcement learning approach, directly to the strong base LLM DeepSeek-V3-Base.
    • It implements a multi-stage training framework, DeepSeek-R1, that combines an initial readability-oriented SFT, continued RL with a language-consistency reward, and a second SFT stage using both reasoning and non-reasoning data.
    • It uses knowledge distillation to transfer DeepSeek-R1's advanced reasoning ability to smaller open-source models, improving accessibility.
    • The work shows that sophisticated reasoning can emerge in LLMs through outcome-based reinforcement learning without explicit human-annotated reasoning traces.
    AbstractPDF
  24. 021Scientific AI / ChemistryarXiv:2501.06590

    ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning

    • LLMs struggle with complex chemical reasoning tasks that require precise computation.
    • Existing models lack the ability to learn from and reuse past problem-solving experience.
    • Current approaches rely heavily on human-curated knowledge or fixed workflows.
    • Even small mistakes can cascade into failure in chemical reasoning.
    • We developed ChemAgent, a framework with a dynamic, self-updating library system.
    • It implements three types of memory: Planning Memory, Execution Memory, and Knowledge Memory.
    AbstractPDF
  25. 020Long-form GenerationarXiv:2501.09751

    OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking

    • It proposes a method that expands reflection before generation by leveraging graphs of related concepts.
    • This enables the model to produce longer, more interesting, and more informative responses.
    • Machine writing with large language models often relies on retrieval-augmented generation.
    • However, such approaches keep the model within a preselected boundary, which limits the generation of richly informative content.
    • More specifically, simply retrieved information tends to lack depth and novelty and often suffers from redundancy, which harms generated text quality and leads to shallow, unoriginal, and repetitive output.
    • To address these issues, it proposes OmniThink, a slow-thinking machine writing framework that mimics the human process of iterative expansion and reflection.
    AbstractPDF
  26. 019Architecture / MemoryarXiv:2501.00663

    Titans: Learning to Memorize at Test Time

    • Existing models struggle to process and remember very long sequences effectively.
    • Transformers have quadratic complexity, which limits the context window.
    • Current approaches lack an effective way to combine different memory types, such as short-term, long-term, and persistent memory.
    • Models face challenges in generalization and length extrapolation.
    • It introduces a neural long-term memory module that learns to remember at test time based on a surprise signal.
    • It presents three architectural variants for integrating memory effectively: Memory as Context, Memory as Gate, and Memory as Layer.
    AbstractPDF
  27. 018RAG / VideoarXiv:2501.05874

    VideoRAG: Retrieval-Augmented Generation over Video Corpus

    • Large language models and large vision-language models often generate inaccurate or stale information because they rely on parametric knowledge.
    • Conventional retrieval-augmented generation, or RAG, focuses mainly on text data and includes static images only in a limited way, while video is often converted to text and loses rich multimodal information such as temporal dynamics and visual context.
    • Long videos challenge LVLMs because of computational infeasibility and limited context windows, and many videos do not have associated text such as subtitles.
    • VideoRAG embeds both the query and the video, using frames and subtitles, with an LVLM and selects the top-k relevant videos by cosine similarity to enable dynamic video retrieval from a large corpus.
    • For response generation, the framework connects the selected video frames and associated text, either subtitles or generated auxiliary text, with the user query and feeds this comprehensive multimodal input into the LVLM.
    • An adaptive frame selection mechanism with k-means++ clustering identifies the most useful and least redundant video frames, and when subtitles are unavailable it uses ASR to generate auxiliary text from the audio track.
    AbstractPDF
  28. 017LLMs / Long ContextarXiv:2501.08313

    MiniMax-01: Scaling Foundation Models with Lightning Attention

    • Existing language models are limited by short context windows of 32K to 256K tokens.
    • Linear attention is theoretically promising, but it has not been successfully scaled in practice.
    • It uses a hybrid architecture that combines lightning attention and softmax attention at a 7:1 ratio.
    • It uses a MoE model with 32 experts and a carefully engineered parallel computation strategy.
    • It follows a multi-stage training process optimized for extended context lengths.
    • Linear attention can scale successfully when it is implemented and optimized properly.
    AbstractPDF
  29. 016Architecture / AdaptationarXiv:2501.06252

    Transformer²: Self-adaptive LLMs

    • SVF replaces additive adapters with learned scaling on the singular values of each weight matrix, changing task behavior while preserving the base singular vectors.
    • Transformer² uses dispatch-and-apply inference, and experts can be selected directly, by a learned classifier expert, or mixed through few-shot CEM optimization.
    • The paper reports that SVF and Transformer² outperform LoRA with less than 10% of LoRA's parameter count, including gains on math, code, reasoning, and vision-language benchmarks.
    • The strongest evidence is not just higher scores, but expert composability and transfer across models, which points to reusable skill vectors rather than one-off fine-tuning.
    AbstractPDF
  30. 015Math ReasoningarXiv:2501.04519

    rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking

    • This method uses MCTS both at inference time and during data generation, with a math policy SLM proposing steps and an SLM-based Process Preference Model guiding the search.
    • The core idea is to replace naive step-level reward annotations with preference pairs derived from MCTS Q-values, making the process reward model more robust than direct Q-value regression or outcome-only rewards.
    • In four rounds of self-evolution, the system starts from a large DeepSeek-Coder bootstrap and progressively improves SLM-r1 through SLM-r4 and PPM-r1 through PPM-r4 using millions of verified solutions for 747K problems.
    • With Qwen2.5-Math-7B, it reaches 90.0% on MATH and 53.3% on AIME, but the approach is compute-intensive and best suited to tasks where intermediate or final answers can be checked programmatically.
    AbstractPDF
  31. 014ReasoningarXiv:2501.04682

    Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought

    • Meta-CoT is defined as p(y|x)=∫p(y|x,z)p(z|x)dz, where z denotes a latent reasoning and search trajectory rather than a directly supervised final explanation.
    • It targets the limits of standard CoT: one-shot generation, weak self-verification, poor backtracking, and weak exploratory reasoning on hard math, logic, and planning tasks.
    • The paper proposes a training pipeline that internalizes search-like behavior using process supervision, process reward models, synthetic search traces from algorithms such as MCTS or A*, instruction tuning, and reinforcement-learning post-training.
    • It reports qualitative and empirical signs that larger frontier models can approach more complex solutions with more test-time reasoning compute, while highlighting unresolved scaling, verifier, and consistency issues.
    AbstractPDF
  32. 013Reasoning / SearcharXiv:2501.05366

    Search-o1: Agentic Search-Enhanced Large Reasoning Models

    • As a method, the model pauses generation when it produces the special tokens 'search query start' and 'search query end', retrieves the top k web documents, inserts refined knowledge between the 'search results start' and 'search results end' tokens, and then resumes.
    • The key idea is that search is not a one-time preface as in standard RAG, but a recurrent model-controlled action at uncertain points in the reasoning trajectory.
    • The Reason-in-Documents stage analyzes retrieved pages in light of the current reasoning context and emits concise final information, preventing verbose or irrelevant pages from derailing the chain of thought.
    • In terms of results and implications, the strongest gains appear in knowledge-intensive multi-step settings such as GPQA and multi-hop QA, suggesting that retrieval helps most when the reasoning model can decide both when to search and how to fold evidence back into the proof path.
    AbstractPDF
  33. 012RAG / Long ContextarXiv:2501.01880

    Long Context vs. RAG for LLMs: An Evaluation and Revisits

    • The method uses GPT-4o and strict exact-match checking to filter out questions that can be solved without external context, then evaluates only cases where the provided documents are actually needed.
    • The retrieval study compares BM25, Contriever, OpenAI text-embedding-3-small, LlamaIndex-style indexed retrieval, and RAPTOR. RAPTOR is the strongest RAG setup at about 38.5% accuracy.
    • As a result, LC reaches about 56.3% exact-match accuracy and outperforms RAG's 49.0% overall, with a clear advantage on factual questions from Wikipedia and stories and on who, where, and which-type questions.
    • In tradeoff terms, RAG is still relevant because it works well on fragmented, naturally segmented, dialog-like context and uniquely solves about 10% of evaluation questions that LC fails.
    • The evaluation lesson is that synthetic long context assembled from irrelevant passages can bias comparisons because it resembles RAG preprocessing, so context relevance should be treated as a benchmark variable.
    AbstractPDF
  34. 011Research AgentsarXiv:2501.04227

    Agent Laboratory: Using LLM Agents as Research Assistants

    • In pipeline terms, human research ideas are transformed into a curated arXiv literature review, an experimental plan, executable ML code, and a final research report and code repository.
    • A method-specialized LLM agent team collaborates across PhD student, postdoc, ML engineer, and professor roles. mle-solver uses replacement and edit operations, execution feedback, LLM grading, self-reflection, and a pool of high-scoring programs.
    • As a result, o1-preview scored highest overall, co-pilot mode improved human-evaluated quality from 3.8 to 4.38 out of 10, and gpt-4o execution was reported at about $2.33 per paper, showing an 84% cost reduction versus prior autonomous research methods.
    • Notably, automatic NeurIPS-style reviews averaged 6.1 out of 10, while human reviewers averaged 3.8, so the system is better viewed as a research assistant than a reliable autonomous scientist.
    AbstractPDF
  35. 010RAG / EfficiencyarXiv:2412.15605

    Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks

    • The core idea is to replace online RAG pipelines with a scheme that preloads all relevant documents offline into a long-context LLM and reuses the precomputed KV cache at inference time.
    • The method has three stages: knowledge preloading and KV encoding, query-time generation conditioned on the cached state, and cache reset and truncation that removes query-specific tokens while preserving the knowledge cache.
    • As a result, on SQuAD 1.0 and HotPotQA in small, medium, and large settings, CAG is reported to match or outperform BM25 and dense-index RAG baselines while eliminating retrieval time. For HotPotQA-Large, generation is reported to take 2.26 seconds instead of 92.08 seconds for dynamic in-context processing.
    • In scope and tradeoff, the approach removes the embedding model, vector store, and retriever, which simplifies deployment, but it depends on long-context capacity and is best suited to bounded, relatively stable knowledge bases.
    AbstractPDF
  36. 009Medical AI / ReasoningarXiv:2412.18925

    HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs

    • The method converts difficult closed-book medical exam questions into 40K verifiable questions with objective answers, enabling automatic feedback on both answer accuracy and reasoning quality.
    • The core idea is to use a medical verifier as an active learning component. Failed initial chain-of-thought attempts are improved through backtracking, exploring new paths, verification, and revision until a correct trajectory is found.
    • The training first fine-tunes on complex reasoning trajectories approved by the verifier, then applies PPO reinforcement learning with verifier-based rewards to strengthen independent medical reasoning.
    • In terms of results and limits, the 8B model shows an 8.5-point benchmark gain and the 70B model leads open-source medical LLMs, but verifier reliability and transfer from exam-style questions to real clinical ambiguity remain open challenges.
    AbstractPDF
  37. 008ArchitecturearXiv:2412.09764

    Memory Layers at Scale

    • Each memory layer acts as a low-cost learned lookup table that retrieves the top-k trainable keys for an input query and combines their values, rather than using attention over sequence activations.
    • Multiplicative keys reduce retrieval cost, values are sharded across GPUs, and custom CUDA kernels reach near-H100 memory bandwidth, enabling up to 128B-parameter memory in the reported scaling study.
    • As a result, Memory+ outperforms dense models with more than twice the compute budget and also beats MoE and PEER when compute and parameter counts are matched, with the clearest gains on factual memory benchmarks.
    • The implication is that the best design balances dense reasoning capacity with sparse memory capacity, which suggests that future LLMs can separate compute-intensive transformation from efficient knowledge storage.
    AbstractPDF
  38. 007AgentsarXiv:2412.21154

    Aviary

    • From a framework perspective, Aviary provides a gym-style environment for language agents that use reasoning, tools, code, and iterative action-observation loops.
    • In formal terms, the agent is treated as a policy in a language decision process, which is a language-based partially observable Markov decision process formulation.
    • The tasks include molecular cloning, literature-based research QA, and protein stability engineering, all of which require multi-step scientific reasoning.
    • As a result, online learning and test-time scaling let open-source non-frontier LLM agents compete with frontier agents and human experts at much lower cost.
    AbstractPDF
  39. 006Image GenerationarXiv:2412.18653

    1.58-bit FLUX

    • The method quantizes FLUX.1-dev into ternary -1, 0, and +1 weights, reaching a 1.58-bit representation for a large text-to-image model.
    • The quantization method is data-free, relying on self-supervision from the original model instead of an external image dataset.
    • Reported efficiency gains include 7.7x less storage, 5.1x less inference memory, and lower latency through custom low-bit kernels.
    • The benchmarks suggest that image quality at 1024×1024 remains similar despite the extreme compression.
    AbstractPDF
  40. 005Medical AIarXiv:2412.19260

    MEDEC

    • Instead of asking medical QA questions, MEDEC asks a model to verify the accuracy and consistency of existing or generated clinical records.
    • The dataset contains 3,848 clinical texts covering diagnosis, management, treatment, pharmacotherapy, and etiology-related errors.
    • The evaluation compares 17 shared-task systems and frontier LLMs, with physicians used as the human reference point.
    • In conclusion, LLMs show useful error-detection ability but still fall short of physician-level performance, underscoring the need for careful deployment in clinical documentation.
    AbstractPDF
  41. 004Reasoning / EfficiencyarXiv:2412.21187

    On the Overthinking of LLMs

    • The problem is that o1-family models often spend long reasoning traces on simple tasks, which brings only limited accuracy gains while increasing latency and cost.
    • In evaluation, the paper proposes outcome-level and process-level efficiency metrics to measure whether reasoning compute is being used sensibly.
    • The method trains the model with a self-learning strategy to simplify reasoning while preserving answer quality across tasks of different difficulty.
    • The implication is that future reasoning systems need adaptive test-time compute policies, not a habit of using verbose chains of thought for every query.
    AbstractPDF
  42. 003LLMs / Open ModelsarXiv:2501.00656

    OLMo 2

    • The release includes weights, data, code, recipes, logs, and intermediate checkpoints, making it useful for reproducible research and downstream adaptation.
    • In the training recipe, Dolmino Mix 1124 and late-stage curriculum learning improve downstream capability without relying only on more compute.
    • The instruction model OLMo 2-Instruct builds on the Tülu 3 recipe and extends it with reinforcement learning that uses verifiable rewards in the final stage.
    • As a result, the base model remains more transparent while staying competitive with open-weight systems in the Llama 3.1, Qwen 2.5, and Gemma 2 families.
    AbstractPDF
  43. 002AgentsarXiv:2412.16241

    Agents Are Not Enough

    • The core claim is that stronger generative models alone are not enough to make agents effective or sustainable in real user workflows.
    • From a historical perspective, earlier agent systems stalled or failed because autonomy, context, trust, coordination, and user representation were harder than the agent metaphor suggests.
    • Structurally, the proposed ecosystem separates executing agents, user-representing Sims, and user-facing assistants instead of forcing a single model component to handle everything.
    • In industry terms, useful agent systems need not only autonomous action loops but also persistent user context, workflow coordination, and human-facing control interfaces.
    AbstractPDF