HDATF
日本語
相談する
メニュー

論文アーカイブ

収録論文と要約を番号の大きい順に掲載します。

収録論文 1,430資料更新日 Research Monitoringベンチマーク
  1. 281AI / GeneralarXiv:2605.31264

    COLLEAGUE.SKILL: Automated AI Skill Generation via Expert Knowledge Distillation

    • LLM agents are increasingly expected not only to complete isolated tasks but also to capture human expertise, judgment, and interaction style in a constrained form.
    • This paper proposes an automated trace-to-skill distillation system that generates human-based AI skills through expert knowledge distillation.
    • The authors describe the output contract, generation workflow, revision lifecycle, deployment surface, and domain presets implemented in the open-source system.
    概要PDF
  2. 280Machine LearningarXiv:2605.29713

    The Little Book of Generative AI Foundations: An Intuitive Mathematical Primer

    • This book provides a concise, derivation-driven introduction to the mathematical foundations of modern generative artificial intelligence.
    • Its goal is to make the structure of generative modeling more accessible without stripping away the mathematical content needed to understand how these models are derived and how they relate to one another.
    • The book is intended as a foundational primer for researchers, practitioners, and students who are curious about mathematics.
    概要PDF
  3. 279LLMs / NLParXiv:2605.31268

    Mellum2 Technical Report

    • This report introduces Mellum 2, an open-weight 12B-parameter MoE language model with 2.5B active parameters per token.
    • Across code generation, math and reasoning, tool use, knowledge, and safety benchmarks, Mellum 2 is competitive with open-weight baselines in the 4B to 14B range while operating at the per-token compute cost of a 2.5B dense model.
    • The authors release the base, instruct, and thinking checkpoints under the Apache 2.0 license together with this report, which describes the underlying architectural choices, data pipelines, and training recipe.
    概要PDF
  4. 278LLMs / NLParXiv:2605.31584

    LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards

    • Experiments on five long-context benchmarks with three reasoning LLMs, ranging from 4B to 30B, show that LongTraceRL consistently outperforms strong baselines and encourages comprehensive, evidence-based reasoning.
    • Reinforcement learning with verifiable rewards, or RLVR, has shown promise for this task, but prior methods are limited to low-confusion distractors and sparse, outcome-only reward signals, so they fail to supervise intermediate reasoning steps.
    • This paper proposes LongTraceRL to address these issues.
    概要PDF
  5. 277Computer VisionarXiv:2605.30561

    VLM3: Vision Language Models Are Native 3D Learners

    • The authors therefore propose VLM3, a simple and scalable extension method that enables standard VLMs to handle a wide range of 3D tasks effectively.
    • A deep large-scale study shows that unifying focal length, using text-based pixel references, and mixing and scaling data are sufficient for effective 3D learning.
    • Vision-language models let a single model solve diverse vision tasks through prompting.
    概要PDF
  6. 276Computer VisionarXiv:2605.31604

    Representation Forcing for Bottleneck-Free Unified Multimodal Models

    • The paper proposes Representation Forcing (RF), a technique that closes this gap by making representation prediction an intrinsic capability of the model.
    • If this is simply removed, the model must learn both high-level structure and low-level details directly from raw pixels, which creates a quality gap.
    • Specifically, RF forces the decoder to autoregressively predict visual representations as intermediate tokens before pixels, and those tokens are then kept in context to guide pixel diffusion within the same backbone.
    概要PDF
  9. 273LLMs / NLParXiv:2605.29682

    Scaling Laws for Agent Harnesses via Effective Feedback Compute

    • Step complexity, L, is the minimum number of reasoning or action steps required.
    • A higher-quality router greatly increases eta by ensuring that the model calls the right tool at the right time.
    • Better state management ensures that useful information is preserved and used, which prevents redundant interactions.
    概要PDF
  10. 272Computer VisionarXiv:2605.30248

    GenClaw: Code-Driven Agentic Image Generation

    • In the final rendering stage, the image model uses this physical draft as a visual guide to generate a realistic scene.
    • If the search result is wrong, it is a cognition error.
    • If the layout is illogical, it is a code error.
    概要PDF
  11. 271Computer VisionarXiv:2605.30215

    Déjà View: Looping Transformers for Multi-View 3D Reconstruction

    • Register tokens are four tokens used as a global scratchpad for the transformer.
    • Camera tokens are special tokens that store information about the pose and intrinsics of the current view.
    • Frame attention is a subblock that operates independently on each image, allowing the model to infer local geometry within a single view.
    概要PDF
  12. 270Computer VisionarXiv:2605.30263

    minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models

    • Training stage: Camera controllability does not appear immediately. With the HY1.5 backbone, the model showed no responsiveness at 2,000 steps, began showing signs of control at 5,000 steps, and stabilized only after 8,000 steps.
    • Embodied AI: It gives robots a mental simulator that predicts the visual consequences of their actions.
    • Game development: It generates infinite, interactive environments that respond instantly to player movement.
    概要PDF
  13. 269Machine LearningarXiv:2605.27734

    Learn from your own latents and not from tokens: A sample-complexity theory

    • Predictor: A network that takes a tuple of latent values as input and tries to predict the distribution of its sibling latent values.
    • Clusterer: A network that compresses the output of the predictor into a discrete code or cluster identity, and that code becomes the input to the next layer.
    • Teacher as a lifter: The EMA teacher gradually incorporates the learned latent values into its own targets, which shifts the prediction task from token level to latent level during training.
    概要PDF
  14. 268AI / GeneralarXiv:2605.27276

    SIA: Self Improving AI with Harness & Weight Updates

    • In SIA-H, which uses only the harness, the agent built a classification pipeline and reached 50.0% accuracy, but performance plateaued once the prompt and code hit their limits.
    • In SIA-W+H, which adds weight updates, the model learned to internalize legal nuance, and accuracy jumped to 70.1%, far above the previous best of 45.0%.
    • In SIA-W+H, the system achieved a 14.02x speedup over the initial baseline. Here, weight updates were decisive because the model learned specific H100 hardware patterns such as shared-memory tiling and register accumulation, which scaffold iteration alone could not discover.
    概要PDF
  15. 267Machine LearningarXiv:2605.29157

    Parallax: Parameterized Local Linear Attention for Language Modeling

    • Perplexity: On datasets such as LAMBADA and WikiText, Parallax achieved consistently lower perplexity than Transformer baselines, a measure of how well the model predicts samples.
    • Downstream tasks: On zero-shot benchmarks such as HellaSwag and ARC, which cover question answering and commonsense reasoning, Parallax showed clear performance gains.
    • Control experiments: To confirm that the gains were not simply due to more parameters or more compute, the authors built parameter-matched and compute-matched versions of the standard Transformer. Parallax outperformed both, suggesting that the improvement comes from the parameterized local linear mechanism itself.
    概要PDF
  16. 266Computer VisionarXiv:2605.27310

    How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

    • The key failure mode of visual thinking is identified as "generated but not used."
    • View Dropout, or VDrop, is a training method that masks part of the input view during the answer segment to force the model to pass through a thinking image.
    • The Learnability-Informativeness, or L-I, tradeoff framework compares three visual-thinking representations: panorama, top-down, and point matching.
    • The model is trained on 8K synthetic Infinigen Indoors samples and evaluated on five real OOD benchmarks.
    • Only panorama visual thinking combined with VDrop scores highly on both axes and achieves the best OOD performance.
    • It achieves a 6.7-point OOD improvement over prior methods that were trained with three times more data.
    概要PDF
  17. 265LLMs / NLParXiv:2605.28773

    Rethinking Memory as Continuously Evolving Connectivity

    • Success: Through these topological and unit-level edits, the agent successfully completed the task.
    • Self-correcting retrieval: Environmental feedback dynamically fixes over-connection and under-connection problems.
    • Granularity adjustment: The level of detail in stored memory is adapted to the current task requirements.
    概要PDF
  18. 264Machine LearningarXiv:2605.28109

    Long Live The Balance: Information Bottleneck Driven Tree-based Policy Optimization

    • It improves efficiency by reusing prefixes and taking advantage of KV caching, which lets it generate more trajectories, or a higher G, within the same token budget than independent sampling.
    • By branching according to the IB-Score, the model focuses exploration on the 'critical' steps where both high diversity and a strong signal toward the correct answer are present.
    概要PDF
  19. 263LLMs / NLParXiv:2605.28814

    Self-Improving Language Models with Bidirectional Evolutionary Search

    • The merge operator takes two different reasoning paths that share the same starting point and merges their unique suffixes, allowing the model to synthesize successful components from multiple attempts.
    • Deletion removes a single internal step from a trajectory, which helps simplify reasoning or eliminate redundant filler steps that can send the model in the wrong direction.
    • Transposition replaces one step in a trajectory with a step from another trajectory, similar to swapping out a piece of logic to see whether a different approach fits the same context better.
    概要PDF
  20. 262RoboticsarXiv:2605.30280

    Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

    • The auxiliary vision-language data, 8.5 percent of the training mix, is general-purpose VLM data that helps the model avoid forgetting how to describe scenes or reason about objects.
    • The unified architecture shows that a single model can control multiple embodiments, including arms, hands, and mobile bases, through a shared action expert and text conditioning.
    • The paper shows that vision-language training directly helps robot behavior by providing better object recognition and command parsing.
    概要PDF
  21. 261AI / GeneralarXiv:2605.26340

    ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence

    • Score verification, I1: the auditor reruns the agent's code on a gold evaluator to confirm that the reported score is reproducible.
    • Spec violation, I2: the system checks whether the agent bypassed the rules, for example by hard-coding answers or importing restricted data.
    • Reference verification, I3: every citation is cross-checked against global scholarly databases to detect hallucinations.
    概要PDF
  22. 260AI / GeneralarXiv:2605.28655

    AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation

    • The shared forum (F) is a space where agents post structured discussions, hypothesize why specific changes worked, and critique the proposals of other agents.
    • Removing the analyst led to a substantial drop in proposal quality because the experimental agents lacked strategic oversight to avoid redundant or weakly justified attempts.
    • Removing inter-agent feedback increased the rate of false starts, where the system pursued hypotheses that other agents could easily identify as flawed.
    概要PDF
  23. 259Computer VisionarXiv:2605.27367

    SpatialBench: Is Your Spatial Foundation Model an All-Round Player?

    • DENSE focuses on long-time-series video sequences and feeds in nearly every frame up to the 500-frame limit.
    • For robotics, use models trained on DA-Next-5M or similar first-person datasets; standard indoor and outdoor foundation models are likely to fail on wrist-mounted manipulation tasks.
    • For AR and VR, streaming or chunk-based models are preferred because they must handle long sessions without exceeding the memory limits of mobile devices or headsets.
    概要PDF
  24. 258ResearcharXiv:2605.28781

    The sum-product conjecture is false for real numbers

    • A counterexample to the sum-product conjecture over the reals (1.1), constructing sets A with both |A+A| and |AA| at most |A|^{2-c}.
    • A higher-dimensional Balog-Wooley-type structure using algebraic integers from totally real number fields.
    • A counterexample to the many sums and products conjecture as well: max(|kA|, |A^(k)|) ≤ |A|^{C log k / log log k}.
    • Variants over various rings, including p-adics, the finite field Fp, and the positive-characteristic function field Fq((t)).
    • New lower bounds for the number of solutions to the linear equation x1+...+xk=1 in multiplicative groups, along with applications to unit equations.
    • GPT-5.5 Pro was used for initial exploration, but the final proof was generated mostly by humans, inspired by the OpenAI counterexample to the unit-distance conjecture.
    概要PDF
  25. 257Computer VisionarXiv:2605.26519

    R³: 3D Reconstruction via Relative Regression

    • Scale stability: The relative motion between two adjacent frames, the baseline, is usually small and predictable, whereas the distance from the starting point of a long trajectory is not.
    • Origin independence: Any frame in the sequence can serve as the reference for another frame. The system is no longer chained to the first observation.
    • Novelty: If the visual features of a new frame differ enough from those already stored, that frame is added to the bank.
    概要PDF
  26. 256Machine LearningarXiv:2605.26535

    Recursive Flow Matching

    • Tradeoff removal: while most models force a choice between speed and accuracy, RecFM uses recursive training to improve both at once.
    • Explicit scale alignment: unlike consistency models that distill a student from a teacher, RecFM uses a single network and enforces multi-scale alignment internally, which provides a more robust gradient signal during training.
    • Physical fidelity: by straightening trajectories, RecFM preserves high-frequency details, such as waves and vortices, that are usually lost when generative models are accelerated.
    概要PDF
  27. 255Machine LearningarXiv:2605.26282

    Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization

    • Identify the structural mismatch between exploration and value learning as the key bottleneck in world-model RL.
    • Recast policy optimization as a diffusion process on the latent world model to unify exploration and policy optimization.
    • Anchor the policy to the behavior distribution with a dataset-induced implicit energy function, namely a KL trust region.
    • Correct the score field with cumulative reward from imagined trajectories so it converges toward the optimal Gibbs policy.
    • Show consistent gains across multi-task offline pretraining, online, and offline-to-online settings.
    • Demonstrate scalability through monotonic performance improvements as model capacity increases.
    概要PDF
  28. 254Computer VisionarXiv:2605.28816

    Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players

    • The hub token attends to all agent tokens and to other hub tokens.
    • Bidirectional teacher training: the model is first trained as a standard diffusion model with access to the full temporal sequence. This teacher learns the full distribution of multi-agent interaction with high fidelity.
    • Causal student training: to enable real-time rollouts, the student model is trained with a causal structure. Unlike the teacher, the student sees only the past and the present, and it uses a Sparse Hub Attention mask plus noise levels sampled independently for each frame block.
    概要PDF
  30. 252AI / GeneralarXiv:2605.27366

    MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation

    • Testing is the unit test used to verify that a skill works correctly before it is allowed into the Skill Bank.
    • Memory is a dedicated file, .memory.md, that stores observations and lessons from previous runs.
    • At Level 1, single-node compression, when a particular tool output is extremely large, for example a 20,000-token log file, the method replaces it with a concise summary while leaving the surrounding conversation intact.
    概要PDF
  31. 251Information RetrievalarXiv:2605.24938

    Your Embedding Model is SMARTer Than You Think

    • In efficiency terms, developers can upgrade existing retrieval systems to support multiple vectors without reindexing the entire database or performing expensive retraining.
    • In accuracy terms, SMART points toward a universal retrieval model that works well both for global tasks such as image classification and for local tasks such as finding a specific sentence in a 100-page manual.
    概要PDF
  32. 250LLMs / NLParXiv:2605.25604

    DVAO: Dynamic Variance-adaptive Advantage Optimization for Multi-reward Reinforcement Learning

    • In terms of usability, the algorithm adaptively finds the optimal balance from data, reducing the need for practitioners to manually tune reward weights.
    • In terms of stability, it preserves the efficiency of the GRPO framework while adding the theoretical stability needed to avoid size explosion.
    • In terms of synergy, cross-objective regularization ensures that the model does not optimize only the easiest reward while sacrificing harder and more important goals such as reasoning accuracy.
    概要PDF
  33. 249Computer VisionarXiv:2605.26102

    InstructSAM: Segment Any Instance with Any Instructions

    • During decoding, the final score head determines whether the query corresponds to a valid target instance, and the segmentation head produces a binary mask.
    • For a single target, it identifies the one specific object mentioned in the instruction.
    • For multiple targets, it finds all instances that satisfy a complex condition, such as "all wooden chairs."
    概要PDF
  35. 247Computer VisionarXiv:2605.25979

    LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence

    • In stage 3, which focuses on long videos, the model is exposed to a length-hierarchical video caption corpus of about 8 million clips and 104 billion tokens in total, with videos ranging from 30 seconds to 15 minutes, so that it learns to maintain context over long durations.
    • In stage 4, which focuses on codec and spatial reasoning, codec-native training is introduced and 10 to 15 minute videos are re-encoded with a variable-length GOP pipeline, while the model is also trained on the LLaVA-OneVision-2-Spatial-4M corpus containing 4 million samples focused on 2D and 3D spatial reasoning such as identifying objects in indoor scans or tracking motion in embodied simulators.
    • The codec-stream approach performs substantially better on temporal grounding and visual perception because it reallocates tokens to event-bearing moments and captures motion dynamics that uniform sampling misses.
    概要PDF
  36. 246LLMs / NLParXiv:2605.26431

    Probing Minimalist Phase Structure in LLMs: What Universal Dependencies Cannot Represent

    • Use wh-movement stimuli designed so that UD tree distance remains invariant across conditions to detect structure beyond UD.
    • In 12 of 13 LLMs across four families, the authors find a distance slope proportional to Minimalist phase count, with βfin > βinf > 0.
    • Discover 13 out of 13 sign asymmetries that exactly match within-phase cohesion predictions, a result impossible to capture with UD-based probing.
    • Introduce canonical-layer reporting, which removes degrees of freedom from per-contrast peak reporting and applies a stricter standard.
    • Confirm the causal relevance of probe results with activation patching in 12 of 13 models.
    • The results suggest that distributional pretraining can induce formal syntactic representations that go beyond the limits of annotation-based probing.
    概要PDF
  37. 245Machine LearningarXiv:2605.26106

    Looped Diffusion Language Models

    • The Head is made of a fixed number of layers that run once at the start of the forward pass, and its main role is to process the initial input embedding and provide a starting representation for the rest of the network.
    • The Tail is made of a fixed number of layers that run once after the Mid-block finishes looping, and these layers finalize the representation and project it into vocabulary space for token prediction.
    概要PDF
  38. 244Machine LearningarXiv:2605.26379

    When Does LeJEPA Learn a World Model?

    • SIGReg and VICReg enforce whitening, meaning they make each dimension independent with equal variance, and both methods maintained very high identifiability across all dimensions.
    • InfoNCE is a widely used contrastive learning method. It works well in low dimensions, but in high-dimensional spaces it became extremely sensitive to hyperparameter tuning and often failed to preserve structure recovery without careful adjustment.
    • For data collection, if we want a robot to learn a high-quality representation of its environment, the data it 'dreams' during pretraining should be biased and goal-oriented, rather than close to isotropic random walks.
    概要PDF
  39. 243Computer VisionarXiv:2605.27295

    Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini

    • Gemini-based native multimodal processing maps text, images, video, audio, and interleaved combinations into a single embedding space.
    • It is trained with vectors of up to 3,072 dimensions and large-scale multi-task, multi-stage contrastive learning.
    • It achieves SOTA with 62.9 R@1 on MSCOCO, 68.8 NDCG@10 on Vatex, 69.9 on MTEB Multilingual, and 84.0 on MTEB Code.
    • Compared with traditional late-fusion models such as CLIP and SigLIP, deep modal fusion provides cross-modal synergy.
    • It shows strong zero-shot generalization in specialized domains such as astronomy, biology, art, and cooking.
    • It delivers out-of-the-box performance that can be used immediately for practical downstream tasks such as RAG, recommendation, and search.
    概要PDF
  40. 242AI / GeneralarXiv:2605.25624

    CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

    • The discriminator is isolated from the generator's implementation details and sees only the task t and the resulting environment state, and it then writes a reward.py script that should return 0.0 for the initial state and 1.0 for the golden state.
    • The Qwen3.5-35B model (A3B) improved success rate from 54.5 percent to 62.1 percent.
    • The Qwen3.5-397B model (A17B) reached a 72.6 percent success rate, which is an increase of more than 10 percentage points.
    概要PDF
  41. 241AI / GeneralarXiv:2605.26494

    The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

    • A MoE architecture with about 10 billion active parameters achieves both cost efficiency and competitive performance.
    • It builds a large synthetic-data pipeline for a wide range of agent tool environments, including code, search, spreadsheet, and slide generation.
    • Interleaved-thinking SFT trains trajectories where reasoning and action alternate, enabling the model to handle complex agent loops.
    • CISPO policy optimization and agent RL strengthen agent capability by modeling interaction with the environment as an MDP.
    • It demonstrates strong performance across practical domains such as financial analysis, document management, and role playing.
    • A three-axis scaling strategy for reasoning data, across query side, response side, and training side, improves out-of-distribution generalization.
    概要PDF
  42. 240LLMs / NLParXiv:2605.26099

    Language Models Need Sleep

    • GSM-Infinite is a math reasoning benchmark that generates long problems with many arithmetic operations and tests a model's ability to preserve intermediate results and logic across thousands of tokens.
    • Latency retention: By moving heavy computation to an offline stage, the model stays fast during interactive use.
    • Deep reasoning activation: It allows the model to solve problems that are logically deeper than its physical layer count would normally permit.
    概要PDF
  43. 239AI / GeneralarXiv:2605.26114

    MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research

    • On a subset of 59 tasks run on real phones, the pass rate of the trained model jumped from 32.2% to 72.9%.
    • Analysis showed that the model retained 95.1% of the gains when moving from simulation to the real environment.
    • Democratizing training: Because it uses little RAM and runs on standard servers, it enables online RL for research groups that do not have large compute clusters.
    概要PDF
  44. 238Machine LearningarXiv:2605.22731

    Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation

    • SFT is not inherently destructive, but its off-policy nature becomes risky under high pressure. Developers should monitor how closely the dataset distribution aligns with the model's natural distribution.
    • On-policy learning is a stabilizer. Whether through RL or distillation, sampling from the learner's own distribution helps keep updates well behaved and preserves the model's native capabilities better than off-policy methods.
    • Distillation is more than imitation. By separating states from signals, on-policy distillation can let a student surpass the teacher when supervision is dense enough, using continuations rather than one-step token logits.
    概要PDF
  45. 237RoboticsarXiv:2605.23733

    Any2Any: Efficient Cross-Embodiment Transfer for Humanoid Whole-Body Tracking

    • A sparse scattering matrix handles the basic correspondence mapping target joints to source-batch joints, discarding extra joints and filling missing joints with zeros.
    • The parallel joint coupling matrix handles nonserial mechanisms used by some robots, such as parallelogram-driven ankles, and corrects the mismatch between actuated joint angles and actual kinematic angles.
    概要PDF
  46. 236Computer VisionarXiv:2605.25874

    WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation

    • The benchmark uses a comprehensive evaluation framework built from five dimensions: video quality, setting compliance, interaction compliance, consistency, and physical compliance.
    • It covers 289 cases and 1,058 interaction turns across four interaction types: navigation, action, event editing, and viewpoint switching.
    • For navigation, it integrates text, 6-DoF pose, and discrete action inputs to support different input interfaces.
    • It combines 22 automatic sub-metrics with evaluation from expert vision models and LMMs, and it validates alignment with human judgments.
    • Evaluation of 20 SOTA models showed that no single model dominated all dimensions.
    • It provides detailed diagnostic insights into each model's strengths, weaknesses, and open problems.
    概要PDF
  48. 234RoboticsarXiv:2605.23856

    Point Tracking Improves World Action Models

    • Register token r is a learnable token that helps global information processing.
    • Models that use only latent representations lacked the precision needed for long-horizon action, and they failed when visual cues were subtle.
    • Models that use only tracks lacked the semantic grounding needed to identify which object to interact with.
    概要PDF
  49. 233Computer VisionarXiv:2605.25343

    Toward Native Multimodal Modeling: A Roadmap

    • This roadmap systematically classifies NMM models along three axes: T2M for generation, M2T for understanding, and M2M for symmetric modeling.
    • It compares two design philosophies: full discretization through digital tokens versus preserving modality characteristics through a continuous feature space.
    • It summarizes trends in extreme VAE compression and dynamic sparse-attention pruning techniques for addressing token explosion.
    • It analyzes explicit-rule and implicit-emergence approaches for teaching physical law understanding, such as rigid-body dynamics, gravity, and collision, in video generation.
    • It proposes unified timeline anchoring and deep architectural coupling for precise audio-visual synchronization.
    • It includes data and evaluation strategies, such as physical decoupling and encoder-free modeling, to address the comprehension-generation dilemma.
    概要PDF
  50. 232Machine LearningarXiv:2605.23857

    Strong Teacher Not Needed? On Distillation in LLM Pretraining

    • Downstream accuracy refers to performance on benchmarks such as MMLU and GSM8K.
    • Do not wait for the perfect teacher. When training a larger new model, it is beneficial to use the best model you currently have as the teacher, even if that teacher is smaller or less trained than the new model's target.
    • Prioritize compatibility. A moderately strong teacher that is structurally similar to the student may be more effective than a huge, state-of-the-art model.
    概要PDF