Paper Archive

Every archived paper with its summary, highest number first.

Papers 1,193Archive updated BenchmarksRSS feed
  1. 244Machine LearningarXiv:2605.26379

    When Does LeJEPA Learn a World Model?

    • SIGReg and VICReg enforce whitening, meaning they make each dimension independent with equal variance, and both methods maintained very high identifiability across all dimensions.
    • InfoNCE is a widely used contrastive learning method. It works well in low dimensions, but in high-dimensional spaces it became extremely sensitive to hyperparameter tuning and often failed to preserve structure recovery without careful adjustment.
    • For data collection, if we want a robot to learn a high-quality representation of its environment, the data it 'dreams' during pretraining should be biased and goal-oriented, rather than close to isotropic random walks.
  2. 243Computer VisionarXiv:2605.27295

    Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini

    • Gemini-based native multimodal processing maps text, images, video, audio, and interleaved combinations into a single embedding space.
    • It is trained with vectors of up to 3,072 dimensions and large-scale multi-task, multi-stage contrastive learning.
    • It achieves SOTA with 62.9 R@1 on MSCOCO, 68.8 NDCG@10 on Vatex, 69.9 on MTEB Multilingual, and 84.0 on MTEB Code.
    • Compared with traditional late-fusion models such as CLIP and SigLIP, deep modal fusion provides cross-modal synergy.
    • It shows strong zero-shot generalization in specialized domains such as astronomy, biology, art, and cooking.
    • It delivers out-of-the-box performance that can be used immediately for practical downstream tasks such as RAG, recommendation, and search.
  3. 242AI / GeneralarXiv:2605.25624

    CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

    • The discriminator is isolated from the generator's implementation details and sees only the task t and the resulting environment state, and it then writes a reward.py script that should return 0.0 for the initial state and 1.0 for the golden state.
    • The Qwen3.5-35B model (A3B) improved success rate from 54.5 percent to 62.1 percent.
    • The Qwen3.5-397B model (A17B) reached a 72.6 percent success rate, which is an increase of more than 10 percentage points.
  4. 241AI / GeneralarXiv:2605.26494

    The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

    • A MoE architecture with about 10 billion active parameters achieves both cost efficiency and competitive performance.
    • It builds a large synthetic-data pipeline for a wide range of agent tool environments, including code, search, spreadsheet, and slide generation.
    • Interleaved-thinking SFT trains trajectories where reasoning and action alternate, enabling the model to handle complex agent loops.
    • CISPO policy optimization and agent RL strengthen agent capability by modeling interaction with the environment as an MDP.
    • It demonstrates strong performance across practical domains such as financial analysis, document management, and role playing.
    • A three-axis scaling strategy for reasoning data, across query side, response side, and training side, improves out-of-distribution generalization.
  5. 240LLMs / NLParXiv:2605.26099

    Language Models Need Sleep

    • GSM-Infinite is a math reasoning benchmark that generates long problems with many arithmetic operations and tests a model's ability to preserve intermediate results and logic across thousands of tokens.
    • Latency retention: By moving heavy computation to an offline stage, the model stays fast during interactive use.
    • Deep reasoning activation: It allows the model to solve problems that are logically deeper than its physical layer count would normally permit.
  6. 239AI / GeneralarXiv:2605.26114

    MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research

    • On a subset of 59 tasks run on real phones, the pass rate of the trained model jumped from 32.2% to 72.9%.
    • Analysis showed that the model retained 95.1% of the gains when moving from simulation to the real environment.
    • Democratizing training: Because it uses little RAM and runs on standard servers, it enables online RL for research groups that do not have large compute clusters.
  7. 238Machine LearningarXiv:2605.22731

    Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation

    • SFT is not inherently destructive, but its off-policy nature becomes risky under high pressure. Developers should monitor how closely the dataset distribution aligns with the model's natural distribution.
    • On-policy learning is a stabilizer. Whether through RL or distillation, sampling from the learner's own distribution helps keep updates well behaved and preserves the model's native capabilities better than off-policy methods.
    • Distillation is more than imitation. By separating states from signals, on-policy distillation can let a student surpass the teacher when supervision is dense enough, using continuations rather than one-step token logits.
  8. 237RoboticsarXiv:2605.23733

    Any2Any: Efficient Cross-Embodiment Transfer for Humanoid Whole-Body Tracking

    • A sparse scattering matrix handles the basic correspondence mapping target joints to source-batch joints, discarding extra joints and filling missing joints with zeros.
    • The parallel joint coupling matrix handles nonserial mechanisms used by some robots, such as parallelogram-driven ankles, and corrects the mismatch between actuated joint angles and actual kinematic angles.
  9. 236Computer VisionarXiv:2605.25874

    WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation

    • The benchmark uses a comprehensive evaluation framework built from five dimensions: video quality, setting compliance, interaction compliance, consistency, and physical compliance.
    • It covers 289 cases and 1,058 interaction turns across four interaction types: navigation, action, event editing, and viewpoint switching.
    • For navigation, it integrates text, 6-DoF pose, and discrete action inputs to support different input interfaces.
    • It combines 22 automatic sub-metrics with evaluation from expert vision models and LMMs, and it validates alignment with human judgments.
    • Evaluation of 20 SOTA models showed that no single model dominated all dimensions.
    • It provides detailed diagnostic insights into each model's strengths, weaknesses, and open problems.
  10. 234RoboticsarXiv:2605.23856

    Point Tracking Improves World Action Models

    • Register token r is a learnable token that helps global information processing.
    • Models that use only latent representations lacked the precision needed for long-horizon action, and they failed when visual cues were subtle.
    • Models that use only tracks lacked the semantic grounding needed to identify which object to interact with.
  11. 233Computer VisionarXiv:2605.25343

    Toward Native Multimodal Modeling: A Roadmap

    • This roadmap systematically classifies NMM models along three axes: T2M for generation, M2T for understanding, and M2M for symmetric modeling.
    • It compares two design philosophies: full discretization through digital tokens versus preserving modality characteristics through a continuous feature space.
    • It summarizes trends in extreme VAE compression and dynamic sparse-attention pruning techniques for addressing token explosion.
    • It analyzes explicit-rule and implicit-emergence approaches for teaching physical law understanding, such as rigid-body dynamics, gravity, and collision, in video generation.
    • It proposes unified timeline anchoring and deep architectural coupling for precise audio-visual synchronization.
    • It includes data and evaluation strategies, such as physical decoupling and encoder-free modeling, to address the comprehension-generation dilemma.
  12. 232Machine LearningarXiv:2605.23857

    Strong Teacher Not Needed? On Distillation in LLM Pretraining

    • Downstream accuracy refers to performance on benchmarks such as MMLU and GSM8K.
    • Do not wait for the perfect teacher. When training a larger new model, it is beneficial to use the best model you currently have as the teacher, even if that teacher is smaller or less trained than the new model's target.
    • Prioritize compatibility. A moderately strong teacher that is structurally similar to the student may be more effective than a huge, state-of-the-art model.
  13. 231Computer VisionarXiv:2605.26115

    TriSplat: Simulation-Ready Feed-Forward 3D Scene Reconstruction

    • It proposes a structure that uses oriented triangles instead of Gaussians as the primitive, so the rendered output is already a mesh.
    • It derives normals from the predicted point map and refines them with an image-conditioned U-Net to keep triangle orientation aligned with geometry.
    • Mono-normal bootstrapping plus validity-aware masking stabilizes early training, and opacity and blur scheduling gradually sharpen the surface.
    • It achieves better geometry fidelity and competitive novel-view synthesis quality than Gaussian feed-forward baselines on RealEstate10K and DL3DV.
    • Zero-shot ScanNet evaluation confirms cross-dataset generalization.
    • It can be used directly in physics engines, collision detectors, and standard rendering pipelines without post-processing such as TSDF fusion.
  14. 230Machine LearningarXiv:2605.23872

    Training-Free Looped Transformers

    • This method delivered consistent gains on knowledge-intensive benchmarks; for example, Qwen3-4B-Instruct improved by 2.64 percentage points on MMLU-Pro.
    • Unlike simple looping, which collapses as more iterations are added, the damped RK method remains stable or keeps improving as K increases, as shown in Figure 3.
    • The extra cost is proportional to the number of layers passed through, but because only a small window, such as four layers, is looped, the total runtime overhead is often only about 20 percent for a 32-layer model when the full decode loop is used; the authors also introduce a bypass mode that loops only during initial prompt processing, which makes token-generation overhead close to zero.
  15. 229Computer VisionarXiv:2605.23903

    Geo-Align: Video Generation Alignment via Metric Geometry Reward

    • Standard training losses compare pixels or abstract features, and they do not understand physical units such as meters or degrees.
    • By using a 3D evaluator, the model learns to interpret camera motion in physical units, which resolves the scale drift that has long been a problem in video synthesis.
    • This RL framework removes the need for hard-to-obtain synchronized multiview video, allowing the model to learn from a wide range of real-world distributions.
  16. 228Machine LearningarXiv:2605.23901

    LLMs as Noisy Channels: A Shannon Perspective on Model Capacity and Scaling Laws

    • Irreducible noise, e, corresponds to the constant term in Chinchilla-style laws and represents a fundamental structural limit or system entropy that cannot be eliminated by scaling.
    • Strategic scaling: the presence of a U-shaped curve suggests that there is an optimal sweet spot for scaling N and D at a given noise level. Blind scaling is not always the answer, and developers should consider the signal-to-noise ratio of their training setup.
    • Noise awareness: exponent analysis shows that data noise often grows faster than the learning signal, with delta greater than beta. This underscores the critical importance of data curation and noise mitigation. If your training data is noisy, simply adding more of it can eventually backfire.
  17. 227ResearcharXiv:2605.23463

    StepAudio 2.5 Technical Report

    • ASR accuracy: The model achieved an average character error rate (CER) of 2.97% on Chinese benchmarks and a word error rate (WER) of 3.68% on English benchmarks. Its 32K context window makes it especially strong on long-form audio.
    • Decoding efficiency: Thanks to the MTP-5 mechanism, the model achieved a real-time factor (RTF) of 0.0053, making it faster than several specialized ASR systems. The acceptance rate of MTP tokens shows a consistent decay factor of about 0.9, validating the choice of five preceding branches.
    • TTS preference: In arena-style evaluation, StepAudio 2.5 TTS achieved an overall win rate of 67.6% against existing competitors such as ElevenLabs and Gemini Flash TTS.
  18. 226Computer VisionarXiv:2605.23888

    GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction

    • These chunk-wise predictions are merged back into the global grid by averaging the values in overlapping regions.
    • Scalability: chunk-based generation allows generative models, which were previously limited to single objects, to be applied to large multi-room scenes.
    • Geometric control: spatially grounded conditioning keeps the model's hallucinations anchored to the physical reality defined by camera pose and input images.
  19. 225AI / GeneralarXiv:2605.23904

    SkillOpt: Executive Strategy for Self-Evolving Agent Skills

    • A skill optimized for a large model, such as GPT-4o, can be successfully applied to a smaller model, such as GPT-4o-mini, and the smaller model often captures much of the improvement, which suggests that the skill provides general procedural knowledge missing from the smaller model's weights.
    • A skill learned in one execution environment, such as the default chat interface, often remains valid when transferred to a more complex agent harness, such as a code execution sandbox, which indicates that the learned rule captures task-level logic rather than exploiting quirks of the training environment.
    • Procedures learned on one dataset, such as OlympiadBench, also improved performance on a related but distinct dataset, such as Omni-MATH, further showing that SkillOpt does not merely overfit to a specific set of training examples.
  20. 223Computer VisionarXiv:2605.23902

    PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion

    • The pixel diffusion prior is a 1.3B-parameter Transformer optimized for high-resolution synthesis with a rectified-flow objective.
    • The latent adapter is a convolutional network that processes the input latent z and aligns it to the spatial grid of the pixel Transformer, ensuring that the latent preserves global structure and semantic information.
    • Sigma-aware gating is the key innovation that controls how much the model relies on the latent condition versus its own learned prior, with the latent influence adjusted by the noise level sigma through a gating function.
  21. 222RoboticsarXiv:2605.22283

    Spatial Memory for Out-of-Vision Manipulation in Vision-Language-Action

    • The geometric estimator network, VGGT, estimates the camera pose and rough scene geometry to establish a global 3D coordinate system.
    • The time needed to find the target object was reduced by about 40 to 59 percent.
    • The robot showed fewer viewpoint corrections, meaning it less often had to readjust after the head camera located the target, which suggests that spatial memory provided sufficiently reliable estimates for the robot to commit to grasping.
  22. 221RoboticsarXiv:2605.22183

    Action with Visual Primitives

    • Instruction and scene understanding: a pretrained VLM processes images and text to produce multimodal context tokens that capture semantic relationships between commands, such as "pick up the red chess piece," and visual scenes.
    • Spatial compositional generalization: the model was trained to move pieces in two steps, such as moving piece A to C and then C to B. At test time, it was asked to move A directly to B, a sequence it had never seen before. Baseline models failed completely, with a 0 percent success rate, while AVP retained an 83 percent success rate. Because the VLM could infer a new where-to primitive for the direct path, the action expert could execute how even though the exact trajectory was novel.
    • Cross-domain generalization: AVP was tested on unseen object categories and different backgrounds, such as switching from a chessboard to a plain white cloth.
  23. 220AI / GeneralarXiv:2605.22681

    Forecasting Scientific Progress with Artificial Intelligence

    • The knowledge gap is the observed performance improvement when a model can access relevant information that existed before the cutoff date, and it measures how much of a model's failure comes simply from not knowing already known facts.
    • The prediction gap is the remaining difference between performance and actual outcomes even when the model has full pre-cutoff knowledge, and it captures the intrinsic difficulty of predicting the unknown.
    • The models consistently reported high confidence in their predictions even when empirical accuracy was low, and this calibration error makes it difficult for human researchers to know when to trust AI predictions.
  24. 219LLMs / NLParXiv:2605.22821

    Tokenisation via Convex Relaxations

    • For the integer-only approach, linear programming selects only tokens it is almost certain about, meaning tokens with values of 0.999 or higher. This often yields a vocabulary smaller than the allowed budget.
    • Proven optimality limit: now we know how much room remains to squeeze tokenization for compression. Since BPE is already within 1% of the limit at scale, researchers can focus more on other properties such as language morphology or robustness rather than compression.
    • Non-greedy alternative: ConvexTok builds a vocabulary by considering the entire dataset at once, which makes better use of the vocabulary budget.
  25. 218AI / GeneralarXiv:2605.22166

    Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents

    • Action realization failure, or 33.3 percent, occurs when the model has the right intent but fails to express it in an executable format, such as by omitting arguments, using the wrong delimiter, or generating free-form text instead of structured tool calls.
    • Environment contract mismatch, or 23.2 percent, occurs when the model generates a syntactically valid action but violates the concrete rules of the tool or API, such as by accessing a non-existent database column or using an invalid flight code.
    • Trajectory degeneration, or 33.6 percent, occurs when the agent falls into a degenerate state, such as repeating the same invalid action, stalling without progress, or failing to recover from an error even after receiving feedback.
  26. 217AI / GeneralarXiv:2605.22763

    Advancing Mathematics Research with AI-Driven Formal Proof Search

    • Lemma hallucinations: Sometimes the agent claimed that a false lemma was a well-known result in the literature. Because the agent did not attempt to formally prove that specific lemma, the error was only discovered late in the process.
    • Formalization as a filter: A formal proof assistant provides ground truth that makes AI-generated mathematics useful to researchers. It removes the need for humans to check basic logical errors.
    • AI for strategy discovery: AI can help mathematicians by proposing high-level plans or identifying candidate counterexamples that humans might miss.
  27. 216Bioinformatics / GenomicsbioRxiv:2025.07.24.666556

    Scalable, Fast and Accurate Differential Gene Expression Testing from Millions of Cells of Multiple Patients

    • Core problem: cells in single-cell RNA-seq are statistically dependent, which breaks the assumptions of traditional differential expression tests designed for independent bulk samples.
    • Bayesian framework: it restores the independence structure at the patient level, cleanly separating inter-individual variation from true transcriptional differences.
    • Multi-GPU variational inference enables computational scalability to more than 10 million cells across thousands of samples from clinical cohorts, atlas projects, and drug-response screens.
    • Practical impact: it makes single-cell differential expression analysis possible at scale for the first time, without the computational bottlenecks or mathematical limitations of existing methods.
  28. 215AI Safety / TrustarXiv:2605.01428

    Hallucinations Undermine Trust; Metacognition is a Way Forward

    • The central argument is that most factuality gains come from expanding the boundary of knowledge, not from improving awareness of that boundary, meaning the distinction between what the model knows and does not know.
    • The model may fundamentally lack the ability to separate truth from error, which creates an inherent tradeoff between removing hallucinations and preserving usefulness.
    • The paper proposes faithful uncertainty, meaning that the model's linguistic uncertainty should match its intrinsic uncertainty, which offers a third path beyond either answering or refusing.
    • For agent systems, uncertainty awareness becomes a control layer that governs when to explore, what to trust, and how to weight retrieved information.
    • Empirical evidence, including poor generalization of truth probes, confident hallucinations, and failures of alignment methods, supports the existence of the discriminative gap.
  29. 214AI / GeneralarXiv:2605.22794

    MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systems

    • A rejection gate is implemented in a pre-call hook to block inefficient batching patterns.
    • Root-cause remediation is possible because, by targeting the harness itself, MOSS can fix architectural flaws that text-based evolution cannot touch.
    • Production usefulness comes from the integration of failure batching, test verification, and safe container replacement, which makes the idea of self-rewriting practical in real applications.
  30. 213Machine LearningarXiv:2605.21768

    Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents

    • Surprisingly, these performance gains were achieved by training on only two full dialogue trajectories, which demonstrates the effectiveness of the RL paradigm.
    • It shows strong performance on out-of-distribution benchmarks such as LongMemEval and MSC-Self-Instruct.
    • The training gains observed on the 3B model transferred to the 7B model as well.
  31. 212Computer VisionarXiv:2605.21605

    GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation

    • Failure memory is a lesson about how to avoid failure.
    • The prompt pool divides 20,000 requests into knowledge-anchored categories, such as facts and events, and quality-anchored categories, such as spatial composition and text layout.
    • Teacher trajectories are created by using a strong model to generate initial successful trajectories and then having a VLM judge inspect them for correctness.
  32. 210AI / GeneralarXiv:2605.22791

    Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention

    • On multi-key retrieval tasks, where several similar associations exist, Gated DeltaNet-2 showed a substantial advantage, confirming that separated gates help the model distinguish competing pieces of information within a fixed-size state.
    • By selectively erasing specific channels, the model avoids overwriting or blurring existing stored data that may still be relevant when new information arrives.
    • SWA handles local, fine-grained interactions within a small fixed window, such as the most recent 2,000 tokens.
  33. 209Machine LearningarXiv:2512.04695

    TRINITY: An Evolved LLM Coordinator

    • The core idea is that a compact coordinator, a 0.6B LLM plus a 10,000-parameter lightweight head, assigns Thinker, Worker, and Verifier roles on each turn and delegates work across multiple LLMs.
    • Optimized with a separable covariance matrix adaptation evolution strategy (CMA-ES), it outperforms RL, imitation learning, and random search under high dimensionality and strict budget constraints.
    • It achieves state of the art on standard benchmarks, including 86.2% on LiveCodeBench, and consistently outperforms individual models and existing multi-agent methods across coding, math, reasoning, and domain knowledge.
    • The coordinator's hidden-state representation provides rich contextualization, and block-epsilon separability in the optimization landscape enables efficient evolutionary search.
    • It generalizes robustly to out-of-distribution tasks without task-specific tuning.
  34. 207Computer VisionarXiv:2605.18746

    ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop

    • Directional bias: AI models often show confirmation bias, repeatedly choosing actions that confirm their initial impression rather than exploring new information, even though that initial impression may be wrong, and they often commit to an answer early in the process with high confidence regardless of whether the evidence is sufficient.
    • Intelligent action planning: Develop an agent that can plan specialized trajectories to resolve spatial uncertainty.
    • Uncertainty management and calibration: Build a model that can understand when it does not have enough information and adjust its confidence accordingly.
  35. 206Machine LearningarXiv:2605.22817

    Vector Policy Optimization: Training for Diversity Improves Test-Time Search

    • Maze Navigation is a synthetic task in which an agent must collect items such as gold and diamonds while avoiding lava hazards. Because the goals are deliberately conflicting, for example, gold is often near lava, it is a perfect testbed for validating Pareto-front coverage.
    • MuSiQue is a multi-hop QA task that rewards both final answer accuracy and citation accuracy for intermediate reasoning steps.
    • ToolRL is a function-calling benchmark in which the model must satisfy structural constraints while also achieving high attribute-level accuracy.
  36. 205Computer VisionarXiv:2605.22718

    WorldKV: Efficient World Memory with World Retrieval and Compression

    • Token pruning: tokens with high similarity scores are treated as redundant and discarded, and only the most distinctive tokens, the bottom P percent by similarity, are kept.
    • LPIPS measures perceptual similarity, and lower is better.
    • PSNR and SSIM measure signal and structural similarity, and higher is better.
  37. 204Computer VisionarXiv:2605.22818

    MotiMotion: Motion-Controlled Video Generation with Visual Reasoning

    • Semantic consistency measures how well a video matches the user's high-level intent.
    • Accessibility is improved by reducing the need for manual frame-by-frame animation or precise technical specifications, which makes advanced video creation available to a broader range of users.
    • Educational benchmarking gives the research community a specialized tool, MotiBench, that focuses on the logical and physical aspects of video generation rather than on visual fidelity alone.
  38. 201Computer VisionarXiv:2605.20743

    Draw2Think: Harnessing Geometry Reasoning through Constraint Engine Interaction

    • Construction fidelity is a model-level property that measures whether a VLM's intended construction matches the requirements of the problem.
    • Explicit reading channel: observations showed that 74 to 91 percent of the system's derived answers included at least one value explicitly returned by the engine, confirming that the VLM actively uses the engine as a source of truth rather than relying only on its internal prior knowledge.
    • Correction and replanning capability: after receiving an error message and using the object deletion tool, the model can correct its path. When a construction path leads to a contradiction or an invalid state, the model can backtrack and try an alternative strategy, just like a human mathematician using compass and straightedge.
  39. 200Computer VisionarXiv:2605.18324

    Improved Baselines with Representation Autoencoders

    • This paper identifies three insights from systematically examining several design choices to simplify and improve RAE.
    • First, it studies a generalized formulation in which the representation is defined as the sum of the last k encoder layers, rather than only the final encoder layer.
  40. 199Machine LearningarXiv:2605.21488

    Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning

    • Maze-Unique: the authors found that task ambiguity is poison for attractor learning, because datasets with multiple valid solutions, such as different shortest paths in a maze, make it hard for the model to form stable attractors. They created a maze variant with a unique path and showed that EqR scales to solving complex 30x30 mazes with high confidence.
    • Scale generalization: at test time, simply increasing the number of iterations lets the model generalize well beyond the complexity seen during training.
    • Self-diagnosis: the fixed-point residual provides an intrinsic confidence score that does not require an external verifier.
  41. 198Computer VisionarXiv:2605.18740

    Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation

    • Question generation: For each small region, the MLLM generates a question q that can be answered only by seeing the details inside that region.
    • Grounding to the full image: The region's bounding box is overlaid on the full image I to form the student input x. A spatial instruction, such as 'look at the red box', is added to the question so the student knows where to focus.
    • Crop generation: The region is cropped and resized, often by 2x, to form the teacher input x'.
  42. 197Computer VisionarXiv:2605.22819

    Cambrian-P: Pose-Grounded Video Understanding

    • Annotation: use the off-the-shelf pose estimator ViPE to generate external and internal camera parameters for the remaining clips.
    • Model size: as the LLM size increases from 0.5B to 7B parameters, both VQA performance and pose accuracy improve consistently.
    • Data size: increasing the amount of training data widens the performance gap between the pose-grounding model and the baseline.
  43. 196Computer VisionarXiv:2605.18678

    Lance: Unified Multimodal Modeling by Multi-Task Synergy

    • For understanding, generation helps: training on generative tasks such as video editing makes the model develop a deeper understanding of temporal dynamics and spatial relationships, which improves performance on video understanding benchmarks.
    • For video generation, Lance scores 85.11 on VBench, the highest among unified models at the time of publication, with strong performance on object grounding and temporal consistency.
    • For multimodal understanding, Lance scores 62.0 on MVBench, which evaluates temporal cognition and video reasoning. This is an 11.3% improvement over the next-best unified model, Show-o2 (7B).