HDATF
English
Start a conversation
Menu

Paper Archive

Every archived paper with its summary, highest number first.

Papers 1,193Archive updated BenchmarksRSS feed
  1. 294Computer VisionarXiv:2606.02565

    Policy-based Foveated Imaging and Perception

    • Existing acquisition strategies, such as spatial or temporal downsampling, solve this problem by irreversibly discarding information before task relevance can even be assessed.
    • The paper introduces a real-time, predictive, task-aware foveated imaging system that operates directly at image acquisition time.
    • Extensive simulation across multiple perception tasks shows that this approach achieves high task performance under strict pixel budgets and substantially outperforms related baselines operating at the same bandwidth.
    AbstractPDF
  2. 293AI / GeneralarXiv:2606.02568

    ClinEnv: An Interactive Multi-Stage Long Horizon EHR Environment for Agents

    • Static benchmarks cannot validate this, and existing interactive medical benchmarks each compromise on at least one dimension.
    • This paper presents ClinEnv, an interactive benchmark that evaluates LLMs as attending physicians on real inpatient cases under a paradigm called Longitudinal Inpatient Simulation.
    • Each case is automatically organized into a sequence of ordered decision steps, and at every step the model must actively query four specialized agents before finalizing medications, procedures, or diagnoses.
    AbstractPDF
  3. 292Computer VisionarXiv:2606.02569

    AdaCodec: A Predictive Visual Code for Video MLLMs

    • However, existing video multimodal large language models, or video MLLMs, usually encode each sampled frame independently as an RGB image, which causes visual tokens to repeat content that already appears in earlier frames.
    • Across 11 benchmarks, AdaCodec outperforms the frame-wise RGB baseline of Qwen3-VL-8B under the same visual-token budget.
    • Even with one-seventh of the budget, AdaCodec with 32K tokens beats the 224K baseline on every long-form video benchmark, and on five general video benchmarks it raises the average score while reducing time to first token from 9.26 seconds to 1.62 seconds.
    AbstractPDF
  4. 291Computer VisionarXiv:2606.02572

    VISReg: Variance-Invariance-Sketching Regularization for JEPA training

    • Sketching-based methods such as SIGReg address this by aligning embeddings to an isotropic Gaussian, but they lack flexibility and suffer from vanishing gradients under collapse.
    • VISReg scales linearly, outperforms existing regularization methods on low-quality datasets, and shows robustness in long-tail and low-rank settings.
    • Pretrained on ImageNet-1K, VISReg achieves SOTA performance on out-of-distribution datasets.
    AbstractPDF
  5. 290Computer VisionarXiv:2606.02573

    HumanNOVA: Photorealistic, Universal and Rapid 3D Human Avatar Modeling from a Single Image

    • Achieving both photorealism and generalization is difficult because diverse, high-quality 3D human data are scarce.
    • These two strategies allow scaling to as many as 100,000 assets, greatly increasing both the quantity and diversity of data needed to train robust models.
    • Extensive experiments on multiple benchmarks quantitatively and qualitatively demonstrate the superiority of this approach and its robustness under diverse input-image conditions.
    AbstractPDF
  6. 289Computer VisionarXiv:2606.02575

    From Zero to Hero: Training-Free Custom Concept Spawning in World Models

    • The paper introduces a training-free method for concept spawning called SPAWN, which stands for Swapping Pinned Anchor with Windowed iNjection.
    • SPAWN exploits a structural property of image-to-video backbones: the first slot in context memory is pinned to the reference frame and serves as the anchor for every generated chunk.
    • Experiments show that SPAWN integrates concepts with consistent lighting, scale, and perspective while preserving identity and temporal consistency, demonstrating that controllable concept spawning is possible in existing autoregressive world models without any training.
    AbstractPDF
  7. 288Computer VisionarXiv:2606.02576

    ProtoAda: Prototype-Guided Adaptive Adapter Expansion and Geometric Consolidation for Multimodal Continual Instruction Tuning

    • Multimodal large language models achieve strong performance through instruction tuning, but real-world deployment requires them to keep acquiring new vision-language abilities, so multimodal continual instruction tuning is essential.
    • To address this problem, the paper proposes ProtoAda, a prototype-guided adaptive tuning framework.
    • Extensive experiments across multiple benchmarks show that ProtoAda achieves superior performance, especially on tasks where answer structure is easily damaged by sequential tuning.
    AbstractPDF
  8. 287RoboticsarXiv:2606.02577

    RoboDream: Compositional World Models for Scalable Robot Data Synthesis

    • This paper proposes a generalizable body-centric world model that achieves scalable data generation by synthesizing realistic demonstrations in new objects, new scenes, and new viewpoints.
    • The approach anchors generation to rendered robot motions while conditioning on explicit scene and object priors, effectively separating trajectory execution from environment synthesis.
    • Real-world experiments show that the generated data consistently improves downstream policy performance across diverse manipulation tasks and substantially reduces the need for real-world data.
    AbstractPDF
  9. 286Computer VisionarXiv:2606.02578

    Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling

    • To address this issue, the paper introduces the Perceptually Perturbed Judgment Dataset, which constructs minimally edited counterfactual responses that separate perceptual errors and enable verifiable supervision.
    • Experiments across multiple MLLM-as-a-Judge benchmarks show that this approach substantially improves perceptual fidelity, ranking consistency, and alignment with human judgments.
    • The result establishes a scalable and generalizable path toward training multimodal judges that are perceptually grounded, interpretable, and robust to vision reasoning conflicts.
    AbstractPDF
  10. 285Computer VisionarXiv:2606.02580

    Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Models

    • The paper introduces Staged Executable Inverse Graphics, or SEIG, an agent framework for reconstructing 3D scenes from a single image by progressively refining scene factors, including geometry, materials, composition, and lighting, directly in executable Blender code space.
    • The experiments show that staged reconstruction substantially improves reconstruction fidelity and highlights the importance of task decomposition in executable inverse graphics with a general-purpose VLM.
    • It finally demonstrates a variety of downstream applications enabled by the reconstructed and editable Blender scenes.
    AbstractPDF
  11. 284Computer VisionarXiv:2605.31598

    Linear Scaling Video VLMs for Long Video Understanding

    • Existing efficiency methods improve scalability, but they often lose accuracy relative to full self-attention, for example through aggressive frame or token dropping or coarse attention approximations.
    • These results suggest practical progress toward scalable long-video understanding.
    • Video vision-language models, or VLMs, are increasingly used in long-horizon and streaming settings, but most video encoders still rely on spatiotemporal self-attention, so compute and latency grow quadratically with the number of frames.
    AbstractPDF
  12. 283Machine LearningarXiv:2605.29548

    Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention

    • The result points to data-induced competition for resources, or neurons.
    • The paper develops a simple phenomenological argument that power-law scaling already suggests larger models can learn parts of the data distribution that smaller models cannot, even with infinite training data.
    • To test this claim and identify its cause, the authors study model scaling in a synthetic setting made of a mixture of tasks with monotonic scaling curves.
    AbstractPDF
  13. 282Machine LearningarXiv:2605.30865

    GlucoFM: A Dual-Stream Foundation Model for Continuous Glucose Monitoring

    • Across four diverse cohorts and seven clinical prediction tasks, GlucoFM achieves the strongest subject-separation linear probing performance among evaluated baselines and improves average PR-AUC by 4.1 points over the best CGM-specific foundation model.
    • The gains are most pronounced on core metabolic outcomes, where it leads PR-AUC on all diabetes risk and beta-cell dysfunction tasks and on 3 of 4 insulin resistance tasks.
    AbstractPDF
  14. 281AI / GeneralarXiv:2605.31264

    COLLEAGUE.SKILL: Automated AI Skill Generation via Expert Knowledge Distillation

    • LLM agents are increasingly expected not only to complete isolated tasks but also to capture human expertise, judgment, and interaction style in a constrained form.
    • This paper proposes an automated trace-to-skill distillation system that generates human-based AI skills through expert knowledge distillation.
    • The authors describe the output contract, generation workflow, revision lifecycle, deployment surface, and domain presets implemented in the open-source system.
    AbstractPDF
  15. 280Machine LearningarXiv:2605.29713

    The Little Book of Generative AI Foundations: An Intuitive Mathematical Primer

    • This book provides a concise, derivation-driven introduction to the mathematical foundations of modern generative artificial intelligence.
    • Its goal is to make the structure of generative modeling more accessible without stripping away the mathematical content needed to understand how these models are derived and how they relate to one another.
    • The book is intended as a foundational primer for researchers, practitioners, and students who are curious about mathematics.
    AbstractPDF
  16. 279LLMs / NLParXiv:2605.31268

    Mellum2 Technical Report

    • This report introduces Mellum 2, an open-weight 12B-parameter MoE language model with 2.5B active parameters per token.
    • Across code generation, math and reasoning, tool use, knowledge, and safety benchmarks, Mellum 2 is competitive with open-weight baselines in the 4B to 14B range while operating at the per-token compute cost of a 2.5B dense model.
    • The authors release the base, instruct, and thinking checkpoints under the Apache 2.0 license together with this report, which describes the underlying architectural choices, data pipelines, and training recipe.
    AbstractPDF
  17. 278LLMs / NLParXiv:2605.31584

    LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards

    • Experiments on five long-context benchmarks with three reasoning LLMs, ranging from 4B to 30B, show that LongTraceRL consistently outperforms strong baselines and encourages comprehensive, evidence-based reasoning.
    • Reinforcement learning with verifiable rewards, or RLVR, has shown promise for this task, but prior methods are limited to low-confusion distractors and sparse, outcome-only reward signals, so they fail to supervise intermediate reasoning steps.
    • This paper proposes LongTraceRL to address these issues.
    AbstractPDF
  18. 277Computer VisionarXiv:2605.30561

    VLM3: Vision Language Models Are Native 3D Learners

    • The authors therefore propose VLM3, a simple and scalable extension method that enables standard VLMs to handle a wide range of 3D tasks effectively.
    • A deep large-scale study shows that unifying focal length, using text-based pixel references, and mixing and scaling data are sufficient for effective 3D learning.
    • Vision-language models let a single model solve diverse vision tasks through prompting.
    AbstractPDF
  19. 276Computer VisionarXiv:2605.31604

    Representation Forcing for Bottleneck-Free Unified Multimodal Models

    • The paper proposes Representation Forcing (RF), a technique that closes this gap by making representation prediction an intrinsic capability of the model.
    • If this is simply removed, the model must learn both high-level structure and low-level details directly from raw pixels, which creates a quality gap.
    • Specifically, RF forces the decoder to autoregressively predict visual representations as intermediate tokens before pixels, and those tokens are then kept in context to guide pixel diffusion within the same backbone.
    AbstractPDF
  22. 273LLMs / NLParXiv:2605.29682

    Scaling Laws for Agent Harnesses via Effective Feedback Compute

    • Step complexity, L, is the minimum number of reasoning or action steps required.
    • A higher-quality router greatly increases eta by ensuring that the model calls the right tool at the right time.
    • Better state management ensures that useful information is preserved and used, which prevents redundant interactions.
    AbstractPDF
  23. 272Computer VisionarXiv:2605.30248

    GenClaw: Code-Driven Agentic Image Generation

    • In the final rendering stage, the image model uses this physical draft as a visual guide to generate a realistic scene.
    • If the search result is wrong, it is a cognition error.
    • If the layout is illogical, it is a code error.
    AbstractPDF
  24. 271Computer VisionarXiv:2605.30215

    Déjà View: Looping Transformers for Multi-View 3D Reconstruction

    • Register tokens are four tokens used as a global scratchpad for the transformer.
    • Camera tokens are special tokens that store information about the pose and intrinsics of the current view.
    • Frame attention is a subblock that operates independently on each image, allowing the model to infer local geometry within a single view.
    AbstractPDF
  25. 270Computer VisionarXiv:2605.30263

    minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models

    • Training stage: Camera controllability does not appear immediately. With the HY1.5 backbone, the model showed no responsiveness at 2,000 steps, began showing signs of control at 5,000 steps, and stabilized only after 8,000 steps.
    • Embodied AI: It gives robots a mental simulator that predicts the visual consequences of their actions.
    • Game development: It generates infinite, interactive environments that respond instantly to player movement.
    AbstractPDF
  26. 269Machine LearningarXiv:2605.27734

    Learn from your own latents and not from tokens: A sample-complexity theory

    • Predictor: A network that takes a tuple of latent values as input and tries to predict the distribution of its sibling latent values.
    • Clusterer: A network that compresses the output of the predictor into a discrete code or cluster identity, and that code becomes the input to the next layer.
    • Teacher as a lifter: The EMA teacher gradually incorporates the learned latent values into its own targets, which shifts the prediction task from token level to latent level during training.
    AbstractPDF
  27. 268AI / GeneralarXiv:2605.27276

    SIA: Self Improving AI with Harness & Weight Updates

    • In SIA-H, which uses only the harness, the agent built a classification pipeline and reached 50.0% accuracy, but performance plateaued once the prompt and code hit their limits.
    • In SIA-W+H, which adds weight updates, the model learned to internalize legal nuance, and accuracy jumped to 70.1%, far above the previous best of 45.0%.
    • In SIA-W+H, the system achieved a 14.02x speedup over the initial baseline. Here, weight updates were decisive because the model learned specific H100 hardware patterns such as shared-memory tiling and register accumulation, which scaffold iteration alone could not discover.
    AbstractPDF
  28. 267Machine LearningarXiv:2605.29157

    Parallax: Parameterized Local Linear Attention for Language Modeling

    • Perplexity: On datasets such as LAMBADA and WikiText, Parallax achieved consistently lower perplexity than Transformer baselines, a measure of how well the model predicts samples.
    • Downstream tasks: On zero-shot benchmarks such as HellaSwag and ARC, which cover question answering and commonsense reasoning, Parallax showed clear performance gains.
    • Control experiments: To confirm that the gains were not simply due to more parameters or more compute, the authors built parameter-matched and compute-matched versions of the standard Transformer. Parallax outperformed both, suggesting that the improvement comes from the parameterized local linear mechanism itself.
    AbstractPDF
  29. 266Computer VisionarXiv:2605.27310

    How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

    • The key failure mode of visual thinking is identified as "generated but not used."
    • View Dropout, or VDrop, is a training method that masks part of the input view during the answer segment to force the model to pass through a thinking image.
    • The Learnability-Informativeness, or L-I, tradeoff framework compares three visual-thinking representations: panorama, top-down, and point matching.
    • The model is trained on 8K synthetic Infinigen Indoors samples and evaluated on five real OOD benchmarks.
    • Only panorama visual thinking combined with VDrop scores highly on both axes and achieves the best OOD performance.
    • It achieves a 6.7-point OOD improvement over prior methods that were trained with three times more data.
    AbstractPDF
  30. 265LLMs / NLParXiv:2605.28773

    Rethinking Memory as Continuously Evolving Connectivity

    • Success: Through these topological and unit-level edits, the agent successfully completed the task.
    • Self-correcting retrieval: Environmental feedback dynamically fixes over-connection and under-connection problems.
    • Granularity adjustment: The level of detail in stored memory is adapted to the current task requirements.
    AbstractPDF
  31. 264Machine LearningarXiv:2605.28109

    Long Live The Balance: Information Bottleneck Driven Tree-based Policy Optimization

    • It improves efficiency by reusing prefixes and taking advantage of KV caching, which lets it generate more trajectories, or a higher G, within the same token budget than independent sampling.
    • By branching according to the IB-Score, the model focuses exploration on the 'critical' steps where both high diversity and a strong signal toward the correct answer are present.
    AbstractPDF
  32. 263LLMs / NLParXiv:2605.28814

    Self-Improving Language Models with Bidirectional Evolutionary Search

    • The merge operator takes two different reasoning paths that share the same starting point and merges their unique suffixes, allowing the model to synthesize successful components from multiple attempts.
    • Deletion removes a single internal step from a trajectory, which helps simplify reasoning or eliminate redundant filler steps that can send the model in the wrong direction.
    • Transposition replaces one step in a trajectory with a step from another trajectory, similar to swapping out a piece of logic to see whether a different approach fits the same context better.
    AbstractPDF
  33. 262RoboticsarXiv:2605.30280

    Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

    • The auxiliary vision-language data, 8.5 percent of the training mix, is general-purpose VLM data that helps the model avoid forgetting how to describe scenes or reason about objects.
    • The unified architecture shows that a single model can control multiple embodiments, including arms, hands, and mobile bases, through a shared action expert and text conditioning.
    • The paper shows that vision-language training directly helps robot behavior by providing better object recognition and command parsing.
    AbstractPDF
  34. 261AI / GeneralarXiv:2605.26340

    ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence

    • Score verification, I1: the auditor reruns the agent's code on a gold evaluator to confirm that the reported score is reproducible.
    • Spec violation, I2: the system checks whether the agent bypassed the rules, for example by hard-coding answers or importing restricted data.
    • Reference verification, I3: every citation is cross-checked against global scholarly databases to detect hallucinations.
    AbstractPDF
  35. 260AI / GeneralarXiv:2605.28655

    AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation

    • The shared forum (F) is a space where agents post structured discussions, hypothesize why specific changes worked, and critique the proposals of other agents.
    • Removing the analyst led to a substantial drop in proposal quality because the experimental agents lacked strategic oversight to avoid redundant or weakly justified attempts.
    • Removing inter-agent feedback increased the rate of false starts, where the system pursued hypotheses that other agents could easily identify as flawed.
    AbstractPDF
  36. 259Computer VisionarXiv:2605.27367

    SpatialBench: Is Your Spatial Foundation Model an All-Round Player?

    • DENSE focuses on long-time-series video sequences and feeds in nearly every frame up to the 500-frame limit.
    • For robotics, use models trained on DA-Next-5M or similar first-person datasets; standard indoor and outdoor foundation models are likely to fail on wrist-mounted manipulation tasks.
    • For AR and VR, streaming or chunk-based models are preferred because they must handle long sessions without exceeding the memory limits of mobile devices or headsets.
    AbstractPDF
  37. 258ResearcharXiv:2605.28781

    The sum-product conjecture is false for real numbers

    • A counterexample to the sum-product conjecture over the reals (1.1), constructing sets A with both |A+A| and |AA| at most |A|^{2-c}.
    • A higher-dimensional Balog-Wooley-type structure using algebraic integers from totally real number fields.
    • A counterexample to the many sums and products conjecture as well: max(|kA|, |A^(k)|) ≤ |A|^{C log k / log log k}.
    • Variants over various rings, including p-adics, the finite field Fp, and the positive-characteristic function field Fq((t)).
    • New lower bounds for the number of solutions to the linear equation x1+...+xk=1 in multiplicative groups, along with applications to unit equations.
    • GPT-5.5 Pro was used for initial exploration, but the final proof was generated mostly by humans, inspired by the OpenAI counterexample to the unit-distance conjecture.
    AbstractPDF
  38. 257Computer VisionarXiv:2605.26519

    R³: 3D Reconstruction via Relative Regression

    • Scale stability: The relative motion between two adjacent frames, the baseline, is usually small and predictable, whereas the distance from the starting point of a long trajectory is not.
    • Origin independence: Any frame in the sequence can serve as the reference for another frame. The system is no longer chained to the first observation.
    • Novelty: If the visual features of a new frame differ enough from those already stored, that frame is added to the bank.
    AbstractPDF
  39. 256Machine LearningarXiv:2605.26535

    Recursive Flow Matching

    • Tradeoff removal: while most models force a choice between speed and accuracy, RecFM uses recursive training to improve both at once.
    • Explicit scale alignment: unlike consistency models that distill a student from a teacher, RecFM uses a single network and enforces multi-scale alignment internally, which provides a more robust gradient signal during training.
    • Physical fidelity: by straightening trajectories, RecFM preserves high-frequency details, such as waves and vortices, that are usually lost when generative models are accelerated.
    AbstractPDF
  40. 255Machine LearningarXiv:2605.26282

    Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization

    • Identify the structural mismatch between exploration and value learning as the key bottleneck in world-model RL.
    • Recast policy optimization as a diffusion process on the latent world model to unify exploration and policy optimization.
    • Anchor the policy to the behavior distribution with a dataset-induced implicit energy function, namely a KL trust region.
    • Correct the score field with cumulative reward from imagined trajectories so it converges toward the optimal Gibbs policy.
    • Show consistent gains across multi-task offline pretraining, online, and offline-to-online settings.
    • Demonstrate scalability through monotonic performance improvements as model capacity increases.
    AbstractPDF
  41. 254Computer VisionarXiv:2605.28816

    Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players

    • The hub token attends to all agent tokens and to other hub tokens.
    • Bidirectional teacher training: the model is first trained as a standard diffusion model with access to the full temporal sequence. This teacher learns the full distribution of multi-agent interaction with high fidelity.
    • Causal student training: to enable real-time rollouts, the student model is trained with a causal structure. Unlike the teacher, the student sees only the past and the present, and it uses a Sparse Hub Attention mask plus noise levels sampled independently for each frame block.
    AbstractPDF
  43. 252AI / GeneralarXiv:2605.27366

    MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation

    • Testing is the unit test used to verify that a skill works correctly before it is allowed into the Skill Bank.
    • Memory is a dedicated file, .memory.md, that stores observations and lessons from previous runs.
    • At Level 1, single-node compression, when a particular tool output is extremely large, for example a 20,000-token log file, the method replaces it with a concise summary while leaving the surrounding conversation intact.
    AbstractPDF
  44. 251Information RetrievalarXiv:2605.24938

    Your Embedding Model is SMARTer Than You Think

    • In efficiency terms, developers can upgrade existing retrieval systems to support multiple vectors without reindexing the entire database or performing expensive retraining.
    • In accuracy terms, SMART points toward a universal retrieval model that works well both for global tasks such as image classification and for local tasks such as finding a specific sentence in a 100-page manual.
    AbstractPDF
  45. 250LLMs / NLParXiv:2605.25604

    DVAO: Dynamic Variance-adaptive Advantage Optimization for Multi-reward Reinforcement Learning

    • In terms of usability, the algorithm adaptively finds the optimal balance from data, reducing the need for practitioners to manually tune reward weights.
    • In terms of stability, it preserves the efficiency of the GRPO framework while adding the theoretical stability needed to avoid size explosion.
    • In terms of synergy, cross-objective regularization ensures that the model does not optimize only the easiest reward while sacrificing harder and more important goals such as reasoning accuracy.
    AbstractPDF
  46. 249Computer VisionarXiv:2605.26102

    InstructSAM: Segment Any Instance with Any Instructions

    • During decoding, the final score head determines whether the query corresponds to a valid target instance, and the segmentation head produces a binary mask.
    • For a single target, it identifies the one specific object mentioned in the instruction.
    • For multiple targets, it finds all instances that satisfy a complex condition, such as "all wooden chairs."
    AbstractPDF
  48. 247Computer VisionarXiv:2605.25979

    LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence

    • In stage 3, which focuses on long videos, the model is exposed to a length-hierarchical video caption corpus of about 8 million clips and 104 billion tokens in total, with videos ranging from 30 seconds to 15 minutes, so that it learns to maintain context over long durations.
    • In stage 4, which focuses on codec and spatial reasoning, codec-native training is introduced and 10 to 15 minute videos are re-encoded with a variable-length GOP pipeline, while the model is also trained on the LLaVA-OneVision-2-Spatial-4M corpus containing 4 million samples focused on 2D and 3D spatial reasoning such as identifying objects in indoor scans or tracking motion in embodied simulators.
    • The codec-stream approach performs substantially better on temporal grounding and visual perception because it reallocates tokens to event-bearing moments and captures motion dynamics that uniform sampling misses.
    AbstractPDF
  49. 246LLMs / NLParXiv:2605.26431

    Probing Minimalist Phase Structure in LLMs: What Universal Dependencies Cannot Represent

    • Use wh-movement stimuli designed so that UD tree distance remains invariant across conditions to detect structure beyond UD.
    • In 12 of 13 LLMs across four families, the authors find a distance slope proportional to Minimalist phase count, with βfin > βinf > 0.
    • Discover 13 out of 13 sign asymmetries that exactly match within-phase cohesion predictions, a result impossible to capture with UD-based probing.
    • Introduce canonical-layer reporting, which removes degrees of freedom from per-contrast peak reporting and applies a stricter standard.
    • Confirm the causal relevance of probe results with activation patching in 12 of 13 models.
    • The results suggest that distributional pretraining can induce formal syntactic representations that go beyond the limits of annotation-based probing.
    AbstractPDF
  50. 245Machine LearningarXiv:2605.26106

    Looped Diffusion Language Models

    • The Head is made of a fixed number of layers that run once at the start of the forward pass, and its main role is to process the initial input embedding and provide a starting representation for the rest of the network.
    • The Tail is made of a fixed number of layers that run once after the Mid-block finishes looping, and these layers finalize the representation and project it into vocabulary space for token prediction.
    AbstractPDF