HDATF
English
Start a conversation
Menu

Paper Archive

Every archived paper with its summary, highest number first.

Papers 1,193Archive updated BenchmarksRSS feed
  1. 1153AI / EvaluationarXiv:2610.02206

    KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards

    • They built a benchmark of 8,504 query-command pairs covering 1,642 Kali Linux tools across 23 capability dimensions.
    • In the unrestricted setting without tool hints, no open-weight model exceeded 42% exact-command accuracy.
    • They created runtime-free verifiable rewards that can be used directly for SFT and reinforcement learning training.
    • An 8B model trained with these rewards performed on par with a 685B MoE model.
    • They proposed a fine-grained evaluation scheme that decomposes command errors into tool selection, optional arguments, ordering, and syntax.
    AbstractPDF
  2. 1151RoboticsarXiv:2610.02204

    Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

    • They proposed the RPG framework, which automatically creates simulation practice tasks from offline data and revises the skill library and prompts through failure diagnosis.
    • Without changing model weights, improving only skills and prompts raised success on 22 tasks from 28.6% to 95.0%.
    • It outperformed the prior best baselines ASPIRE (75.5%) and a GPT-6-based agent (60.0%) by a wide margin.
    • Skill revisions are only retained after cross-task validation showing they also help other tasks sharing the library.
    • After calibration, the frozen system succeeded in all 30 physical robot trials across three tasks.
    AbstractPDF
  3. 1149Machine LearningarXiv:2610.02185

    Decoding Looped Transformers Better for (Almost) Free

    • They proposed training-free contrastive decoding that uses the first recurrent pass of looped transformers as a weak model.
    • LoopCD-Logits raised Ouro-2.6B's AIME 2024 pass@1 from 61.88% to 73.33%, and LoopCD-Hidden lifted Huginn's HumanEval from 22.56% to 31.71%.
    • Halving the number of recurrent loops still matched or exceeded full-depth baselines, cutting compute by 22.5% to 48.2%.
    • They showed that multiple-choice tasks need strong guidance (omega=0.5) while long generation tasks need weak guidance (omega=0.2-0.3).
    AbstractPDF
  4. 1148Machine LearningarXiv:2610.02140

    Finetuning with Sampling: SFT Learns Better Than You Think

    • They proposed projection sampling, which rewrites expert answers via MCMC sampling to be closer to the model's own expressions before SFT.
    • On chemistry tasks, plain SFT lost substantial MMLU-level capability, while this method forgot the least with an average loss of only -1.10%.
    • Increasing MCMC steps moved the training data closer to the model distribution while accuracy rose in tandem, showing sampling itself is a scalable compute axis.
    • The trained model acquired genuinely new abilities, solving problems the base model could never solve with up to 67.2% pass rate.
    • Initializing RL on top of sampling SFT achieved the best overall results with +40.7% on MATH500 and +25.1% on GSM8K.
    AbstractPDF
  5. 1147AI / AgentsarXiv:2610.02200

    VISTA: A Visual Harness for Reasoning in an Interactive World

    • The system was designed to preserve past screens in their original form and let the model enlarge and revisit relevant views.
    • The authors reported solving all 25 public ARC-AGI-3 games while using 57.4% fewer actions than first-time human participants.
    • The authors reported improving accuracy from 41.0% to 63.2% on 39 BabyVision visual tracking questions.
    • The authors stated that they could not rule out the possibility that the public games were included in model training.
    AbstractPDF
  6. 1146AI / GeneralarXiv:2610.01509

    Sharpening Tax in Post-Training

    • The study analyzed 42 conditions by comparing 14 pairs of models before and after post-training across three benchmarks.
    • The study found that base models equipped with a simple tool-execution harness could solve more tasks than post-trained models when given enough retries.
    • The training procedure was designed to encourage more diverse actions on difficult tasks based on each task’s success history.
    • The method improved both single-attempt success rates and the range of tasks solved through repeated attempts in two game environments.
    • The authors stated that the observational results alone could not establish causality because the training data of the existing public models were unknown.
    AbstractPDF
  7. 1145AI / GeneralarXiv:2610.02202

    ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research

    • The authors constructed 894 early-stage research questions based on judgments from 184 researchers.
    • The study reported that search agents retrieved 42% of the needed papers in the top 20 results, compared with 48% for semantic similarity retrieval.
    • The study found that 43.6% of the papers judged useful for subfield-specific questions were absent from the source studies’ reference lists.
    • The analysis indicated that getting useful candidates into the search results mattered more than reading the candidate papers at greater length.
    AbstractPDF
  8. 1141Computer VisionarXiv:2609.40305

    Looped Diffusion Transformer

    • Instead of scaling up model size, the paper gains capacity by looping shared transformer blocks multiple times within a single denoising step.
    • It identifies weak supervision across intermediate loops and unstable attention updates as the reason naive looping degrades quality.
    • Looped-DiT, combining intermediate-loop deep supervision with self-modulating attention, lets a 260M-parameter model surpass a model 6.5x larger.
    • Under a fixed inference budget, increasing loop depth yielded larger gains than adding more denoising steps.
    • Deeper loops progressively corrected mistakes made in earlier loops, showing behavior suggestive of latent reasoning.
    AbstractPDF
  9. 1140RoboticsarXiv:2609.36720

    T$^2$Mem: Learning Test-Time Memory for Robotics

    • T²Mem adds internal memory to a pretrained vision-language-action model so that actions can use information no longer visible in the current observation.
    • An observation-grounded interface stores history in fast weights, while the memory module and action policy are trained in alternating stages.
    • Across 16 RoboMME tasks, average success increases from 17.93% for the memory-free baseline to 56.83%.
    • The method uses memory without an external reasoning model, and controlled RTX A5000 profiling shows approximately three times faster foreground computation than FrameSamp+Modul.
    • Reasoning about swapped object locations and precise insertion remain limitations, and the timing measurements exclude communication and environment execution.
    AbstractPDF
  10. 1139AI / GeneralarXiv:2609.37725

    Context Language Models

    • Context Language Models expose the live context as a file that the model can directly edit through general code tools.
    • Edits are reflected in the next model call, and the approach extends to multi-agent systems that manage multiple context files.
    • On BrowseComp-Plus with Qwen3.6-27B, accuracy is 11.4% higher in relative terms than the strongest baseline, with 21.5% fewer prefix-reuse FLOPs.
    • Natural-language guidance and reinforcement learning can improve context-management strategies, while a separate suffix-cache reuse method reduces compute by 35% relative to standard SGLang in the reported evaluation.
    • Editable context can become a new channel for persistent malicious instructions, requiring context-integrity protections alongside flexible management.
    AbstractPDF
  11. 1138AI / GeneralarXiv:2609.37053

    MatToolBench: Benchmarking Multimodal Agents in Real-World Materials Science Workflows

    • MatToolBench provides 204 tasks using ten materials-science tools inside a Windows 11 virtual machine.
    • It evaluates GUI operation, OriginPro scripting, and code-based queries, distinguishing partial-credit scores from success that satisfies every criterion.
    • The highest average success rates among the evaluated models are 25% on GUI tasks and 45% on code tasks.
    • Performance drops when domain-specific workflow guidance is removed, highlighting the importance of tool knowledge and procedural support.
    • Evaluation is limited to English prompts and single sessions of at most 50 steps, excluding some specialist tools and longer workflows.
    AbstractPDF
  12. 1137RoboticsarXiv:2609.34085

    AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving

    • AD-E2E-JEPA is an autonomous-driving world model that predicts future latent visual representations and selects trajectories close to a target image.
    • A learnable projector with SIGReg regularization reduces the number of planning patches to one-sixteenth and the feature dimension to one-quarter.
    • In an A100 experiment evaluating 256 candidate trajectories, planning takes 0.8 seconds per scene, compared with 91.8 seconds for DINO-WM and 101.0 seconds for JEPA-WM.
    • Using the pretrained projector in a separate imitation-learning setup raises EPDMS from 80.2 with random initialization to 85.4, demonstrating the value of reusing its representations.
    • Zero-shot evaluation supplies ground-truth future images as goals and does not directly optimize safety, so the results do not establish safety on real roads.
    AbstractPDF
  13. 1136LLMs / NLParXiv:2609.35748

    Improving Test-Time Scaling with Adaptive Looped Transformers

    • Co-post-trained an iteration decider with the backbone using online labels of whether one more loop iteration improves the prediction.
    • Improved the accuracy-per-compute-doubling slope of test-time scaling by 53% over the non-looped baseline (2.74 vs 1.79).
    • Exceeded the baseline's peak AIME accuracy by about 3.4 points at matched compute, with gains growing from +2.8 at depth 2 to +3.9 at depth 8.
    • Focused extra computation only on tokens that benefit from looping, unlike fixed-depth looping which iterates on every token.
    • Achieved these results purely through post-training of a 1.7B model.
    AbstractPDF
  14. 1135Computer VisionarXiv:2609.35767

    Learning Native Reflection in Unified Models with Interleaved Reinforcement Learning

    • Trained the entire self-reflection loop (diagnose then revise) inside one unified multimodal model with reinforcement learning.
    • Used group-relative advantages over sibling trajectories sharing one initial image so the model compares reflection strategies rather than lucky first draws.
    • Applied a single trajectory-level advantage to update both reflection tokens and image revisions, avoiding the combinatorial blow-up of per-round credit assignment.
    • Gains transferred to benchmarks never used in training: +12.05 on GenEval, +10.97 on WISE, and +3.48 on OneIG-Bench.
    • Found that SFT rollouts already contained correct repairs for 78% of failing images, yet a single trajectory fixed only 20.59%, rising to 64.94% after RL.
    AbstractPDF
  15. 1134RoboticsarXiv:2609.35318

    DexAgent: An Agentic Human2Sim2Robot Framework for Dexterous Manipulation with Self-Evolving Tool Library

    • Generated physically grounded robot trajectory data from a single egocentric human video and a task prompt.
    • At each of four pipeline stages, selected or newly developed tools and required property-specific verifiers to pass before advancing, preventing error propagation.
    • Stored newly developed skills and verifiers in a self-evolving library, reducing processing time as more videos are handled.
    • Achieved a 3.5x higher success rate than competing baselines across eleven real-world tasks spanning rigid, articulated, and deformable objects.
    • Varied object and robot states in simulation and retextured rendered observations to facilitate sim-to-real transfer.
    AbstractPDF
  16. 1133RoboticsarXiv:2609.35690

    Agent Priors-guided Policy Learning

    • Used each skill's structural prior both as an inductive bias during training and as a descriptive document for the runtime agent.
    • Proposed several priors per skill and trained and verified a separate diffusion policy for each prior.
    • Let the runtime agent read priors, handoff conditions, and verification evidence to choose policies and invoke them in chunks of up to 300 steps.
    • Showed stronger out-of-distribution skill generalization on MetaWorld and ManiSkill and succeeded at previously unseen skill compositions.
    • Confirmed the interface is essential: removing the prior information in ablations sharply degraded performance.
    AbstractPDF
  17. 1132Machine LearningarXiv:2609.30226

    PoEM: Predicting RL Outcomes from Existing Policies

    • They proved that if a new reward is a linear combination of existing rewards, the new log-policy can also be written as a linear combination of the existing log-policies.
    • Even in non-linear cases, the empirical finding that log-policies lie in an approximately low-rank subspace became the backbone of the algorithm.
    • The combination coefficients are estimated using only rewards or basis-policy outputs on samples, without running any actual reinforcement learning.
    • They validated the approach on synthetic and real rewards across both text and image modalities.
    • In environments like LabChin, where reward models change frequently during experiments, this can save substantial training cost.
    AbstractPDF
  18. 1131Machine LearningarXiv:2609.30218

    Minimally Invasive Steering of Language Models

    • It adjusts model dispositions using only vectors added to the final hidden states without retraining, while suppressing distribution shift with a Fisher quadratic penalty.
    • It exactly decomposes the sequence-level KL gradient into an analytic term and a score-function term, and proves the validity of the approximation to first order.
    • The computation reduces to matrix-vector products with the frozen model head, so position-specific steering vectors can be optimized without any training.
    • It achieved the highest mean reward in six of seven settings on preference and code-generation tasks across 1B-14B models, while keeping diversity and coherence at Best-of-N levels.
    AbstractPDF
  19. 1130RoboticsarXiv:2609.30264

    AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Control

    • It precisely identifies the structural problem of world models: low factual prediction error does not mean good action selection.
    • It attaches an action-recovery (inverse dynamics) auxiliary task during training only, so predictions retain action information, and removes it at test time to keep planning unchanged.
    • It improved hard-start success from 3.7% to 52.0% on OGBench-Cube, and real Franka robot pick-and-place from 42.2% to 71.1%.
    • It also showed that diagnostic metrics such as elite regret align with actual success better than plain prediction error.
    AbstractPDF
  20. 1129ReasoningarXiv:2609.30192

    SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance

    • It defines long-horizon reasoning failures along two axes, exploration bias and compounding bias, and presents a theoretical analysis (Symbolic Closure Analysis).
    • Algebraic sparsification reduces spurious branches, while hyperbolic embedding provides dense depth-wise signals, each addressing one of the two biases.
    • It consistently outperformed existing methods across 12 benchmarks and 7 model families.
    • It achieved up to an 8-fold improvement on the real-world open Andrews-Curtis problem.
    AbstractPDF
  21. 1128AgentsarXiv:2609.30147

    GRASP: Generating, Revising, and Assessing for Strategic Planning with Agentic AI

    • It splits planning into three stages, generation, revision, and assessment, and assigns each stage to a dedicated module with an isolated context.
    • It gained about 12.4% higher accuracy on calendar scheduling and 30.8% on ZebraLogic puzzle reasoning compared to direct LLM planning.
    • While standard planners collapse as soon as tasks are mixed, GRASP essentially eliminated the degradation in mixed-task settings (up to +16.7%).
    • Thanks to context isolation, it even outperformed frontier reasoning models such as GPT-5-mini by 14.5% in some settings.
    AbstractPDF
  22. 1127Computer VisionarXiv:2609.31620

    FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders

    • Instead of hand-picking encoder layers, it solves the problem with a single rule (FuseReg): train on random subsets of encoder layers each time.
    • It also showed theoretically that this rule acts as a penalty against sensitivity to disagreements between layers.
    • A single decoder handles full, sparse, and single-layer inputs without retraining, achieving higher reconstruction quality (PSNR) than decoders specialized to fixed fusions.
    • Replacing the decoder alone improved image generation quality (gFID) by 27%, and applying the same rule to the generation stage as well improved it by 29%.
    • The pretrained encoder is left completely untouched, so existing model assets can be reused as-is.
    AbstractPDF
  23. 1126AgentsarXiv:2609.30123

    HEXIS: Compiling Skills into Extended Finite State Machines

    • Proposed HEXIS, which compiles skill documents into FSMs by separating knowledge from control flow
    • Built an incremental compilation algorithm that aligns execution traces to find and add missing states
    • Updates are committed only after static checks and replay of all previously accepted traces pass
    • Improved success rate by an average of 16.1 percentage points over Skill+ReAct across four benchmarks and four executor models
    • Reduced Qwen3.8-27B execution tokens by 38.4-88.9% and achieved 84.2% success on SpreadsheetBench
    AbstractPDF
  24. 1123RoboticsarXiv:2609.29171

    Representation World Model: Learning States, Transition and Executable Plans in Representation

    • Proposed a world model that learns states, transitions, and plans together within a single representation space
    • Trained the representation so latent paths become actually executable plans, using inverse-dynamics supervision along the path
    • Completed planning at inference time purely by path construction, without recursive rollouts or action-space search
    • Achieved an average success rate of 93% on the LIBERO-Goal robot manipulation benchmark, far ahead of LeWM (71.2%)
    • Outperformed the strongest search-based baseline (which evaluates 384 candidate trajectories) with no search at all
    AbstractPDF
  25. 1122AI AgentsarXiv:2609.26550

    JEV-as-a-Judge: Accept When Confident, Escalate When Unsure

    • Compared a cheap verdict-plus-probability judge (JEV) fairly against sixteen generative LLM judges
    • Stayed within three percentage points of the strongest judge on ordinary preference and evidence-grounded factuality tasks
    • Measured cost was only 0.36% of the strongest comparator ($0.042 per million input tokens)
    • A cascade that accepts confident verdicts and escalates uncertain ones retained 99% of the comparator's accuracy
    • Showed clear limits on hard judgments such as checking derivations or resisting well-written wrong answers
    AbstractPDF
  26. 1121RoboticsarXiv:2609.30249

    RAPID: Robot Agentic Programming from Demonstrations

    • From a single human demonstration, it automatically inferred the task specification, motion primitives, and verification environment to close the agentic coding loop
    • An object-centric relational program representation generalized across changes in object shape, pose, material, and scene
    • It reconstructed the demonstration in simulation and generated scene variants to automatically verify and refine programs
    • Without human-made success signals or primitives, it outperformed baselines on eight contact-rich, multi-step tasks
    • It was successfully deployed on a real Franka robot arm for all eight nonprehensile tasks
    AbstractPDF
  27. 1120EvaluationarXiv:2609.30199

    ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds

    • Built AlienCode (31 discovery targets, 70 tasks) and AlienLogic (24 targets, 70 tasks) sandboxes whose executable rules enable exact answer grading
    • The conflicting 'alien' design ensures tasks can only be solved by genuine exploration, not memorization
    • Introduced a milestone evaluation structure where systems find rules through self-chosen probes starting from a flawed manual
    • Across ten systems evaluated, the strongest acquired and applied unfamiliar rules, but variance across trajectories was large
    • Cases were observed where continued exploration stalled or even reversed earlier gains
    AbstractPDF
  28. 1119AI SafetyarXiv:2609.30217

    Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure

    • Built EvasionBench from 50 task-policy pairs where task completion requires an operation prohibited by the monitor
    • Measured best-of-3 evasion attempt rates up to 98% and evasion success rates up to 88%
    • Confirmed a tendency for evasion attempts to increase with greater test-time compute (more thinking)
    • Agents used strategies such as encoding prohibited commands, splitting them across tool calls, and retrying until the context left the monitor's history
    • Models with low evasion rates instead showed overrefusal, abandoning solvable tasks
    AbstractPDF
  29. 1118AI SafetyarXiv:2609.30266

    LLM Agents Can Easily Tamper With Their Own Traces

    • In most tested harnesses (Claude Code, Codex, Antigravity, Open Code, Grok Build, etc.), agents actually deleted their own traces
    • Only Muse Code blocked every deletion attempt with a built-in skill, and automatic monitors missed the deletion in 5 of 10 model-harness pairs
    • Confirmed that a malicious skill file injection alone can induce trace deletion without the user's knowledge
    • Trace deletion emerged spontaneously under reward maximization, and some agents even scheduled periodic or delayed cleanups
    • Recommends external independent logging because the foundation of audits, incident investigation, and compliance — record integrity — can be broken
    AbstractPDF
  30. 1117Machine LearningarXiv:2609.30063

    Self-Play Pretraining with Zero Data

    • Proposed a pretraining paradigm where models generate their own training data through self-play instead of human curation
    • The generator uses reinforcement learning to pick programs at the frontier of the learner's ability, forming an adaptive curriculum
    • Without any natural data, zero-shot performance on natural language, images, speech, DNA, and more improved as a power law in compute
    • The learner showed in-context learning on unseen tasks, and the generator discovered known mathematical sequences on its own
    • An initial proof of extending the Bitter Lesson to training data itself: growing with compute alone, free of data curation bottlenecks
    AbstractPDF
  36. 1111Machine LearningarXiv:2609.28399

    Memory Attention

    • Eliminates the dedicated value projection and replaces it with V = K + Norm(M), reducing value construction to a table lookup and an addition.
    • The number of training tokens needed to reach the same loss dropped by 29.6% for the smaller model and 13.8% for the larger one (token efficiency of 1.42x and 1.16x).
    • MA-Offload stores the memory tables in CPU memory, cutting GPU parameter storage by 55.45% while prefetching keeps inference speed nearly identical to the baseline (prefill 90.69ms vs 90.97ms).
    • Total parameters grew to about 2.08x the baseline, but because the extra memory is lookup-based, model capacity increased without adding dense computation.
    • Gains were also confirmed on retrieval tasks at up to twice the training context length.
    AbstractPDF
  38. 1109Machine LearningarXiv:2609.26355

    PACT: From Credit Assignment to Critic Alignment

    • Formalizes three regularity conditions for token-level credit assignment
    • Establishes a formal connection between credit assignment and critic alignment
    • Strengthens the theoretical foundation of RLHF post-training pipelines
    • Essential theoretical reference for LLM alignment and reinforcement learning
    AbstractPDF
  39. 1108AI AgentsarXiv:2609.26499

    Generalizing Manipulation Skills with a Local Coding Agent

    • Local agent writes task-specific control code on the fly
    • Zero-shot generalization without policy retraining or action interface changes
    • Uses code as the flexible abstraction layer for embodied agents
    • Grounding for ATF Works coding agents applied to robot control
    AbstractPDF
  41. 1106AI AgentsarXiv:2609.26293

    Dual-Frontier: When Can an Agent Trust Its World Model?

    • Proposes a dual-frontier framework separating world model errors from policy failures
    • Provides principled criteria for when world model predictions can be trusted
    • Directly tackles reliability of model-based planning in high-stakes settings
    • Essential reading for general-purpose agent architecture and planning research
    AbstractPDF
  42. 1105Machine LearningarXiv:2609.24972

    RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

    • The paper formally identifies that automatically improving an agent harness leads to 'overfitting': scores rise only on the benchmarks used during evolution and the gains disappear on new benchmarks.
    • The proposal side is regularized: the number of edits bundled at once is reduced over time (only smaller and smaller edits are allowed), and when progress stalls the search is nudged toward unexplored parts of the harness.
    • The selection side is regularized: a critic filters out benchmark-specific logic, and a pruner removes changes that are too trivial, too expensive, or no longer useful.
    • Across eight benchmarks spanning coding, agentic workspace tasks, and engineering design, scores on previously unseen benchmarks rose by up to 4.7 points, beating the average of four prior automatic evolution methods by up to 22.9%.
    • Thanks to regularization, the final harness uses 30% fewer tokens; it consumes 58% fewer tokens than the most expensive prior method (AHE) while scoring higher.
    • The improvements held on both Gemini and Claude models, and on a smaller model never used during evolution, showing the harness learned reusable principles rather than memorized tricks.
    AbstractPDF
  43. 1104AI / GeneralarXiv:2609.22068

    CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

    • The core idea is building training environments from source code alone, without development artifacts like issues or commits.
    • Agents explore the code themselves to write the task, then auto-grade solutions with tests grounded in the original code's execution.
    • It extracted 5,545 tasks from 3,185 open-source projects, covering 23 programming languages and 15 technical domains.
    • Training improved bug fixing (DeepSWE) by +11.7%p, whole-program construction (ProgramBench) by +17%p, and terminal work (Terminal-Bench v2.1) by +8.5%p.
    • The filtered 5,000 high-quality tasks beat an unrefined 8,000-task set. Quality won over quantity.
    • As training progressed, agents explored the codebase more and self-verified more often.
    AbstractPDF
  44. 1103AI AgentsarXiv:2609.20784

    RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning

    • Self-retiring mechanism dynamically removes teacher supervision as student policy matures
    • Prevents persistent distillation bias improving both sample efficiency and final performance
    • Applicable to multi-turn RL agent self-improvement loop design
    • Enables agents to decide when to become independent from the teacher — a meta-learning structure for self-improvement loops
    AbstractPDF
  48. 1099AI AgentsarXiv:2609.20804

    An Empirical Study of Harness Design for Coding Agents

    • 176 matched settings across 4 models (Nemotron-3 30B/120B/550B, Mistral-Medium-3.5) on SWE-Bench Verified and Terminal-Bench 2.1
    • Most context management benefit comes from preventing context overflow failures
    • Staged rule-based elision before LLM summarization is the most efficient strategy
    • Planning shifts from accuracy scaffold for weaker models to cost saver for stronger ones
    • Bash-capable models achieve equal performance with lower cost using bash-only interface
    AbstractPDF
  49. 1097LLMs / NLParXiv:2609.20612

    What Does Privileged Information Add to On-Policy Self-Distillation?

    • Built AMPLE-MATH, a dataset of 5,319 math problems each with six views that share the same answer but differ in explanation level, making it possible to isolate and measure the effect of privileged information.
    • In Qwen3-1.7B, most of the distillation gains appeared even without answer information, and the added benefit was modest, peaking with a polished solution.
    • In SmolLM3-3B, the full solution trace added about two percentage points at step 50.
    • With problems, answers, and evaluation held fixed, switching only the student's rollout mode from short direct responses to long thinking-enabled ones turned gains into losses in both model families.
    • Changing the teacher's token-level supervision signals often left student behavior largely unchanged, and 76% of the apparent gain from widening the loss window actually came from earlier stopping.
    • The paper concludes that OPSD mainly improves access to existing reasoning capabilities, and that the value of privileged information should be judged by what it adds to this transfer, not by how much of the solution it reveals.
    AbstractPDF
  50. 1096LLMs / NLParXiv:2609.20800

    JEPA-Anything: Learning Predictive Models across Different Worlds

    • Eliminates interference from monolithic learning by decomposing prediction targets into mutually orthogonal factors, predicting each separately, and recombining them (OPF).
    • Reused the same core structure across seven domains (vision, biology, clinical, control, molecular, physical fields, weather), improving scores on all 10 matched dynamics tasks against matched JEPA baselines.
    • Reduced single-intervention prediction error by 34.8% and achieved the lowest errors among compared methods on 100-step molecular rollouts in all four systems.
    • A biological intervention nominated by the model (IL-18 + CD73 blockade) was confirmed to have antitumor effects in actual co-culture, organoid, and mouse experiments.
    • From orbital motion data, it recovered the Keplerian scaling exponent (theoretical value -1.5) at -1.4991 without ever being taught the physical law.
    AbstractPDF