VISReg: Variance-Invariance-Sketching Regularization for JEPA training
- Published
- Source
- arXiv
- Paper number
- 291
- Field
- Computer Vision
- arXiv ID
- 2606.02572
Key points
- Sketching-based methods such as SIGReg address this by aligning embeddings to an isotropic Gaussian, but they lack flexibility and suffer from vanishing gradients under collapse.
- VISReg scales linearly, outperforms existing regularization methods on low-quality datasets, and shows robustness in long-tail and low-rank settings.
- Pretrained on ImageNet-1K, VISReg achieves SOTA performance on out-of-distribution datasets.
Paper links
External research summaries. These are not HDATF publications or measured product results.