LpWM: A Case for Sparse Representations in World Models
- Published
- Source
- arXiv
- Paper number
- 991
- Field
- Machine Learning
- arXiv ID
- 2608.22764
Key points
- It theoretically showed that nonlinear dynamics can be approximated linearly in a sufficiently high-dimensional one-hot latent space.
- It proposed LpWorldModel, which uses RDMReg regularization to learn sparse latent codes in which most coordinates are zero.
- On PushT, sparse representations achieved planning success rates up to 57 percentage points higher than dense representations in LeWM.
- A structure emerged spontaneously in which the positions of nonzero coordinates encode discrete dynamical modes, while their magnitudes encode continuous states within each mode.
- However, the benefit of sparse representations depends on predictor capacity: the difference from dense representations nearly disappeared in the easy Wall environment or with a sufficiently expressive predictor.
Paper links
External research summaries. These are not HDATF publications or measured product results.