Toward Native Multimodal Modeling: A Roadmap
- Published
- Source
- arXiv
- Paper number
- 233
- Field
- Computer Vision
- arXiv ID
- 2605.25343
Key points
- This roadmap systematically classifies NMM models along three axes: T2M for generation, M2T for understanding, and M2M for symmetric modeling.
- It compares two design philosophies: full discretization through digital tokens versus preserving modality characteristics through a continuous feature space.
- It summarizes trends in extreme VAE compression and dynamic sparse-attention pruning techniques for addressing token explosion.
- It analyzes explicit-rule and implicit-emergence approaches for teaching physical law understanding, such as rigid-body dynamics, gravity, and collision, in video generation.
- It proposes unified timeline anchoring and deep architectural coupling for precise audio-visual synchronization.
- It includes data and evaluation strategies, such as physical decoupling and encoder-free modeling, to address the comprehension-generation dilemma.
Paper links
External research summaries. These are not HDATF publications or measured product results.