NoProp: Training Neural Networks without Full Back-propagation or Full Forward-propagation
- Published
- Source
- arXiv
- Paper number
- 059
- Field
- Architecture / Training
- arXiv ID
- 2503.24322
Key points
- Traditional backpropagation is not biologically plausible because of the weight transport problem and the need to synchronize forward and backward passes.
- Full backpropagation requires storing intermediate activations, which creates substantial memory overhead and limits true parallelization across network layers because of sequential dependencies.
- The global error signal and sequential credit assignment in backpropagation can cause problems such as catastrophic forgetting and interference in lifelong learning settings.
- It introduces NoProp, a block-wise neural network learning method inspired by denoising diffusion and flow matching models.
- Each network block independently learns to denoise a noisy version of the target label using local backpropagation only within that block.
- It broadcasts the input and target labels to all blocks, allowing each block to optimize its objective independently without global error-signal propagation.
Paper links
External research summaries. These are not HDATF publications or measured product results.