SpecPilot: Autonomous Algorithm Discovery and Systems Optimization for Speculative Decoding
- Published
- Source
- arXiv
- Paper number
- 1415
- Field
- Reasoning / Efficiency
- arXiv ID
- 2611.02501
Key points
- SpecPilot is an autonomous agentic loop that automatically optimizes speculative decoding, which speeds up LLM decoding with a lightweight drafter, for each deployment.
- A cost model locates the largest gaps between the current system and its ceiling, and parallel agents explore directions and integrate the successful ones.
- Token forcing checks every output token against stock to detect reward hacking.
- The loop discovered W12, a general lossless 12-bit weight format, tree verification for hybrid recurrent targets, two-level verification, and three batching mechanisms.
- Without training or quality loss, it raised the speedup over AR from 4.46× to 10.87× with one request, and from 1.94× to 3.93× with batching.
- A structured memory holds 140 validated mechanisms and reuses them for new drafters, models, and GPU platforms.
Paper links
External research summaries. These are not HDATF publications or measured product results.