Automating Feature Flag Cleanup at Scale with Multi-Agent LLM Systems: An Industrial Experience at DoorDash

Key points

• DoorDash has over 56,000 feature flags across approximately 623 repositories with 1,659 currently stale, and the authors estimate one to two hours per manual cleanup.
• The orchestrator agent processed Jira tickets, queried the experimentation platform via MCP for the live rollout state to determine the target value, and after a human checkpoint spawned removal agents that edited code in parallel in git worktrees.
• Of the 50 evaluated flags, 31 (62%) were accepted in the first shot, 14 (28%) after minor revisions, and 5 (10%) failed, for an overall success rate of 45/50 (90%).
• The average time per cleanup was 13.8 minutes and the average LLM API cost was $4.79, and even complex flags reached an 85% (22/26) cleanup rate.
• Each removal agent could create a PR only after passing 95% patch coverage and the Detekt linter, and across all 50 removals no bugs or regressions were introduced by the changed code.
• The common cause of the 5 failures was incomplete removal of dead code in deep call chains, and the authors located the reliability boundary in completeness rather than correctness.

Paper links

External research summaries. These are not HDATF publications or measured product results.