Agentic Ensemble with Vision-Language Models for Visual Anomaly Detection

Key points

It normalized the scores and anomaly maps of fixed pretrained detectors and summarized disagreement statistics into a structured evidence state.
Only samples whose conflict score exceeded the threshold τ = 0.5 invoked the VLM, while the rest kept the deterministic preliminary fusion result.
The VLM was constrained to select one image-score action and one map action from predefined actions such as consensus, robust aggregation, detector weighting, and disagreement correction, so it cannot freely generate a new mask.
On MVTec AD and VisA it achieved the best mean image AUROC and pixel AP among the compared fusion methods.
On the large-scale Real-IAD evaluation it improved image AUROC/AP/F1 from 72.47/58.85/59.66 to 75.53/62.54/62.49.
It used LLaVA-7B as the VLM backend, adding only 489 VLM calls in the main experiments and 1286 calls in the transfer experiments.

External research summaries. These are not HDATF publications or measured product results.