Google releases MKA benchmark for visual knowledge work agents

Summary

Google DeepMind and evaluation firm Artificial Analysis launched the Multimodal Knowledge Agent (MKA) benchmark. Experts across 10 industries and 83 occupations created 200 real-world tasks that take a seasoned professional three to eight hours, graded by rubrics weighting factual accuracy at 75% and visual design at 25%. Every model runs the tasks in the same open-source agent harness, and tested models averaged between 30 and 130 turns per task, a wide efficiency gap. Google reports that its automated grading agreed with human experts 96% of the time.

Why it matters for our work

The benchmark provides a yardstick for how far agents can take over knowledge work that depends on reading visual files such as blueprints, decks, and spreadsheets. By reporting efficiency in turns alongside quality, it helps judge readiness for real-world adoption.

Translated from the Korean original. Summaries may be translated and edited. Commentary reflects our perspective; forecasts remain the source’s views.