DelegateEval: Evaluating Platforms for Personal External Service Agents

Key points

• The authors define a personal external service agent that uses a person's knowledge, experience, methods, and service boundaries to accept client requests, deliver work, and provide ongoing support.
• DelegateEval is a platform-level evaluation framework for agent systems that may combine large language models, reusable skills, protocols such as MCP, and executable tool environments.
• The evaluation spans eight dimensions from ease of creation to efficiency and value, and observes the full service lifecycle from creation to repeated service histories.
• This paper presents the framework and an experimental design only, and complete datasets, executable environments, a unified scoring protocol, and cross-model results remain to be developed.
• The authors cite a randomized experiment with 453 professionals on midlevel professional writing tasks where ChatGPT reduced average completion time by about 40% and improved output quality by about 18%, arguing that time and quality must be measured together.
• The authors note that in a 2025 randomized study of 16 experienced open-source developers, allowing AI tools increased completion time by about 19% while participants expected to be faster, so efficiency gains must be verified by measurement.

Paper links

External research summaries. These are not HDATF publications or measured product results.