Initial request
The user gives a short request that names the visible deliverable, but not every preference, constraint, or dependency.
Proactive Personal Assistant Benchmark
π-Bench Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows
Benchmark Overview
100multi-turn tasks
5domain personas
524hidden intents
678checklist and rule graders
Hidden intents
Missing habits, preferences, constraints, and task dependencies are recoverable from profiles, prior sessions, workspace files, app state, tool results, and cross-session context.
Agent behavior
π-Bench tests whether the agent infers what it can, asks focused questions when needed, and carries those decisions through later turns, tool use, and artifact revisions.
π-Bench evaluates long-horizon personal assistant workflows in persistent project environments. The user gives a short request, but completing it well may depend on preferences, constraints, files, and decisions revealed in earlier sessions and reused later.
Each task starts with a natural but underspecified instruction. The agent works inside a persistent workspace, interacts with the user, uses tools, and creates or revises artifacts. Hidden intents are the missing but recoverable requirements: for example, a deck template, preferred metrics, naming conventions, project-specific constraints, or dependencies established in prior work. Some hidden intents are available from the start, while others are revealed gradually through interaction, tool use, or workspace inspection.
This is different from evaluating only explicit instructions, isolated memory recall, or short GUI actions. π-Bench asks whether an agent can decide which context matters, when to ask for clarification, and how to carry those decisions into workspace artifacts.
The benchmark separates two questions. Completeness measures whether the final workflow succeeds, including the explicit request and the relevant hidden intents. Proactivity measures whether the agent reduces the user's specification burden by inferring hidden intents from context or asking targeted clarifying questions early enough to guide later work.
Benchmark Design
π-Bench is organized around persistent user episodes, underspecified task sessions, and two complementary scores: proactive intent recovery and task completion.
Persistent user workflow
Each persona has one 20-session episode in a shared workspace. Preferences, files, prior outputs, and dependencies can carry over.
Researcher
Marketer
Law Trainee
Pharmacist
Financier
Interaction with hidden intents
A natural initial request leaves some requirements unstated. The agent uses context, tools, files, and focused clarification while hidden intents are tracked.
history
workspace
apps
tools
skills
Proactivity and completeness
Proc counts agent-driven hidden-intent resolution. Comp checks verifiable requirements across the full trajectory, tool records, and artifacts.
Proc
completed or elicited hidden intents
Comp
satisfied checklist criteria
Evaluation
π-Bench reports Proc and Comp as two main metrics.
Did the agent reduce underspecification in a proactive way?
Proc measures the share of hidden intents resolved proactively: an intent counts when the agent satisfies it directly through its response, tool use, or artifacts, or asks a targeted clarifying question about the specific missing preference, constraint, or dependency before proceeding. Intents surfaced only by user-provided information are not counted as proactive.
Did the final trajectory satisfy the task?
Comp is the average checklist score over verifiable task requirements. Checklist graders read the full trajectory, including tool records and produced artifacts, to assess whether the resulting workflow satisfies the required outcomes.
Leaderboard
Benchmark Leaderboard
The leaderboard ranks by Avg Proc by default. Use the Rank by control to switch to Avg Comp; the two rankings can differ, so model quality should be read across both metrics.
Current view: ranked by Avg Proc on all tasks.
Rank by metric
Loading leaderboard...
Metric
Avg Proc = hidden intents completed by the agent or elicited through focused clarification
higher is better
Overall Performance
Resources
Citation
@misc{zhang2026pibenchevaluatingproactivepersonal,
title={$\pi$-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows},
author={Haoran Zhang and Luxin Xu and Zhilin Wang and Runquan Gui and Shunkai Zhang and Haodi Lei and Zihao He and Bingsu He and Chicheng Qin and Tong Zhu and Xiaoye Qu and Yang Yang and Yu Cheng and Yafu Li},
year={2026},
eprint={2605.14678},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2605.14678}
}