ComposioBENCH
Try Bench

Pi Agent

pi.dev#6 of 7 by coverage
Compare harnesses
Best success66%with GPT-6 Astra
Cheapest run$0.16per task with DeepSeek V4 Pro
Fastest run127sper task with GPT-6 Astra
Model support
Open
Graded runs
2 of 2 models
Latest run
2026-09-10

Success rate

% of tasks passed by model · higher is better
GPT-6 Astra
65.5%
DeepSeek V4 Pro
56.7%

Cost per task

USD per task by model · lower is better
DeepSeek V4 Pro
$0.16
GPT-6 Astra
$1.29

Time per task

Seconds per task by model · lower is better
GPT-6 Astra
127s
DeepSeek V4 Pro
459s

Runs

  1. 01GPT-6 Astra
    66%
    $ / task$1.29Time / task127s
  2. 02DeepSeek V4 Pro
    57%
    $ / task$0.16Time / task459s