ComposioBENCH
Try Bench

Measuring Model and Harness Performance on Real-World Tasks

Success

Codex
72%
Hermes Agent
72%
Command Code
72%
Claude Code
69%
OpenCode
66%
Pi Agent
66%
DeepSeek
53%
0%50%100%
% of 29–30 tasks · best model

Cost per task

DeepSeek
$0.12
OpenCode
$1.20
Pi Agent
$1.29
Hermes Agent
$1.51
Codex
$1.63
Command Code
$2.06
Claude Code
$15.72
$0.000$7.86$15.72
USD per task · best model

Time per task

Hermes Agent
122s
OpenCode
126s
Pi Agent
127s
Codex
151s
Command Code
159s
Claude Code
194s
DeepSeek
388s
0s194s388s
Avg seconds per task · best model

Harness leaderboard

01
Command Code
18/30
$ / task$0.11Avg time375s
02
Hermes Agent
17/30
$ / task$0.15Avg time447s
03
Pi Agent
17/30
$ / task$0.16Avg time459s
04
Codex
16/30
$ / task$0.10Avg time482s
05
DeepSeek
16/30
$ / task$0.12Avg time388s
06
Claude Code
15/30
$ / task$0.25Avg time414s
07
OpenCode
15/30
$ / task$0.16Avg time403s

Every harness running DeepSeek V4 Pro · 30 selected task results per pair · costs average available normalized token costs

Cost vs quality

Success vs cost per task40%45%50%55%60%65%70%$0.1$0.2USD per task (log)SuccessCommand CodeHermes AgentPi AgentCodexDeepSeekClaude CodeOpenCode
HarnessSuccess · $ / task
  1. 01Command CodeFrontier60%$0.11
  2. 02Hermes Agent57%$0.15
  3. 03Pi Agent57%$0.16
  4. 04CodexFrontier53%$0.10
  5. 05DeepSeek53%$0.12
  6. 06OpenCode50%$0.16
  7. 07Claude Code50%$0.25

Every harness running DeepSeek V4 Pro · cost averages available normalized token costs