LeaderboardEvery agent setup scored on χ-Bench, ours and the community's.
Submissions run χ-Bench's 75 tasks across prior authorization, utilization management and care management, graded by the two-layer verifier under pass@1 with a binary reward; a run that skipped a domain shows a dash there. The paper explains how a run is graded.
45
Agent setups scored
54.7%
Best overall, erius · claude-opus-5
2026-08-12
Last updated
Select a column to sort
- 1claude-opus-5erius · Michael Johnson (MJ)54.7%
- 2claude-opus-4-8erius · Michael Johnson (MJ)37.3%
- 3claude-opus-5claude-code37.3%
- 4claude-opus-4-8claude-code33.3%
- 5claude-opus-4-6claude-code28.0%
- 6claude-sonnet-4-6claude-code26.2%
- 7gpt-5.6-solcodex25.3%
- 8kimi-k3openai-agents25.3%
- 9claude-opus-4-7claude-code24.4%
- 10claude-fable-5claude-code24.0%
- 11MedGuardhermes · cuilinke22.7%
- 12gpt-5.5codex20.9%
- 13claude-sonnet-5claude-code20.0%
- 14glm-5.1openai-agents18.7%
- 15glm-5.1hermes18.7%
- 16glm-5.2openai-agents18.7%
- 17claude-opus-4-7openclaw17.3%
- 18glm-5.1openclaw16.9%
- 19qwen-3.6-maxhermes16.4%
- 20gpt-5.4codex16.0%
- 21qwen-3.6-maxopenai-agents15.6%
- 22kimi-k2.6hermes15.6%
- 23kimi-k2.6openai-agents15.1%
- 24deepseek-v4-proopenai-agents14.2%
- 25deepseek-v4-prohermes13.8%
- 26gpt-5.6-terracodex13.3%
- 27gpt-5.6-lunacodex13.3%
- 28gemini-3-flashgemini-cli12.5%
- 29deepseek-v4-proopenclaw11.1%
- 30glm-5.1deepagents11.1%
- 31deepseek-v4-prodeepagents10.7%
- 32kimi-k2.6openclaw10.2%
- 33qwen-3.6-maxdeepagents9.3%
- 34gpt-5.4-minicodex8.4%
- 35TML Inkling 256Kopenai-agents8.0%
- 36gemini-3.1-progemini-cli7.1%
- 37claude-haiku-4-5claude-code6.2%
- 38grok-4.3openai-agents5.8%
- 39qwen-3.6-maxopenclaw4.9%
- 40grok-4.3hermes4.4%
- 41kimi-k2.6deepagents3.1%
- 42grok-4.3deepagents2.2%
- 43grok-4.3openclaw0.4%
- 44Nemotron 3 Ultra 256Kopenai-agents0.0%
- 45gpt-5.5pa-codex-agent · MedArise–
Ranked by pass@1 on all domains. A setup that did not run a domain shows a dash and sorts last. The arrow opens each run's submission packet.
Have an agent to put on the board?
Run the suite with your harness and model, package the results with
cb submission prepare, and open a pull request.