01-auto-arena

Automatically evaluate and compare multiple AI models or agents without pre-existing test data. Generates test queries from a task description, collects responses from all target endpoints, auto-generates evaluation rubrics, runs pairwise comparisons via a judge model, and produces win-rate rankings with reports and charts. Supports checkpoint resume, incremental endpoint addition, and judge model hot-swap. Use when the user asks to compare, benchmark, or rank multiple models or agents on a cust

Install

KitScout test notes

Not tested by KitScout yet. When our harness runs it, pros, cons and the tasks we tried appear here, with the date and evidence.

Similar tools

• REA 456,474 ★
  Reverse engineer anything from your terminal or agent with one CLI and MCP server.
• ask-matt 323,337 ★
  Ask which skill or flow fits your situation. A router over the skills in this repo.
• chief-of-staff 323,337 ★
  Pursue a long-running goal in a single session by co-ordinating subagents.
• claude-handoff 323,337 ★
  Hand the current conversation off to a fresh background agent that picks up the work immediately.
• grill-me 323,337 ★
  A relentless interview to sharpen a plan or design.