Categories › Software Development › Testing & QA
Testing, benchmarking and auditing autonomous AI agents — methods, harnesses, evidence
Remote server: your requests and data go to agentreliability.dev. Claude Code: claude mcp add --transport http agent-reliability https://agentreliability.dev/mcp?via=manifest · Claude app: Settings → Connectors → Add custom connector.
The tools this server listed when we connected to it, in its own words.
search — Full-text search over the knowledge graph. Matching ignores accents and apostrophes, so query in the user's own words; every hit carries theget_entity — Fetch one knowledge object by id, with its claims and the sources each claim cites. Use this once search, answer or get_topic has given you get_topic — List the knowledge objects carrying a tag (topics are content-backed tags). PAGED: 25 objects by default, 200 at most, and a response budgetanswer — Answer a question from the corpus, or refuse. Returns only the claims that bear on the question, each with the sources it cites and its editget_sources — The instance's source registry — each entry with its evidence tier, reliability and access date. PAGED: 25 entries by default, 200 at most, get_related — Graph neighbours of an object: outgoing and incoming relations, each with its relation type. PAGED: 25 relations by default, 200 at most, anget_latest — Most recently verified knowledge objects (freshness signal). Use this to judge how current the corpus is, or to see what changed since you lget_overview — Corpus overview: what this instance knows, counts by type, published tags, freshness. Use this first when you land here and do not yet know compare — Two to six knowledge objects side by side: their cards, every indexed attribute as a matrix (the same fields api/index.json publishes, null Not tested by KitScout yet. When our harness runs it, pros, cons and the tasks we tried appear here, with the date and evidence.