KitScout

Categories › Web & Browser › Scraping, crawling & page-to-text

Atalaya AI crawler check

MCP serverVerified domain · atalaya-ai.comRemote● LiveFree to try

Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.

Install

Remote server: your requests and data go to atalaya-ai.com. Claude Code: claude mcp add --transport http atalaya-ai-crawler-check https://atalaya-ai.com/mcp · Claude app: Settings → Connectors → Add custom connector.

What it can do

The tools this server listed when we connected to it, in its own words.

  • check_ai_crawler_access — Checks whether AI crawlers can reach and read a public website or web app: which AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot
  • list_ai_crawlers — Lists the AI crawler user-agent tokens Atalaya checks, by vendor and purpose (training, search index, user-triggered fetch), so you can writ

KitScout test notes

Not tested by KitScout yet. When our harness runs it, pros, cons and the tasks we tried appear here, with the date and evidence.

Similar tools