Categories › Web & Browser › Scraping, crawling & page-to-text
Check which AI crawlers a site allows and see pages the way AI crawlers do. Free, read-only.
Remote server: your requests and data go to siteiz.com. Claude Code: claude mcp add --transport http siteiz https://siteiz.com/mcp · Claude app: Settings → Connectors → Add custom connector.
The tools this server listed when we connected to it, in its own words.
check_ai_crawlers — Reads a website's robots.txt and reports, for 11 AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBoview_page_as_ai_crawler — Fetches one page the way most AI crawlers do (plain HTTP, no JavaScript) and reports what they actually receive: title, meta description, caexplain_ai_crawler — Explains what an AI crawler user agent does (operator, purpose, and what blocking it in robots.txt changes). Pass a name such as GPTBot or Oget_ai_visibility_report — Returns a published, dated Siteiz AI visibility report for a well-known company's homepage (score, grade, pillar scores, top issues). Omit tlist_my_sites — Lists the sites in your Siteiz account with their latest AI readability score, open fixes and linked Search Console property. Needs a Siteizget_search_performance — Clicks, impressions, CTR and average position from your connected Google Search Console, by query, page, device or country, for the last N dget_traffic_ranked_fixes — Joins your Search Console impressions with your latest Siteiz scan: scanned pages ranked by real Google impressions, each with its AI readabinspect_url_index — Runs Google Search Console URL inspection on one page of your site: index verdict, coverage, robots.txt, last crawl, user and Google canonicNot tested by KitScout yet. When our harness runs it, pros, cons and the tasks we tried appear here, with the date and evidence.