ActableSite AI Crawler Monitor
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
Connect it
Settings → Connectors → Add custom connector
URL: https://actablesite.com/mcp/Settings → Apps & Connectors → Advanced → Developer mode
Create connector with URL: https://actablesite.com/mcp/{
"mcpServers": {
"ai-crawler-robots-monitor": {
"url": "https://actablesite.com/mcp/"
}
}
}claude mcp add --transport http ai-crawler-robots-monitor https://actablesite.com/mcp/{
"servers": {
"ai-crawler-robots-monitor": {
"type": "http",
"url": "https://actablesite.com/mcp/"
}
}
}5 tools, verified live
audit_public_websiteread-onlyRun ActableSite's free three-signal readiness scan for a public website. Returns observable evidence for business identity, semantic reading path, and explicit action path. This does not guarantee AI crawling, indexing, citation, ranking, recommendation, traffic, or revenue.
check_ai_crawler_policyread-onlyCheck the homepage robots.txt rule for eight named OpenAI, Anthropic, Perplexity, and Google AI crawler tokens. The free result stays complete and includes a bounded optional handoff to exact monitoring terms. Results describe robots policy only and do not prove network access, authentic bot identity, crawling, indexing, citation, or ranking.
get_crawler_watch_offerread-onlyReturn the exact $9 monthly price, one-site scope, 15-minute check cadence, two-check confirmation, delivery and cancellation paths, and synthetic-check limitations. This tool is informational: present the offer and require explicit user confirmation before any separate checkout or purchase action.
get_full_report_offerread-onlyReturn the exact one-time price, deliverables, delivery path, checkout URL, support contact, and limitations for ActableSite's paid 15-check repair report. Present this information to the user and require explicit user confirmation before opening checkout or attempting any purchase.
get_practice_radar_offerread-onlyReturn the current Practice Radar weekly-edition receipt, public sample, exact $19 one-time and $39 monthly options, included fields, delivery and cancellation paths, and material NPI limitations. This tool is informational: it cannot open checkout or purchase. Present the offer and require explicit user confirmation before any separate checkout or purchase action.
Alternatives in Browser & Scraping
Compare →BRBrowserstackB
BrowserStack's Official MCP Server
SupadataB
Video transcripts, metadata and structured data from YouTube, TikTok, Instagram, X; web scraping.
Slack MCP ServerB
Catch up on Slack without reading it. Unreads, threads, search. Browser-session or hosted OAuth.
cloroB
Scrape AI answer engines and Google Search/News with country and state-level geo-targeting.
PDpdfrenderB
HTML and CSS to PDF MCP server with page headers, footers and page numbers. No headless browser.
PlayDropB
Publish, test and share browser games on PlayDrop from your AI agent.