{"enrichment":{"faq":[{"a":"geo-crawlers audits your site's accessibility to AI crawlers from ChatGPT, Claude, Perplexity, Gemini, and Google. It analyzes your robots.txt file, meta tags, and HTTP headers to identify which crawlers can reach your content and flags any blocking issues preventing proper indexing by AI search products.","q":"How can I check if AI crawlers can access my website?"},{"a":"geo-crawlers helps you configure robots.txt directives for specific AI crawlers like GPTBot, PerplexityBot, and Bytespider. You can strategically allow or block crawlers based on your content strategy\u2014enabling visibility in ChatGPT and Claude while optionally restricting training-only crawlers or aggressive bots like Bytespider.","q":"What robots.txt AI crawler rules should I configure?"},{"a":"geo-crawlers identifies blocking issues preventing ChatGPT-User bot and other AI search crawlers from accessing your content. Common causes include restrictive robots.txt rules, noindex meta tags, or X-Robots-Tag headers. The tool provides specific recommendations to resolve each barrier and maximize your AI search visibility.","q":"Why isn't my site appearing in ChatGPT search results?"},{"a":"geo-crawlers supports multiple control methods: robots.txt Disallow rules for training-focused crawlers, meta noai tags, X-Robots-Tag headers, and llms.txt file configuration. You can distinguish between crawlers used for live AI search (which you may want to allow) versus those used only for model training (which you can block separately).","q":"How do I block AI crawlers from training on my content?"},{"a":"geo-crawlers helps you understand that some crawlers like GPTBot serve live ChatGPT search, while others like Google-Extended support Gemini training. You can allow live search crawlers to improve visibility in AI products while blocking training-only crawlers from using your content for model development, giving you granular control over both use cases.","q":"What is the difference between AI search visibility and model training access?"},{"a":"geo-crawlers guides you through adding meta noai tags to block AI crawlers, configuring X-Robots-Tag HTTP headers for fine-grained control, and setting up llms.txt files per IETF draft standards. The tool validates your implementation and provides an access map showing exactly which crawlers can reach each part of your site.","q":"How do I implement meta robots noai tags and content signals?"}],"shadow_tags":["crawler-governance","ai-indexing","search-visibility","bot-management","content-control","training-data-policy","technical-seo","ai-discovery"],"summary_rewrite":"geo-crawlers helps you understand how AI-powered search engines interact with your site. Detect which crawlers from ChatGPT, Claude, Perplexity, Gemini, and Google can access your content, then resolve any barriers preventing proper indexing. Stay ahead of AI search trends while preserving your traditional SEO performance."},"files":[{"bytes":17893,"path":"skills/geo-crawlers/SKILL.md","sha256":"e9429e5b16192b596931f5485ef9d2b3b746370d47d787212e6158d9884c0247","url":"https://skillfed.io/files/zubair-trabzada/geo-seo-claude/geo-crawlers/d73d7a90/SKILL.md"}],"id":"zubair-trabzada/geo-seo-claude/geo-crawlers","links":{"html":"https://skillfed.io/zubair-trabzada/geo-seo-claude/geo-crawlers","md":"https://skillfed.io/zubair-trabzada/geo-seo-claude/geo-crawlers.md","repo":"https://github.com/zubair-trabzada/geo-seo-claude"},"meta":{"agents_supported":[],"first_seen":"2026-07-28","forks":1456,"language":"Python","last_updated":"2026-05-27","license":"MIT","name":"geo-crawlers","publisher":"zubair-trabzada","stars":9138},"relations":{"similar":[{"id":"kostja94/marketing-skills/robots"},{"id":"alinaqi/maggy/site-architecture"},{"id":"40RTY-ai/shopify-admin-skills/shopify-admin-agentic-crawler-access"},{"id":"zubair-trabzada/geo-seo-claude/geo-technical"},{"id":"AgriciDaniel/codex-seo/seo-technical"},{"id":"guia-matthieu/clawfu-skills/ai-bot-log-audit"},{"id":"AgriciDaniel/claude-seo/seo-technical"},{"id":"AgriciDaniel/codex-seo/seo-geo"},{"id":"AgriciDaniel/claude-seo/seo-geo"},{"id":"schwepps/skills/seo-technical-audit"}]},"slug":{"owner":"zubair-trabzada","repo":"geo-seo-claude","skill":"geo-crawlers"},"version":"d73d7a90"}