site-architecture
Build a solid SEO foundation by implementing core technical elements like robots.txt directives, XML sitemaps, and structured meta tags. This skill guides you through configuring each component to ensure search engines can discover and properly index your site's content. Establish best practices that support long-term organic visibility.
site-architecture guides you through creating an effective robots.txt file that controls search engine crawler access. Place the file in your root directory and use directives like User-agent, Disallow, and Allow to specify which bots can access which paths. You can also set the crawl delay and point to your sitemap. Proper robots.txt configuration prevents indexing of duplicate or sensitive content while ensuring search engines can reach your important pages.
AI-generated summary based on this skill's SKILL.md
Install
alinaqi/maggy/site-architecture · repository language: Python
git clone https://github.com/alinaqi/maggy
cp -r maggy/skills/site-architecture ~/.claude/skills/site-architectureFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I set up robots.txt for SEO?
site-architecture guides you through creating an effective robots.txt file that controls search engine crawler access. Place the file in your root directory and use directives like User-agent, Disallow, and Allow to specify which bots can access which paths. You can also set the crawl delay and point to your sitemap. Proper robots.txt configuration prevents indexing of duplicate or sensitive content while ensuring search engines can reach your important pages.
What's included in a sitemap XML configuration guide?
site-architecture covers XML sitemap setup to help search engines discover all your pages efficiently. Your sitemap should list URLs with metadata like last modified date and change frequency. For large sites, use a sitemap index file to organize multiple sitemaps. Include your sitemap URL in robots.txt and submit it to Google Search Console. Dynamic sitemaps work well for Next.js and other frameworks that generate content programmatically.
How should I implement meta tags for search engine optimization?
site-architecture teaches you to implement essential meta tags including title tags, meta descriptions, Open Graph tags, and Twitter Card tags. Title tags should be 50–60 characters and include your primary keyword. Meta descriptions summarize page content in 150–160 characters to improve click-through rates. Open Graph and Twitter Card tags control how your content appears when shared on social platforms, enhancing visibility and engagement.
What are the best practices for improving Core Web Vitals?
site-architecture emphasizes optimizing Core Web Vitals—LCP (Largest Contentful Paint), CLS (Cumulative Layout Shift), and INP (Interaction to Next Paint)—to boost rankings and user experience. Minimize render-blocking resources, optimize images, defer non-critical JavaScript, and use a CDN for faster delivery. Stabilize layout by reserving space for dynamic content. Monitor metrics in Google PageSpeed Insights and Search Console to track improvements over time.
How do I implement canonical URLs and structured data?
site-architecture guides canonical URL implementation to prevent duplicate content issues by specifying the preferred version of a page. Use the rel="canonical" link tag in your page's head section. For structured data, add schema markup (JSON-LD format) to help search engines understand your content type—products, articles, breadcrumbs, or organizations. This enables rich results in search and improves visibility for AI crawlers indexing your site.
How can site-architecture help with AI crawler bot handling?
site-architecture provides guidance on managing AI bot access through robots.txt directives. You can allow or disallow specific bots like GPTBot and ClaudeBot using User-agent rules. This lets you control whether your content is used for AI training while maintaining search engine visibility. Configure these settings based on your content strategy and privacy preferences, balancing discoverability with data protection.
SKILL.md
rendered from the published skill — quoted content, verbatim
Site Architecture Skill
For technical website structure that enables discovery by search engines AND AI crawlers (GPTBot, ClaudeBot, PerplexityBot).
Philosophy
Content is king. Architecture is the kingdom.
Great content buried in poor architecture won't be discovered. This skill covers the technical foundation that makes your content findable by: - Google, Bing (traditional search) - GPTBot (ChatGPT), ClaudeBot, PerplexityBot (AI assistants) - Social platforms (Open Graph, Twitter Cards)
robots.txt
Basic Template
# robots.txt
# Allow all crawlers by default
User-agent: *
Allow: /
Disallow: /api/
Disallow: /admin/
Disallow: /private/
Disallow: /_next/
Disallow: /cdn-cgi/
# Sitemap location
Sitemap: https://yoursite.com/sitemap.xml
# Crawl delay (optional - be careful, not all bots respect this)
# Crawl-delay: 1
AI Bot Configuration
```txt
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 1 file
skills/site-architecture/SKILL.md