site-architecture
Build a solid SEO foundation by implementing core technical elements like robots.txt directives, XML sitemaps, and structured meta tags. This skill guides you through configuring each component to ensure search engines can discover and properly index your site's content. Establish best practices that support long-term organic visibility.
site-architecture guides you through creating an effective robots.txt file that controls search engine crawler access. Place the file in your root directory and use directives like User-agent, Disallow, and Allow to specify which bots can access which paths. You can also set the crawl delay and point to your sitemap. Proper robots.txt configuration prevents indexing of duplicate or sensitive content while ensuring search engines can reach your important pages.
AI-generated summary based on this skill's SKILL.md
Decision gist · record as of 2026-07-14
site-architecture guides you through creating an effective robots.txt file that controls search engine crawler access. Place the file in your root directory and use directives like User-agent, Disallow, and Allow to specify which bots can access which paths. You can also set the crawl delay and point to your sitemap. Proper robots.txt configuration prevents indexing of duplicate or sensitive content while ensuring search engines can reach your important pages.
Use it when
- site-architecture covers XML sitemap setup to help search engines discover all your pages efficiently.
- site-architecture teaches you to implement essential meta tags including title tags, meta descriptions, Open Graph tags.
Verify before relying
Read SKILL.md below before installing (1 file). Open directory: indexed for reading, not audited.
Install
alinaqi/maggy/site-architecture · repository language: Python
Open directory. Skills are indexed for reading, not audited. Review a skill's body before installing it.
Frequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I set up robots.txt for SEO?
site-architecture guides you through creating an effective robots.txt file that controls search engine crawler access. Place the file in your root directory and use directives like User-agent, Disallow, and Allow to specify which bots can access which paths. You can also set the crawl delay and point to your sitemap. Proper robots.txt configuration prevents indexing of duplicate or sensitive content while ensuring search engines can reach your important pages.
What's included in a sitemap XML configuration guide?
site-architecture covers XML sitemap setup to help search engines discover all your pages efficiently. Your sitemap should list URLs with metadata like last modified date and change frequency. For large sites, use a sitemap index file to organize multiple sitemaps. Include your sitemap URL in robots.txt and submit it to Google Search Console. Dynamic sitemaps work well for Next.js and other frameworks that generate content programmatically.
How should I implement meta tags for search engine optimization?
site-architecture teaches you to implement essential meta tags including title tags, meta descriptions, Open Graph tags, and Twitter Card tags. Title tags should be 50–60 characters and include your primary keyword. Meta descriptions summarize page content in 150–160 characters to improve click-through rates. Open Graph and Twitter Card tags control how your content appears when shared on social platforms, enhancing visibility and engagement.
What are the best practices for improving Core Web Vitals?
site-architecture emphasizes optimizing Core Web Vitals—LCP (Largest Contentful Paint), CLS (Cumulative Layout Shift), and INP (Interaction to Next Paint)—to boost rankings and user experience. Minimize render-blocking resources, optimize images, defer non-critical JavaScript, and use a CDN for faster delivery. Stabilize layout by reserving space for dynamic content. Monitor metrics in Google PageSpeed Insights and Search Console to track improvements over time.
How do I implement canonical URLs and structured data?
site-architecture guides canonical URL implementation to prevent duplicate content issues by specifying the preferred version of a page. Use the rel="canonical" link tag in your page's head section. For structured data, add schema markup (JSON-LD format) to help search engines understand your content type—products, articles, breadcrumbs, or organizations. This enables rich results in search and improves visibility for AI crawlers indexing your site.
How can site-architecture help with AI crawler bot handling?
site-architecture provides guidance on managing AI bot access through robots.txt directives. You can allow or disallow specific bots like GPTBot and ClaudeBot using User-agent rules. This lets you control whether your content is used for AI training while maintaining search engine visibility. Configure these settings based on your content strategy and privacy preferences, balancing discoverability with data protection.
SKILL.md
Rendered from the published skill. Quoted content, verbatim.
Site Architecture Skill
For technical website structure that enables discovery by search engines AND AI crawlers (GPTBot, ClaudeBot, PerplexityBot).
Philosophy
Content is king. Architecture is the kingdom.
Great content buried in poor architecture won't be discovered. This skill covers the technical foundation that makes your content findable by: - Google, Bing (traditional search) - GPTBot (ChatGPT), ClaudeBot, PerplexityBot (AI assistants) - Social platforms (Open Graph, Twitter Cards)
robots.txt
Basic Template
# robots.txt
# Allow all crawlers by default
User-agent: *
Allow: /
Disallow: /api/
Disallow: /admin/
Disallow: /private/
Disallow: /_next/
Disallow: /cdn-cgi/
# Sitemap location
Sitemap: https://yoursite.com/sitemap.xml
# Crawl delay (optional - be careful, not all bots respect this)
# Crawl-delay: 1
AI Bot Configuration
```txt
(truncated - see the full file via the links below)
File tree — 1 file
skills/site-architecture/SKILL.md
Let your AI agent find skills like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 56,283 agent skills by what they can do, searchable in plain language.
wish › “Set up technical SEO foundation with robots.txt, sitemap, and meta tags”
Give your agent the search over MCP, or paste the wish link into any chat. No install? Search from any chat →
Related skills
seo-optimizer helps you perform in-depth technical SEO analysis to uncover crawlability barriers and indexing obstacles affecting your site's search visibility. Detect configuration problems, structural issues, and missed optimization chances that prevent search engines from properly accessing and ranking your content.
This skill conducts a complete SEO audit of your website, analyzing technical health, content optimization, and competitive positioning. It identifies issues that may be holding back search visibility and surfaces actionable recommendations to improve rankings and organic traffic.
geo-crawlers helps you understand how AI-powered search engines interact with your site. Detect which crawlers from ChatGPT, Claude, Perplexity, Gemini, and Google can access your content, then resolve any barriers preventing proper indexing. Stay ahead of AI search trends while preserving your traditional SEO performance.
This skill suite delivers actionable SEO and performance recommendations grounded in Google Lighthouse standards and Core Web Vitals metrics. Works across any modern framework—React, Vue, Angular, Svelte, Next.js, Nuxt, Astro, or vanilla HTML—to help you identify and fix the technical factors that impact search rankings and user experience.
This skill equips TYPO3 administrators with practical SEO enhancements to boost search engine visibility and organic reach. Configure metadata, optimize page structure, and implement best practices directly within your TYPO3 environment to drive measurable improvements in search rankings and user discovery.
Manage crawler permissions and site indexing rules through robots.txt setup and validation. This skill helps you configure directives that guide search engines and AI bots on which content to crawl and index, while auditing existing configurations for compliance and optimization.
More skills ai-bot-log-audit (MIT) · xml-sitemap (MIT) · seo-visibility-expert (MIT) · Seo (NOASSERTION) · geo-technical (MIT) · Og Image (NOASSERTION) · Site Crawler (unlicensed) · shopify-admin-agentic-crawler-access (MIT)