firecrawl-scraper
Firecrawl Web Scraper turns website content into structured markdown optimized for AI models. It handles dynamic JavaScript rendering, bypasses bot protection, and supports single-page scraping, full-site crawling, URL discovery, and schema-based data extraction.
Firecrawl Web Scraper converts website content into LLM-ready markdown with JavaScript rendering and bot detection bypass.
AI-generated summary based on this skill's SKILL.md
Decision gist · record as of 2026-07-25
Firecrawl Web Scraper converts website content into LLM-ready markdown with JavaScript rendering and bot detection bypass. Firecrawl Web Scraper turns website content into structured markdown optimized for AI models. It handles dynamic JavaScript rendering, bypasses bot protection, and supports single-page scraping, full-site crawling, URL discovery, and schema-based data extraction.
Use it when
- Yes, Firecrawl Web Scraper is built to extract structured data from dynamic JavaScript-rendered web pages.
- Firecrawl Web Scraper automatically converts HTML to clean, LLM-ready markdown.
Verify before relying
Read SKILL.md below before installing (7 files). Open directory: indexed for reading, not audited.
Install
secondsky/claude-skills/firecrawl-scraper · repository language: TypeScript
Open directory. Skills are indexed for reading, not audited. Review a skill's body before installing it.
Frequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I use firecrawl API to scrape website content?
Firecrawl Web Scraper provides REST and SDK APIs for web scraping. You can make requests to scrape individual pages or crawl entire websites. The API returns content in LLM-ready markdown format. Firecrawl handles JavaScript rendering automatically, so dynamic content is captured. Check the firecrawl api documentation for endpoint details, authentication, and code examples in Python, TypeScript, and Node.js.
Can firecrawl scrape website with JavaScript-rendered pages?
Yes, Firecrawl Web Scraper is built to extract structured data from dynamic JavaScript-rendered web pages. It executes JavaScript in the browser before scraping, ensuring that content loaded after page load is captured. This makes it ideal for modern single-page applications and frameworks like React, Vue, and Angular.
What is firecrawl's approach to convert HTML to markdown?
Firecrawl Web Scraper automatically converts HTML to clean, LLM-ready markdown. The conversion preserves structure, links, and formatting while removing noise. This markdown output is optimized for use with language models, making it easy to feed web content directly into AI applications without additional preprocessing.
How does firecrawl crawl entire website and build knowledge bases?
Firecrawl Web Scraper can crawl entire websites by discovering and mapping all URLs, then systematically scraping each page. It supports batch scraping and can build comprehensive archives or knowledge bases. You can configure crawl depth, URL filters, and rate limiting to control the crawl scope and respect server resources.
Does firecrawl bypass bot detection and CAPTCHA protection?
Firecrawl Web Scraper includes capabilities to handle bot detection and CAPTCHA challenges. It uses techniques like browser automation and intelligent request handling to access protected content. However, always respect website terms of service and robots.txt when scraping, and use these features responsibly and legally.
What license does Firecrawl Web Scraper use?
Firecrawl Web Scraper is released under the MIT license, allowing free use, modification, and distribution in both open-source and commercial projects with minimal restrictions.
SKILL.md
Rendered from the published skill. Quoted content, verbatim.
Firecrawl Web Scraper Skill
Status: Production Ready ✅ Last Updated: 2025-11-21 Official Docs: https://docs.firecrawl.dev API Version: v2.5
What is Firecrawl?
Firecrawl is a Web Data API for AI that turns entire websites into LLM-ready markdown or structured data. It handles:
- JavaScript rendering - Executes client-side JavaScript to capture dynamic content
- Anti-bot bypass - Gets past CAPTCHA and bot detection systems
- Format conversion -
(truncated - see the full file via the links below)
File tree — 7 files
plugins/firecrawl-scraper/skills/firecrawl-scraper/SKILL.md
plugins/firecrawl-scraper/skills/firecrawl-scraper/references/common-patterns.md
plugins/firecrawl-scraper/skills/firecrawl-scraper/references/endpoints.md
plugins/firecrawl-scraper/skills/firecrawl-scraper/templates/firecrawl-crawl-example.py
plugins/firecrawl-scraper/skills/firecrawl-scraper/templates/firecrawl-scrape-python.py
plugins/firecrawl-scraper/skills/firecrawl-scraper/templates/firecrawl-scrape-typescript.ts
plugins/firecrawl-scraper/skills/firecrawl-scraper/templates/firecrawl-worker-fetch.ts
Let your AI agent find skills like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 56,283 agent skills by what they can do, searchable in plain language.
wish › “Scrape and convert website content to LLM-ready markdown format”
Give your agent the search over MCP, or paste the wish link into any chat. No install? Search from any chat →
Related skills
This skill leverages the Firecrawl API to pull content from web pages in multiple formats—markdown, HTML, JSON, screenshots, and PDFs. It supports single-page scraping, full-site crawling with depth and path controls, batch processing of multiple URLs, and browser automation through actions like clicks and scrolls. Use it whenever you need to gather web data, extract structured information with custom schemas, or interact with dynamic pages.
Extract individual web pages into clean, structured markdown using Firecrawl's API. The skill manages JavaScript rendering, anti-bot defenses, and dynamic content automatically, supporting multiple output formats including markdown, HTML, summaries, and screenshots. Configure proxy modes and timeouts to handle everything from standard sites to protected news content.
Web Research provides access to neural search and web scraping through x402-protected endpoints, combining Exa's semantic capabilities with Firecrawl's extraction power. Use it for deep investigations, finding similar content, extracting clean text from blocked sites, and crawling multiple pages—each with transparent per-task pricing starting at $0.002.
Firecrawl exposes web search and page scraping through a simple REST API, returning results as clean markdown. Use the `/search` endpoint to find pages by query, then `/scrape` to extract content from any URL or public PDF. Requires an API key stored in your agent's vault.
seo-firecrawl automates website discovery and analysis by crawling entire domains to extract content, identify all pages, and map structural patterns. It handles JavaScript-rendered sites, supports targeted crawls via path filters, and integrates with other SEO agents for technical audits, content analysis, and schema validation.
Seo Firecrawl enables full-site crawling, URL discovery, and single-page scraping with JavaScript rendering support. Use it to extract all pages and links from a domain, detect broken links, analyze site structure, or audit content at scale—ideal for comprehensive SEO reviews and competitor research.
More skills Firecrawl (unlicensed)