firecrawl-scraper
Firecrawl Web Scraper turns website content into structured markdown optimized for AI models. It handles dynamic JavaScript rendering, bypasses bot protection, and supports single-page scraping, full-site crawling, URL discovery, and schema-based data extraction.
Firecrawl Web Scraper converts website content into LLM-ready markdown with JavaScript rendering and bot detection bypass.
AI-generated summary based on this skill's SKILL.md
Install
secondsky/claude-skills/firecrawl-scraper · repository language: TypeScript
git clone https://github.com/secondsky/claude-skills
cp -r claude-skills/plugins/firecrawl-scraper/skills/firecrawl-scraper ~/.claude/skills/firecrawl-scraperFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I use firecrawl API to scrape website content?
Firecrawl Web Scraper provides REST and SDK APIs for web scraping. You can make requests to scrape individual pages or crawl entire websites. The API returns content in LLM-ready markdown format. Firecrawl handles JavaScript rendering automatically, so dynamic content is captured. Check the firecrawl api documentation for endpoint details, authentication, and code examples in Python, TypeScript, and Node.js.
Can firecrawl scrape website with JavaScript-rendered pages?
Yes, Firecrawl Web Scraper is built to extract structured data from dynamic JavaScript-rendered web pages. It executes JavaScript in the browser before scraping, ensuring that content loaded after page load is captured. This makes it ideal for modern single-page applications and frameworks like React, Vue, and Angular.
What is firecrawl's approach to convert HTML to markdown?
Firecrawl Web Scraper automatically converts HTML to clean, LLM-ready markdown. The conversion preserves structure, links, and formatting while removing noise. This markdown output is optimized for use with language models, making it easy to feed web content directly into AI applications without additional preprocessing.
How does firecrawl crawl entire website and build knowledge bases?
Firecrawl Web Scraper can crawl entire websites by discovering and mapping all URLs, then systematically scraping each page. It supports batch scraping and can build comprehensive archives or knowledge bases. You can configure crawl depth, URL filters, and rate limiting to control the crawl scope and respect server resources.
Does firecrawl bypass bot detection and CAPTCHA protection?
Firecrawl Web Scraper includes capabilities to handle bot detection and CAPTCHA challenges. It uses techniques like browser automation and intelligent request handling to access protected content. However, always respect website terms of service and robots.txt when scraping, and use these features responsibly and legally.
What license does Firecrawl Web Scraper use?
Firecrawl Web Scraper is released under the MIT license, allowing free use, modification, and distribution in both open-source and commercial projects with minimal restrictions.
SKILL.md
rendered from the published skill — quoted content, verbatim
Firecrawl Web Scraper Skill
Status: Production Ready ✅ Last Updated: 2025-11-21 Official Docs: https://docs.firecrawl.dev API Version: v2.5
What is Firecrawl?
Firecrawl is a Web Data API for AI that turns entire websites into LLM-ready markdown or structured data. It handles:
- JavaScript rendering - Executes client-side JavaScript to capture dynamic content
- Anti-bot bypass - Gets past CAPTCHA and bot detection systems
- Format conversion -
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 7 files
plugins/firecrawl-scraper/skills/firecrawl-scraper/SKILL.md
plugins/firecrawl-scraper/skills/firecrawl-scraper/references/common-patterns.md
plugins/firecrawl-scraper/skills/firecrawl-scraper/references/endpoints.md
plugins/firecrawl-scraper/skills/firecrawl-scraper/templates/firecrawl-crawl-example.py
plugins/firecrawl-scraper/skills/firecrawl-scraper/templates/firecrawl-scrape-python.py
plugins/firecrawl-scraper/skills/firecrawl-scraper/templates/firecrawl-scrape-typescript.ts
plugins/firecrawl-scraper/skills/firecrawl-scraper/templates/firecrawl-worker-fetch.ts