---
id: firecrawl-py
version: "4.35.0"
license: MIT License
license_treatment: permissive
maintenance: active
---

# firecrawl-py — Python SDK for the Firecrawl API: web scraping, crawling, web search, and scientific literature search over a research paper index of PubMed, bioRxiv, medRxiv and arXiv abstracts

License: permissive · Maintenance: active · Downloads: 7.3M/mo

## What it is and what it does

Firecrawl-py is a Python client for the Firecrawl web scraping and search service. It provides methods to scrape individual URLs, crawl entire websites, search the web, and query a research paper index covering PubMed, bioRxiv, medRxiv, and arXiv abstracts. The library returns content in multiple formats—Markdown, HTML, video, product data, menu data—and supports both synchronous and asynchronous operations. It depends on requests, httpx, websockets, aiohttp, pydantic, python-dotenv, and nest-asyncio for HTTP, async, and configuration handling.

The package is designed for AI agents and applications that need to extract structured or semi-structured data from web pages or search academic literature. It handles pagination automatically, supports interactive browsing actions, and provides specialized extraction for product pages and restaurant menus. The research paper search methods return raw JSON with camelCase keys, distinct from the rest of the SDK's snake_case convention.

Use it for:

- Scrape product pages to extract title, price, availability, and variants deterministically.
- Crawl a website up to a specified depth and format limit to build a searchable knowledge base.
- Search academic literature by querying abstracts from biomedical and physics preprint repositories.
- Extract Markdown from web pages for ingestion into RAG (retrieval-augmented generation) pipelines.
- Map a website's URL structure with optional sitemap and subdomain inclusion for discovery.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Client library for the Firecrawl API that scrapes, crawls, and searches the web, returning clean Markdown or structured data; also indexes research papers from PubMed, bioRxiv, medRxiv, and arXiv.

Yes. The package is actively maintained, has no known vulnerabilities, uses a permissive MIT license, and installs with low friction. It is well-suited for developers building AI agents or applications that need web scraping, crawling, or research paper search. The primary constraint is the requirement for a Firecrawl API key and service account.

## Install

pip install firecrawl-py
uv add firecrawl-py
poetry add firecrawl-py

## Installing firecrawl-py

Before you install: Low install friction; pure Python wheel with common async and HTTP dependencies. Active maintenance with a release 2 days old and high repository activity (167374 stars).

License in practice: MIT License permits commercial and private use with minimal restrictions; suitable for most projects.

Quickstart:

pip install firecrawl-py

from firecrawl import Firecrawl
firecrawl = Firecrawl(api_key="fc-YOUR_API_KEY")
data = firecrawl.scrape('https://example.com', formats=['markdown', 'html'])
print(data)

Requires a Firecrawl API key from firecrawl.dev; can be set via FIRECRAWL_API_KEY environment variable or passed to the Firecrawl constructor.

Verify before relying:

- Whether the paper index (~43M abstracts) is kept current and how frequently it is updated.
- Rate limits and quota behavior for the Firecrawl API service itself.
- Performance characteristics for large-scale crawls or batch scrapes.

## Package facts

- License: MIT License (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 7.3M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

web scraping sdk, web crawling api client, markdown extraction from urls, research paper search, structured data extraction, firecrawl api wrapper, scientific literature search, web-scraping, research-papers, ai-agents

[View on SkillFed](https://skillfed.io/packages/firecrawl-py) · [View on PyPI](https://pypi.org/project/firecrawl-py/)