icrawler
A multi-thread crawler framework with many builtin image crawlers provided.
Decision gist · record as of 2026-08-14
Yes, if you need a lightweight image crawler and can tolerate aging maintenance. The package is stable for straightforward use cases (downloading from Google Images, Bing, or Flickr), has no known vulnerabilities, and installs cleanly. However, be aware that the last release was 549 days ago—built-in crawlers may break if target websites change their HTML or API structure, and you may need to fork or patch the code yourself. Best suited for one-off scripts or prototypes rather than production systems that require active support.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.7 or later.
- Built-in crawlers depend on external site APIs and HTML structures that may change without notice.
- Low install friction with a pure-Python wheel distribution.
License · maintenance · safety
permissive license (permissive) — MIT license permits unrestricted use, modification, and distribution. No restrictions on commercial or proprietary use.
last release 2025-02-11 (549 days) · last repo commit 2025-12-10 · 930 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 105,039 downloads/mo, #12,726 on PyPI
Alternatives
Verify before relying
pip install icrawler
from icrawler.builtin import GoogleImageCrawler
google_crawler = GoogleImageCrawler(storage={'root_dir': 'images'})
google_crawler.crawl(keyword='cat', max_num=100)- Whether built-in crawlers for Google, Bing, Baidu, and Flickr still work reliably given the 549-day gap since last release.
- Whether the package handles modern anti-scraping measures (rate limiting, user-agent blocking, JavaScript rendering).
- Performance and stability when running with high thread counts on large crawl jobs.
What it is and what it does
icrawler is a Python framework for writing multi-threaded web crawlers that focus on downloading media—images, videos, and other files—from websites. It abstracts away the complexity of thread management, exception handling, and queue coordination, letting you concentrate on what to crawl. The framework uses a three-stage pipeline: a Feeder puts page URLs into a queue, a Parser extracts media URLs from those pages, and a Downloader fetches and saves the files. Each stage runs in its own thread pool, so you can tune concurrency independently.
The package ships with ready-to-use crawlers for Google Images, Bing Images, Baidu Images, and Flickr, so you can start downloading images with a few lines of code. You can also extend the framework to build crawlers for custom websites by implementing your own Feeder, Parser, and Downloader classes. It depends on beautifulsoup4, lxml, pillow, requests, and pyyaml—standard tools for HTTP requests, HTML parsing, and image handling.
Use it for
- Download batches of images from Google Images, Bing, or Baidu for dataset creation or research.
- Build a custom crawler to scrape product images or media from a specific website.
- Automate media collection from Flickr or other image-hosting services with configurable thread pools.
- Extract and download images from search results with filters (size, color, date, license).
- Prototype a web scraper without the overhead of learning Scrapy's full architecture.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you need a lightweight image crawler and can tolerate aging maintenance.
The package is stable for straightforward use cases (downloading from Google Images, Bing, or Flickr), has no known vulnerabilities, and installs cleanly. However, be aware that the last release was 549 days ago—built-in crawlers may break if target websites change their HTML or API structure, and you may need to fork or patch the code yourself. Best suited for one-off scripts or prototypes rather than production systems that require active support.
Install
icrawler on PyPI
Before you install
Low install friction with a pure-Python wheel distribution. Maintenance is aging—last release was 549 days ago—but the repository remains active and the package supports current Python versions (3.7–3.12).
Requires Python 3.7 or later. Built-in crawlers depend on external site APIs and HTML structures that may change without notice.
License in practice
MIT license permits unrestricted use, modification, and distribution. No restrictions on commercial or proprietary use.
Quickstart
pip install icrawler
from icrawler.builtin import GoogleImageCrawler
google_crawler = GoogleImageCrawler(storage={'root_dir': 'images'})
google_crawler.crawl(keyword='cat', max_num=100)Verify before relying
- Whether built-in crawlers for Google, Bing, Baidu, and Flickr still work reliably given the 549-day gap since last release.
- Whether the package handles modern anti-scraping measures (rate limiting, user-agent blocking, JavaScript rendering).
- Performance and stability when running with high thread counts on large crawl jobs.
Package facts
| License | permissive license permissive |
| Python support | Supports the current Python release >=3.7 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 7 packagesbeautifulsoup4bs4lxmlpillowpyyamlrequestssix |
| Maintenance | Aging 549 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 105,039 / month, #12,726 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 4 - BetaIntended Audience :: DevelopersIntended Audience :: EducationIntended Audience :: Science/ResearchLicense :: OSI Approved :: MIT LicenseOperating System :: OS IndependentProgramming Language :: Python :: 3 :: OnlyProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.7Programming Language :: Python :: 3.8Programming Language :: Python :: 3.9Topic :: Internet :: WWW/HTTPTopic :: Software Development :: Libraries :: Python ModulesTopic :: Utilities |
Evidence: icrawler-0.6.10-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “web crawler framework”
- icrawlericrawler is a lightweight, modular web crawler framework that…
- scraplingScrapling is a web scraping and crawling framework that handles…
- crawleeCrawlee is a web scraping and browser automation library that handles…
Give your agent the search over MCP, or paste the wish link into any chat.
More Python Modules packages
Converts domain names between Unicode and ASCII-compatible encoding (Punycode) according to IDNA 2008 and Unicode Technical Standard 46, with security validation and broader script coverage than the standard library.
Install it if you work with internationalized domain names, need to validate domains, or use HTTP clients that depend on it transitively.
Setuptools is a Python build backend and package management tool that handles building, distributing, and installing Python packages, including support for C/C++ extension modules.
PyYAML parses and emits YAML 1.1 data format, enabling serialization and deserialization of configuration files and Python objects to and from human-readable YAML text.
Pydantic validates Python data structures against type hints, coercing and checking input at runtime to ensure it matches a declared schema.
Provides reusable metadata objects for use with PEP-593 `typing.Annotated` to express common constraints like bounds, collection sizes, and predicates on types.
Install it if you use or build libraries that need to express type constraints in a standardized, inspectable way—or if you want to annotate your own types with…
Provides runtime tools to inspect and introspect Python type annotations, enabling programmatic examination of type hints at execution time.
See also crawlee · gallery-dl · crawlerdetect · Crawl4AI · instaloader · Scrapy · DrissionGet · django-robots · DownloadKit · youtube_dl