{"categories":[{"label":"WWW/HTTP","url":"https://skillfed.io/packages/category/internet-www-http/6"}],"enrichment":{"capability":"Identifies bots, crawlers, and spiders by analyzing user agents and HTTP headers against a pattern database of 1462 crawler signatures.","skillfed_tags":["bot-detection","user-agent-parsing"],"use_cases":["Filter crawler traffic from web analytics to isolate genuine user sessions.","Implement rate limiting or access restrictions for detected bots in web applications.","Identify and block malicious or unwanted crawlers at the request handler level.","Log and monitor which crawlers are accessing your site and how frequently.","Serve different content or responses to crawlers versus human visitors."],"what_it_does":"CrawlerDetect is a Python wrapper around a web crawler detection library that identifies bots, crawlers, and spiders by matching user agents and HTTP headers against a curated pattern database. It exposes two main methods: `isCrawler()` to check if a given user agent is a known crawler, and `getMatches()` to retrieve the name of any detected crawler. The library can accept a user agent string directly, initialize with a pre-set user agent, or analyze HTTP headers passed as a dictionary.\n\nThe package maintains zero runtime dependencies and is kept in sync with upstream crawler patterns. It's useful for web applications that need to distinguish legitimate traffic from automated crawlers\u2014for analytics filtering, rate limiting, or access control. The detection is pattern-based rather than behavioral, so it relies on the completeness and currency of its signature database.","worth_installing":"Yes. The package is actively maintained, has no dependencies, carries a permissive MIT license, and solves a common web application need with a straightforward API. The 1462 crawler patterns provide broad coverage. Install if you need reliable bot detection; the low friction and clean maintenance history make it a safe choice."},"id":"crawlerdetect","links":{"html":"https://skillfed.io/packages/crawlerdetect","md":"https://skillfed.io/packages/crawlerdetect.md","pypi":"https://pypi.org/project/crawlerdetect/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-07-30","license_spdx":"MIT","license_treatment":"permissive","name":"crawlerdetect","python_support":"supports_current","summary":"CrawlerDetect is a Python library designed to identify bots, crawlers, and spiders by analyzing their user agents."},"popularity":{"monthly_downloads":136867,"position":11384,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.4.2"}