{"categories":[{"label":"Python Modules","url":"https://skillfed.io/packages/category/software-development-libraries-python-modules/12"},{"label":"Internet","url":"https://skillfed.io/packages/category/internet/3"},{"label":"WWW/HTTP","url":"https://skillfed.io/packages/category/internet-www-http/4"},{"label":"Indexing/Search","url":"https://skillfed.io/packages/category/internet-www-http-indexing-search"}],"enrichment":{"capability":"Python SDK for the Spider Cloud API that enables website scraping, crawling, link extraction, screenshot capture, and LLM-compatible data collection through a managed cloud service.","skillfed_tags":["web-scraping","api-client","llm-data"],"use_cases":["Scrape product listings or pricing data from e-commerce sites and format for LLM ingestion.","Crawl documentation sites or knowledge bases to build training datasets for AI models.","Extract all links from a website for SEO analysis or site mapping.","Bypass anti-bot protections on protected sites using the unblocker feature with AI extraction.","Automate screenshot capture of web pages for visual monitoring or testing.","Stream large crawl results in chunks to handle websites with hundreds or thousands of pages."],"what_it_does":"spider-client is a Python SDK that wraps the Spider Cloud API, a managed web scraping and crawling service. It provides methods to scrape individual URLs, crawl entire websites, extract links, capture screenshots, transform HTML to markdown, and bypass anti-bot protections. The SDK also includes AI-powered methods (requiring a separate subscription) for guided scraping and crawling using natural language prompts, plus an Unlimited plan tier that bills by concurrency seats rather than per-request credits.\n\nThe package handles the HTTP communication with Spider Cloud's backend, manages streaming responses, supports multiple output formats (JSON, CSV, XML, JSONL), and formats scraped content for compatibility with language models. It relies on requests for HTTP calls, ijson for streaming JSON parsing, tenacity for retry logic, aiohttp for async operations, and spider-browser for browser automation features.","worth_installing":"Yes, if you need a managed web scraping solution with built-in anti-bot handling and LLM-ready output formatting. The SDK is actively maintained, has no known vulnerabilities, and low install friction. However, it requires a Spider Cloud account and API key\u2014this is a client library for a paid service, not a standalone scraper. Suitable for production use when the service cost aligns with your scraping volume."},"id":"spider-client","links":{"html":"https://skillfed.io/packages/spider-client","md":"https://skillfed.io/packages/spider-client.md","pypi":"https://pypi.org/project/spider-client/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-07-18","license_spdx":null,"license_treatment":"permissive","name":"spider-client","python_support":"unspecified","summary":"Python SDK for Spider Cloud API"},"popularity":{"monthly_downloads":367559,"position":7195,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.1.93"}