snscrape

What it is and what it does

snscrape is a multi-platform social network scraper that extracts posts, profiles, hashtags, and search results from eight services: Twitter, Instagram, Facebook, Reddit, Mastodon, Telegram, VKontakte, and Weibo. It works both as a command-line tool (outputting URLs or JSONL) and as a Python library, though library usage is undocumented. The tool depends on requests, lxml, beautifulsoup4, filelock, and pytz to handle HTTP requests, HTML parsing, and timezone logic.

The project is in Beta status and has been dormant since mid-2023—the last release was June 2023 and the last commit November 2023. While it has no known security vulnerabilities and low install friction, the lack of recent maintenance means scrapers may break as target platforms change their page layouts or APIs. It is licensed under GPLv3+, requiring any derivative work to remain open-source.

Use it for:

• Archive Twitter threads or user timelines for research, journalism, or personal record-keeping.
• Collect hashtag posts from Instagram or Twitter for sentiment analysis or trend monitoring.
• Scrape Reddit discussions or subreddit posts for data analysis or community research.
• Gather user profile information or public posts from multiple platforms in bulk.
• Export social media data to JSONL format for downstream processing or database ingestion.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Scrapes user profiles, posts, hashtags, and searches from multiple social networks (Twitter, Instagram, Facebook, Reddit, Mastodon, Telegram, VKontakte, Weibo) via CLI or Python library, returning URLs or structured data.

Yes, if you need to scrape social media data and can tolerate dormant maintenance. The low install friction and lack of vulnerabilities make it safe to try. However, be prepared for scrapers to break without warning as platforms change; test against your target service first and monitor the GitHub repository for community fixes. Not suitable for production systems requiring active support.

Install

snscrape on PyPI

pip install snscrape

uv add snscrape

poetry add snscrape

Installing snscrape

Before you install

Low install friction with standard web-scraping dependencies. Maintenance is dormant—last release was June 2023 and last commit November 2023—so expect no active bug fixes or feature updates, though the repository remains public and unarchived.

License in practice

Licensed under GPLv3+, a copyleft license. Any derivative work or distribution must also be open-source under compatible terms; proprietary use or closed-source redistribution is not permitted.

Quickstart

pip install snscrape

# CLI: collect tweets by a user
snscrape twitter-user textfiles

# Or with options
snscrape --max-results 100 --jsonl twitter-hashtag archiveteam

Requires Python 3.8 or higher; lxml dependency also requires libxml2 and libxslt system libraries to be installed.

Verify before relying

• Whether scrapers remain functional given social networks' frequent API and page-structure changes since last update in June 2023.
• Whether library-mode usage (undocumented per the description) is stable or has breaking changes.
• Current rate-limiting or blocking behavior from target platforms.

Package facts

Evidence: snscrape-0.7.0.20230622-py3-none-any.whl

Tags

More WWW/HTTP packages