MechanicalSoup
A Python library for automating interaction with websites
Decision gist · record as of 2026-08-14
Yes. MechanicalSoup is a solid choice for web automation and scraping when you need form submission and stateful interaction but don't require JavaScript execution. It has low install friction, active maintenance, no known vulnerabilities, and a permissive license. The five runtime dependencies are all widely used and stable. Install it if your target sites serve complete HTML and you want a lighter alternative to a full browser automation framework.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.9 or later.
- Does not handle JavaScript-rendered content; pages must be fully formed in the initial HTML response.
- Low friction: pure Python wheel with five common runtime dependencies (requests, beautifulsoup4, lxml, certifi, urllib3).
License · maintenance · safety
MIT (permissive) — MIT license is permissive—you can use, modify, and distribute MechanicalSoup freely in commercial or private projects with minimal restrictions, only requiring attribution.
last release 2025-05-30 (441 days) · last repo commit 2026-08-04 · 4,888 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 164,953 downloads/mo, #10,534 on PyPI
Alternatives
Verify before relying
pip install MechanicalSoup
import mechanicalsoup
browser = mechanicalsoup.StatefulBrowser()
browser.open("https://example.com")
browser.select_form()
browser["field"] = "value"
browser.submit_selected()- Performance characteristics with large-scale scraping or concurrent requests
- Compatibility with modern anti-scraping techniques or rate-limiting headers
What it is and what it does
MechanicalSoup is a Python library that automates interaction with websites by wrapping Requests (for HTTP) and BeautifulSoup (for HTML parsing) into a stateful browser interface. It handles the tedious parts of web automation: managing cookies across requests, following redirects, selecting and filling HTML forms, and navigating links. The library maintains session state automatically, so you can interact with a website as if you were a user clicking through pages and submitting forms.
It's designed for straightforward web scraping and automation tasks where the content is server-rendered HTML—not for sites that rely heavily on JavaScript to generate or modify content. The API is simple and intuitive: open a URL, select a form by CSS selector, fill in fields like a dictionary, and submit. It's a modern Python 3 successor to the older Mechanize library, actively maintained by a small team and tested against both CPython and PyPy3.
Use it for
- Scrape search results or product listings from websites by automating form submission and following result links
- Automate login workflows and session-based data collection from sites requiring authentication
- Extract data from multi-page forms or workflows where you need to fill fields, submit, and parse results across requests
- Test web applications by simulating user interactions like form fills and navigation without a browser driver
- Monitor or archive web content by periodically fetching and parsing pages that require form interaction
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes.
MechanicalSoup is a solid choice for web automation and scraping when you need form submission and stateful interaction but don't require JavaScript execution. It has low install friction, active maintenance, no known vulnerabilities, and a permissive license. The five runtime dependencies are all widely used and stable. Install it if your target sites serve complete HTML and you want a lighter alternative to a full browser automation framework.
Install
mechanicalsoup on PyPI
Before you install
Low friction: pure Python wheel with five common runtime dependencies (requests, beautifulsoup4, lxml, certifi, urllib3). Active maintenance since 2017 with recent commits; last release 441 days ago suggests stable, infrequent updates rather than abandonment.
Requires Python 3.9 or later. Does not handle JavaScript-rendered content; pages must be fully formed in the initial HTML response.
License in practice
MIT license is permissive—you can use, modify, and distribute MechanicalSoup freely in commercial or private projects with minimal restrictions, only requiring attribution.
Quickstart
pip install MechanicalSoup
import mechanicalsoup
browser = mechanicalsoup.StatefulBrowser()
browser.open("https://example.com")
browser.select_form()
browser["field"] = "value"
browser.submit_selected()Verify before relying
- Performance characteristics with large-scale scraping or concurrent requests
- Compatibility with modern anti-scraping techniques or rate-limiting headers
Package facts
| License | MIT permissive |
| Python support | Supports the current Python release >=3.9 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 5 packagesrequestsbeautifulsoup4lxmlcertifiurllib3 |
| Maintenance | Actively maintained 441 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 164,953 / month, #10,534 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | License :: OSI Approved :: MIT LicenseProgramming Language :: Python :: 3Programming Language :: Python :: 3 :: OnlyProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Programming Language :: Python :: 3.9 |
Evidence: mechanicalsoup-1.4.0-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “web scraping with form submission”
- MechanicalSoupMechanicalSoup automates interaction with websites by handling HTTP…
- mechanizeAutomate programmatic interaction with HTTP web servers by simulating…
- browser-use-sdkPython SDK for controlling browsers programmatically through the…
Give your agent the search over MCP, or paste the wish link into any chat.
More WWW/HTTP packages
urllib3 is an HTTP client library that provides thread-safe connection pooling, SSL/TLS verification, multipart file uploads, request retries, compression support, and proxy handling for Python applications.
Requests is a Python HTTP library that simplifies sending HTTP/1.1 requests with automatic handling of headers, authentication, cookies, and response parsing.
h11 is a pure-Python HTTP/1.1 protocol implementation that handles parsing and serializing HTTP messages without any built-in I/O, letting you integrate it with any network layer you choose.
HTTPX is a fully featured HTTP client library for Python that provides both sync and async APIs, with support for HTTP/1.1 and HTTP/2, plus an integrated command-line client.
Install it if you are building new projects or modernizing existing ones that rely on HTTP.
A minimal low-level HTTP client library that sends HTTP requests with thread-safe and task-safe connection pooling, supporting HTTP/1.1, HTTP/2, proxies, and both sync and async interfaces.
aiohttp is an async HTTP client and server framework built on asyncio, supporting both WebSockets and middleware-based routing for building concurrent web applications.
Install it if you need async HTTP client or server capabilities in asyncio-based applications.
See also mechanize · browser-cookie3 · shot-scraper · zope.testbrowser · Scrapy · beautifulsoup4 · pycookiecheat · requestium · cookies · BeautifulSoup