django-robots
Robots exclusion application for Django, complementing Sitemaps.
Decision gist · record as of 2026-08-14
Yes. This is a stable, actively maintained package with zero security vulnerabilities, low install friction, and permissive licensing. It solves a real SEO and crawler-management need for Django sites with no external dependencies. Install it if your site needs to manage robots.txt rules beyond a static file.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Django 2.2 or later and Python 3.7 or later; must be added to INSTALLED_APPS and configured with database models.
- Low friction install with no runtime dependencies.
- Actively maintained with recent commits and a stable release history since 2008.
License · maintenance · safety
permissive license (permissive) — Permissive BSD license; you can use, modify, and distribute this package with minimal restrictions in commercial or open-source projects.
last release 2023-09-07 (1072 days) · last repo commit 2026-08-10 · 475 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 320,496 downloads/mo, #7,636 on PyPI
Alternatives
Verify before relying
pip install django-robots==6.1
# In Django settings.py, add to INSTALLED_APPS:
# 'robots'
# Then define robots rules in the Django admin or database.- Whether the package requires explicit URL configuration beyond INSTALLED_APPS setup.
- How robots.txt rules are persisted (database models vs. static configuration).
- Integration requirements with Django's Sitemap contrib app mentioned in the description.
What it is and what it does
Django-robots is a Django application that handles the generation and management of robots.txt files, the standard mechanism web servers use to communicate crawler access policies to search engines and other bots. It implements the robots exclusion protocol and complements Django's built-in Sitemap contrib app, allowing you to define which crawlers can access which URL patterns on your site.
The package has been maintained since 2008 and is production-stable. It integrates directly into Django's admin interface for rule management and works with Django versions 2.2 through 4.2. With no external runtime dependencies and a straightforward installation process, it's designed to be a lightweight addition to existing Django projects that need fine-grained crawler access control.
Use it for
- Prevent search engines from crawling private or duplicate content areas of your site.
- Allow or disallow specific bots (like Googlebot or Bingbot) from accessing certain URL patterns.
- Manage crawler traffic to reduce server load by blocking aggressive or unnecessary bots.
- Coordinate with your Sitemap to guide search engines to important pages while blocking others.
- Enforce SEO policies by controlling which pages are indexed by different search engines.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes.
This is a stable, actively maintained package with zero security vulnerabilities, low install friction, and permissive licensing. It solves a real SEO and crawler-management need for Django sites with no external dependencies. Install it if your site needs to manage robots.txt rules beyond a static file.
Install
django-robots on PyPI
Before you install
Low friction install with no runtime dependencies. Actively maintained with recent commits and a stable release history since 2008. Supports current Django versions (2.2 through 4.2) and modern Python (3.7+).
Requires Django 2.2 or later and Python 3.7 or later; must be added to INSTALLED_APPS and configured with database models.
License in practice
Permissive BSD license; you can use, modify, and distribute this package with minimal restrictions in commercial or open-source projects.
Quickstart
pip install django-robots==6.1
# In Django settings.py, add to INSTALLED_APPS:
# 'robots'
# Then define robots rules in the Django admin or database.Verify before relying
- Whether the package requires explicit URL configuration beyond INSTALLED_APPS setup.
- How robots.txt rules are persisted (database models vs. static configuration).
- Integration requirements with Django's Sitemap contrib app mentioned in the description.
Package facts
| License | permissive license permissive |
| Python support | Supports the current Python release >=3.7 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | None |
| Maintenance | Actively maintained 1,072 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 320,496 / month, #7,636 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 5 - Production/StableEnvironment :: Web EnvironmentFramework :: DjangoFramework :: Django :: 2.2Framework :: Django :: 3.1Framework :: Django :: 3.2Framework :: Django :: 4.0Framework :: Django :: 4.1Framework :: Django :: 4.2Intended Audience :: DevelopersLicense :: OSI Approved :: BSD LicenseOperating System :: OS IndependentProgramming Language :: PythonProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.7Programming Language :: Python :: 3.8Programming Language :: Python :: 3.9Topic :: Internet :: WWW/HTTP :: Dynamic ContentTopic :: Software DevelopmentTopic :: Software Development :: Libraries :: Application Frameworks |
Evidence: django_robots-6.1-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “django robots.txt management”
- django-robotsA Django application that manages robots.txt files according to the…
- ProtegoProtego parses robots.txt files and determines whether a given URL…
- robotspyParses robots.txt files according to RFC 9309 and determines whether…
Give your agent the search over MCP, or paste the wish link into any chat.
More Software Development packages
Provides backported and experimental type hints for Python 3.9+, allowing use of newer typing features on older Python versions and enabling early experimentation with type system PEPs before they enter the standard library.
NumPy provides an N-dimensional array object and a comprehensive suite of mathematical, linear algebra, Fourier transform, and random number functions for scientific computing in Python.
FastAPI is a Python web framework for building REST APIs using type hints, with automatic request validation, serialization, and interactive API documentation.
Provides a way to document function parameters, class attributes, return types, and variables inline using Python's `Annotated` type hint syntax instead of traditional docstrings.
Typer builds command-line applications from Python functions using type hints, automatically generating help text, argument parsing, and shell completion.
Install it if you are building CLIs in Python.
Distlib provides low-level packaging utilities for building, distributing, and managing Python software—including metadata handling, version specifiers, wheel support, script installation, and dependency resolution.
See also robotspy · Protego · django-decorator-include · crawlerdetect · crawlee · dj-materialized-views · django-push-notifications · bridgekeeper · ultimate-sitemap-parser · django-safedelete