---
id: django-robots
version: "6.1"
license: unclear
license_treatment: permissive
maintenance: active
---

# django-robots — Robots exclusion application for Django, complementing Sitemaps.

License: permissive · Maintenance: active · Downloads: 320.5K/mo

## What it is and what it does

Django-robots is a Django application that handles the generation and management of robots.txt files, the standard mechanism web servers use to communicate crawler access policies to search engines and other bots. It implements the robots exclusion protocol and complements Django's built-in Sitemap contrib app, allowing you to define which crawlers can access which URL patterns on your site.

The package has been maintained since 2008 and is production-stable. It integrates directly into Django's admin interface for rule management and works with Django versions 2.2 through 4.2. With no external runtime dependencies and a straightforward installation process, it's designed to be a lightweight addition to existing Django projects that need fine-grained crawler access control.

Use it for:

- Prevent search engines from crawling private or duplicate content areas of your site.
- Allow or disallow specific bots (like Googlebot or Bingbot) from accessing certain URL patterns.
- Manage crawler traffic to reduce server load by blocking aggressive or unnecessary bots.
- Coordinate with your Sitemap to guide search engines to important pages while blocking others.
- Enforce SEO policies by controlling which pages are indexed by different search engines.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

A Django application that manages robots.txt files according to the robots exclusion protocol, allowing you to configure which web crawlers can access which parts of your site.

Yes. This is a stable, actively maintained package with zero security vulnerabilities, low install friction, and permissive licensing. It solves a real SEO and crawler-management need for Django sites with no external dependencies. Install it if your site needs to manage robots.txt rules beyond a static file.

## Install

pip install django-robots
uv add django-robots
poetry add django-robots

## Installing django-robots

Before you install: Low friction install with no runtime dependencies. Actively maintained with recent commits and a stable release history since 2008. Supports current Django versions (2.2 through 4.2) and modern Python (3.7+).

License in practice: Permissive BSD license; you can use, modify, and distribute this package with minimal restrictions in commercial or open-source projects.

Quickstart:

pip install django-robots==6.1

# In Django settings.py, add to INSTALLED_APPS:
# 'robots'

# Then define robots rules in the Django admin or database.

Requires Django 2.2 or later and Python 3.7 or later; must be added to INSTALLED_APPS and configured with database models.

Verify before relying:

- Whether the package requires explicit URL configuration beyond INSTALLED_APPS setup.
- How robots.txt rules are persisted (database models vs. static configuration).
- Integration requirements with Django's Sitemap contrib app mentioned in the description.

## Package facts

- License: not declared (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 320.5K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

django robots.txt management, robots exclusion protocol django, web crawler access control, django sitemap robots, search engine crawler rules, django-plugin, seo, web-standards

[View on SkillFed](https://skillfed.io/packages/django-robots) · [View on PyPI](https://pypi.org/project/django-robots/)