persist-queue

What it is and what it does

persist-queue implements file-based and SQLite-based persistent queues for Python, storing each queued item on disk so it survives process crashes and restarts. It uses pickle for object serialization by default, supporting built-in types like int, dict, and list directly, with support for custom objects via pickle extension methods. The package is thread-safe and works with multi-threaded producers and consumers.

The library offers multiple queue types: basic file-based FIFO queues, SQLite-backed variants (FIFO, FILO, priority, unique-item, and acknowledgment-based), a persistent dictionary, and MySQL-backed queues. Version 1.1.0 adds async versions of file and SQLite queues. It supports multiple serialization formats beyond pickle (msgpack, cbor, json) via optional dependencies, and is compatible with greenlet and eventlet environments.

Use it for:

• Build a task queue that survives application crashes, recovering pending jobs after restart
• Implement a thread-safe work queue for multi-threaded producer-consumer patterns without external services
• Create a priority-based job scheduler that persists work items to disk for reliable processing
• Store unique items in a queue to prevent duplicate processing across restarts
• Use async queues in event-driven applications requiring persistent, crash-resistant message buffering

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides thread-safe, disk-based queue implementations using file storage or SQLite that persist across process restarts and support multiple serialization formats.

Yes, with conditions. Install if you need a lightweight, dependency-free persistent queue for single-machine or multi-threaded scenarios. The permissive BSD license and low install friction make it accessible. However, the aging maintenance status (293 days since last release) and lack of recent activity suggest evaluating whether active maintenance matters for your use case. Not suitable if you require high-availability distributed queuing or need guarantees beyond single-process crash recovery.

Install

persist-queue on PyPI

pip install persist-queue

uv add persist-queue

poetry add persist-queue

Installing persist-queue

Before you install

Low install friction with no runtime dependencies. Maintenance status is aging—last release was 293 days ago, though the repository remains active with a recent commit on 2026-01-08 and 390 stars.

License in practice

Licensed under BSD (permissive), allowing use in commercial and proprietary projects with minimal restrictions.

Quickstart

pip install persist-queue

from persistqueue import Queue
q = Queue("my_queue_path")
q.put("item1")
item = q.get()
q.task_done()

Verify before relying

• Whether pickle-based serialization is suitable for your object types without custom extension code
• Performance characteristics under high-throughput or large-scale concurrent access patterns
• Compatibility guarantees with greenlet/eventlet environments mentioned in the description

Package facts

Evidence: persist_queue-1.1.0-py3-none-any.whl

Tags

More Libraries packages