nsj-queue-lib

What it is and what it does

nsj-queue-lib is a Python library that implements task queues and worker patterns using a relational database as the queue backend instead of a separate message broker like RabbitMQ. It supports three worker types: simple queuing (one message, one worker), pub/sub (one message replicated to multiple subscribers with independent processing), and webhooks (pub/sub without code implementation, configured via subscriptions). The core design ensures transactional consistency—messages are enqueued atomically with the database transaction that created them, and dequeued only after successful processing, eliminating the need for complex protocols like two-phase commit.

The trade-off is that all queue data and payloads live in the database itself, increasing storage and compute load on the database server. The library is designed for Postgres, MySQL, and Singlestore only. It includes utilities to generate queue schemas, a client module for inserting tasks, and base classes to extend for custom worker logic. Workers typically run as separate processes or containers with environment variables specifying the queue table and database connection.

Use it for:

• Implement asynchronous task processing where transactional consistency with the originating database transaction is critical.
• Monitor queue depth and task status directly via SQL queries using BI tools like Redash or DundasBI.
• Build pub/sub workflows where a single event triggers independent processing by multiple subscribers.
• Deploy webhook handlers that call external services without writing custom subscription code.
• Avoid the operational complexity of running a separate message broker when your application already depends on a relational database.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Implements transactional queues and workers backed by a relational database (Postgres, MySQL, or Singlestore), supporting simple queuing, pub/sub, and webhook patterns without requiring a separate message broker.

Yes, with conditions. Install if you need transactional queue guarantees tightly coupled to database transactions and are willing to accept higher database load in exchange for simpler architecture and direct SQL monitoring. Do not install if your queue traffic is very high, you need a separate broker for isolation, or you cannot verify the license terms and internal dependency availability. The aging maintenance status (397 days since last release, Alpha stage) suggests limited active development; evaluate whether the package is still maintained for your target database versions.

Install

nsj-queue-lib on PyPI

pip install nsj-queue-lib

uv add nsj-queue-lib

poetry add nsj-queue-lib

Installing nsj-queue-lib

Before you install

Low install friction; pure Python wheel. Maintenance status is aging (397 days since last release), though the package remains in Alpha development stage. Runtime dependencies include psycopg2_binary, python-logging-loki, pytest, and two internal Nasajon libraries (nsj_sql_utils_lib, nsj_gcf_utils), which may introduce additional operational considerations.

License in practice

License treatment is unclear—no SPDX identifier or raw license text is available in the package metadata. Verify the actual license terms in the repository before adopting this package in a commercial or restricted-license context.

Quickstart

pip install nsj-queue-lib

from nsj_queue_lib.worker_base import WorkerBase

class MyWorker(WorkerBase):
    def execute(self, payload):
        # Process task
        pass

if __name__ == "__main__":
    MyWorker().run()

Requires Postgres, MySQL, or Singlestore database with a queue table created via the db_gen utility or manual schema setup. Environment variables must be configured to connect to the database and specify the queue table name.

Verify before relying

• Whether the two internal dependencies (nsj_sql_utils_lib, nsj_gcf_utils) are publicly available or require special access.
• Current maintenance timeline and whether the aging status reflects active use or abandonment.
• Performance characteristics and scalability limits when queue payload or message volume is high.
• Compatibility and testing status with modern versions of Postgres, MySQL, and Singlestore.

Package facts

Evidence: nsj_queue_lib-1.2.2-py3-none-any.whl

Tags

More Libraries packages