dirsql

What it is and what it does

dirsql is a Python SDK that creates an ephemeral SQL interface over a filesystem directory. It watches for file changes, ingests structured files (JSON, etc.) into an in-memory SQLite database according to user-defined schemas and glob patterns, and exposes standard SQL queries. The filesystem remains the source of truth; the database is rebuilt from files on startup and kept in sync via background file watching.

You define tables by specifying DDL, a glob pattern to select files, and an extraction function that converts matched files into row dicts. dirsql handles the scanning and indexing asynchronously; you await db.ready() before querying. It supports multiple tables, joins, SQLite extensions, and an async iterator for row-level change events (insert, update, delete). A CLI tool is also included for one-off queries and an HTTP server mode.

Use it for:

• Query a blog directory (posts and authors as JSON files) with SQL joins across tables
• Monitor a config directory for changes and react to inserts/updates/deletes via async events
• Index structured logs or data files and run ad-hoc SQL analysis without a separate database
• Load SQLite extensions and use them in queries over filesystem data
• Expose filesystem data over HTTP via the built-in server for remote SQL access

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Watches a local directory, indexes structured files into an in-memory SQLite database, and exposes a SQL query interface with real-time change tracking.

Yes, if you need to query structured files in a directory with SQL and want real-time change tracking. The native extension adds medium install friction but provides prebuilt wheels for common platforms. MIT license and active maintenance are favorable. No known vulnerabilities. Suitable for development, data analysis, and small-to-medium workloads.

Install

dirsql on PyPI

pip install dirsql

uv add dirsql

poetry add dirsql

Installing dirsql

Before you install

Medium install friction due to native Rust extension; prebuilt wheels cover common platforms (macOS x86_64/arm64, Linux x86_64/aarch64, Windows x64). Requires Python >= 3.10. Active maintenance with recent releases.

License in practice

MIT license permits commercial and private use with minimal restrictions; suitable for most projects.

Quickstart

pip install dirsql

import asyncio
from dirsql import DirSQL, Table

async def main():
    db = DirSQL("./my-blog", tables=[Table(ddl="CREATE TABLE posts (title TEXT, author TEXT)", glob="posts/*.json", extract=lambda path: [__import__('json').loads(open(path).read())])])
    await db.ready()
    posts = await db.query("SELECT * FROM posts WHERE author = 'alice'")
    print(posts)

asyncio.run(main())

Requires Python >= 3.10. Ships as native Rust extension; prebuilt wheels provided for macOS (x86_64, arm64), Linux (x86_64, aarch64), and Windows (x64).

Verify before relying

• Performance characteristics for large directories or complex joins not specified
• Memory overhead for in-memory SQLite index relative to directory size not quantified
• Behavior when files are deleted or moved during active queries not documented

Package facts

Evidence: dirsql-0.4.20-cp310-abi3-macosx_10_12_x86_64.whl; dirsql-0.4.20-cp310-abi3-macosx_11_0_arm64.whl; dirsql-0.4.20-cp310-abi3-manylinux_2_39_aarch64.whl; dirsql-0.4.20-cp310-abi3-manylinux_2_39_x86_64.whl; dirsql-0.4.20-cp310-abi3-win_amd64.whl

Keywords: sql, filesystem, directory, sqlite, index

Tags

More Database packages