morefs

What it is and what it does

morefs extends fsspec with four specialized filesystem implementations. AsyncLocalFileSystem wraps local file I/O with async/await support while retaining sync methods. DictFileSystem and MemFS are in-memory backends optimized for speed—the former uses nested dictionaries to mimic a real filesystem structure, the latter uses tries for flat bucket-like storage. OverlayFileSystem lets you stack multiple fsspec backends (local, S3, etc.) so reads check all layers in order while writes target a primary layer.

The package is a thin, dependency-light layer on fsspec that assumes you're already using fsspec for filesystem abstraction. It's useful when you need async local I/O, fast ephemeral storage, or the ability to transparently read from multiple backends without managing them separately. All filesystems follow fsspec's interface, so they integrate with tools that accept fsspec URLs or filesystem objects.

Use it for:

• Async file operations in concurrent applications without blocking the event loop.
• Fast in-memory caching or temporary storage during data processing pipelines.
• Transparent fallback reads across local, S3, and other fsspec backends in a single filesystem.
• Testing code that uses fsspec by swapping real filesystems with in-memory ones.
• Layered storage where recent writes go to local disk but reads check cloud storage first.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

morefs provides fsspec-based filesystem implementations including async local files, in-memory storage via dictionaries or tries, and overlay filesystems that combine multiple backends.

Yes, if you use fsspec and need async local I/O, fast in-memory storage, or multi-backend overlays. The package is actively maintained, has no known vulnerabilities, installs cleanly, and is permissively licensed. Start with the base install; add extras only if you use AsyncLocalFileSystem or MemFS.

Install

morefs on PyPI

pip install morefs

uv add morefs

poetry add morefs

Installing morefs

Before you install

Low friction: pure Python wheel with a single runtime dependency on fsspec. Actively maintained with recent commits and supports modern Python versions (3.8–3.13).

License in practice

Apache 2.0 is permissive; you can use, modify, and distribute morefs freely in commercial and private projects with minimal restrictions.

Quickstart

pip install morefs

from morefs.asyn_local import AsyncLocalFileSystem
import asyncio

async def main():
    fs = AsyncLocalFileSystem(auto_mkdir=False)
    f = await fs.open_async("foo", mode="w")
    async with f:
        await f.write("foobar")

asyncio.run(main())

AsyncLocalFileSystem requires the aiofile dependency; install with `pip install morefs[asynclocal]` to use it. MemFS requires pygtrie; install with `pip install morefs[memfs]`.

Verify before relying

• Whether in-memory filesystems (DictFileSystem, MemFS) are significantly faster than fsspec's MemoryFileSystem in practice.
• Whether OverlayFileSystem's read/write layering strategy is suitable for production workloads with concurrent access.
• Maturity and stability of AsyncLocalFileSystem for high-concurrency scenarios.

Package facts

Evidence: morefs-0.2.2-py3-none-any.whl

Tags

More Software Development packages