lakefs

What it is and what it does

lakefs is a Python wrapper that sits on top of the lower-level lakefs-sdk, providing a more intuitive object-oriented interface for working with lakeFS repositories. It abstracts away configuration boilerplate by inferring credentials from environment variables or configuration files, and offers higher-level operations like branching, committing, and diffing that mirror Git-like workflows but operate on data lake objects.

The package is designed for developers who want to programmatically manage versioned data repositories without wrestling with low-level SDK details. You work with Repository, Branch, and Reference objects to perform common tasks: listing objects, reading and writing files, creating commits, computing diffs between branches, and traversing version history using ref expressions similar to Git's revision syntax.

Use it for:

• Automate data pipeline workflows that need to version control datasets across multiple branches and commits.
• Search or process objects stored in specific lakeFS references without manual credential management.
• Programmatically upload datasets and commit them with descriptive messages in a CI/CD pipeline.
• Compare data changes between branches to detect drift or validate transformations.
• Build Python applications that treat a lakeFS repository as a versioned data source.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A high-level Python wrapper around the lakeFS SDK that simplifies interaction with lakeFS repositories, branches, and objects through a cleaner API with environment-based credential inference.

Yes. The package is actively maintained, has no known vulnerabilities, carries a permissive Apache-2.0 license, and installs with minimal friction. It is a sensible choice if you are already using lakeFS and want a Pythonic, higher-level interface rather than working directly with the lower-level SDK.

Install

lakefs on PyPI

pip install lakefs

uv add lakefs

poetry add lakefs

Installing lakefs

Before you install

Low friction installation with only two runtime dependencies (lakefs-sdk and pyyaml). The package is actively maintained with a recent release and no known vulnerabilities.

License in practice

Apache-2.0 permissive license allows use in commercial and private projects with minimal restrictions; you must include a copy of the license and state significant changes.

Quickstart

pip install lakefs

import lakefs
repo = lakefs.repository(repository_id="my-repo")
branch = repo.branch("main")
for obj in branch.objects():
    print(f"{obj.path}: {obj.size_bytes}")

Requires Python 3.10 or later; requires a running lakeFS server and valid credentials (via environment variables, .lakectl.yaml, or explicit Client initialization).

Verify before relying

• Whether the package handles concurrent operations safely or has thread-safety guarantees.
• Performance characteristics when working with large numbers of objects or branches.
• Whether all lakeFS server versions are supported or if there are compatibility constraints.

Package facts

Evidence: lakefs-0.16.0-py3-none-any.whl

Keywords: OpenAPI, OpenAPI-Generator, lakeFS API, Python Wrapper

Tags

More Python Modules packages