pylance

What it is and what it does

Pylance is a Python SDK that wraps the Lance columnar data format, a format designed for efficient storage and retrieval of large analytical datasets. It sits on top of Apache Arrow, inheriting Arrow's type system and columnar layout while providing a Python-friendly interface for reading and writing data. The package targets data science and machine learning workflows where columnar storage offers advantages in compression, query performance, and memory efficiency.

The package is in active development (Alpha status) and requires Python 3.10 or later with compiled wheels available for common platforms. Its main dependencies—pyarrow, numpy, and lance-namespace—are typical for data science stacks. With zero known vulnerabilities and a permissive license, it presents a low security risk, though as an Alpha project it may still undergo API changes.

Use it for:

• Store large machine learning training datasets in a columnar format optimized for batch access and feature extraction.
• Build data pipelines that serialize analytical tables to disk with better compression than row-oriented formats.
• Integrate with pyarrow-based workflows to leverage Arrow's ecosystem while using Lance's storage optimizations.
• Archive and query time-series or event data where columnar layout reduces I/O and memory overhead.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Pylance is a Python wrapper for the Lance columnar data format, enabling efficient storage and querying of large datasets using Apache Arrow as its underlying data model.

Yes, if you work with large analytical datasets and want a columnar storage format integrated with the pyarrow ecosystem. The permissive Apache license, active maintenance, and zero known vulnerabilities support adoption. Medium install friction (compiled wheels, multiple dependencies) is typical for data science packages. Be aware that Alpha status means the API may change; suitable for production use only if you can tolerate potential breaking changes in future releases.

Install

pylance on PyPI

pip install pylance

uv add pylance

poetry add pylance

Installing pylance

Before you install

Medium install friction due to compiled wheels for multiple platforms (arm64, x86_64, manylinux variants). Active maintenance with a release 7 days ago. Requires Python 3.10 or later and depends on pyarrow and numpy, which are themselves substantial compiled packages.

License in practice

Licensed under Apache License 2.0 (permissive), allowing commercial and private use with minimal restrictions—suitable for most production and research contexts.

Quickstart

pip install pylance

import lance

# Create a Lance dataset from a PyArrow table or pandas DataFrame
data = lance.write_table(table, uri="./my_dataset.lance")

Requires Python 3.10 or later; pyarrow and numpy must be installed (typically handled by pip automatically).

Verify before relying

• Specific performance characteristics (latency, throughput) compared to other columnar formats.
• Whether the package supports streaming or incremental writes beyond the basic write_table API.
• Current state of documentation and examples beyond the contribution guide reference.
• Actual download volume and adoption metrics beyond position in the top PyPI packages list.

Package facts

Evidence: pylance-10.0.0-cp310-abi3-macosx_11_0_arm64.whl; pylance-10.0.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl; pylance-10.0.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; pylance-10.0.0-cp310-abi3-manylinux_2_28_aarch64.whl; pylance-10.0.0-cp310-abi3-manylinux_2_28_x86_64.whl; pylance-10.0.0-cp310-abi3-win_amd64.whl

Keywords: data-format, data-science, machine-learning, arrow, data-analytics

Tags

More Scientific/Engineering packages