{"categories":[{"label":"Scientific/Engineering","url":"https://skillfed.io/packages/category/scientific-engineering/6"},{"label":"Database","url":"https://skillfed.io/packages/category/database/4"},{"label":"File Formats","url":"https://skillfed.io/packages/category/file-formats"}],"enrichment":{"capability":"Vortex-data provides Python bindings to work with Vortex, a columnar file format optimized for high-performance data processing with Apache Arrow compatibility and pluggable compression encodings.","skillfed_tags":["columnar-storage","arrow-compatible","compression"],"use_cases":["Store and query large columnar datasets in object storage with faster random access than Parquet.","Build data pipelines that need efficient compression without sacrificing read performance.","Integrate Vortex files into Arrow-based analytics workflows (DataFusion, DuckDB, Pandas).","Experiment with alternative encoding strategies for specialized data types or access patterns.","Benchmark columnar formats in performance-critical data systems."],"what_it_does":"Vortex-data is a Python interface to Vortex, a next-generation columnar storage format designed for object-storage-backed data systems. It separates logical schema from physical encoding, allowing pluggable compression strategies (RLE, dictionary, and others) while maintaining zero-copy compatibility with Apache Arrow. The package lets you read, write, and manipulate Vortex files from Python, integrating with the broader Arrow ecosystem.\n\nThe format is optimized for random access and scan performance on wide tables with efficient metadata handling. It comes with built-in encodings compatible with Arrow's memory layout and supports cascading compression schemes. The file format itself is considered stable from version 0.36.0 onwards, though the Python library APIs may evolve. Dependencies include pyarrow, substrait, and typing-extensions.","worth_installing":"Yes, if you work with large columnar datasets and need faster random access or scan performance than Parquet, or if you want to experiment with pluggable compression encodings in an Arrow-compatible format. The package is actively maintained, has no known vulnerabilities, and uses a permissive license. Install friction is moderate due to compiled wheels, but pre-built binaries are available for common platforms. Not necessary if you're already satisfied with Parquet or don't need the performance gains."},"id":"vortex-data","links":{"html":"https://skillfed.io/packages/vortex-data","md":"https://skillfed.io/packages/vortex-data.md","pypi":"https://pypi.org/project/vortex-data/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-07","license_spdx":null,"license_treatment":"permissive","name":"vortex-data","python_support":"supports_current","summary":"Python bindings for Vortex, an Apache Arrow-compatible toolkit for working with compressed array data."},"popularity":{"monthly_downloads":203397,"position":9630,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.84.0"}