vineyard-bdist
An in-memory immutable data manager
Decision gist · record as of 2026-08-14
Yes, if you are building distributed data pipelines that exchange data between multiple big data frameworks (Mars, PyTorch, GraphScope, etc.) and want to eliminate serialization and IO overhead. The package is production-stable, permissively licensed, and actively maintained. Install friction is moderate due to platform-specific binaries, but the zero-copy sharing benefit justifies it for large-scale workloads. Not necessary for single-machine or simple data-passing scenarios.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires a running Vineyard server instance and an IPC socket path; binary wheels are platform-specific (Linux x86_64 or aarch64).
- Medium install friction due to compiled binary wheels for specific platforms (manylinux2014 aarch64 and x86_64).
- The package is marked as aging with last commit on 2026-01-22, but remains actively maintained and marked Production/Stable.
License · maintenance · safety
Apache License 2.0 (permissive) — Licensed under Apache License 2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions—suitable for most production and research contexts.
last release 2024-08-29 (715 days) · last repo commit 2026-01-22 · 962 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 102,682 downloads/mo, #12,854 on PyPI
Alternatives
Verify before relying
pip install vineyard-bdist
import vineyard
client = vineyard.connect(vineyard_ipc_socket)
data = client.get(vineyard_object_id)- Whether vineyard-bdist is the correct distribution for your use case versus the main vineyard package
- Specific performance gains or latency improvements in your distributed workload
- Compatibility with your particular big data framework (Mars, GraphScope, PyTorch, etc.)
What it is and what it does
Vineyard is an in-memory immutable data manager designed to solve the data-sharing bottleneck in distributed big data pipelines. It provides zero-copy sharing of complex data structures (tensors, dataframes, graphs) across systems like Mars, PyTorch, and GraphScope by leveraging shared memory, eliminating the serialization, deserialization, and IO costs that typically occur when exchanging data between different frameworks in a distributed environment.
The package offers out-of-the-box high-level data abstractions and stream pipelining capabilities, allowing jobs to read and write data chunks without waiting for all preceding results. It is positioned as a CNCF sandbox project and supports Python 3.6 through 3.11 on POSIX systems and macOS. The binary distribution (vineyard-bdist) provides pre-compiled wheels for Linux platforms, reducing installation complexity compared to building from source.
Use it for
- Preprocessing large datasets with Mars and training models with PyTorch without intermediate file I/O
- Sharing distributed graphs between GraphScope and downstream analytics or ML systems
- Building data pipelines where multiple frameworks need to exchange tensors or dataframes with minimal overhead
- Reducing memory consumption and latency in multi-stage distributed computations via stream pipelining
- Integrating heterogeneous big data systems (SQL, tensor, graph) into a single coherent pipeline
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are building distributed data pipelines that exchange data between multiple big data frameworks (Mars, PyTorch, GraphScope, etc.) and want to eliminate serialization and IO overhead.
The package is production-stable, permissively licensed, and actively maintained. Install friction is moderate due to platform-specific binaries, but the zero-copy sharing benefit justifies it for large-scale workloads. Not necessary for single-machine or simple data-passing scenarios.
Install
vineyard-bdist on PyPI
Before you install
Medium install friction due to compiled binary wheels for specific platforms (manylinux2014 aarch64 and x86_64). The package is marked as aging with last commit on 2026-01-22, but remains actively maintained and marked Production/Stable.
Requires a running Vineyard server instance and an IPC socket path; binary wheels are platform-specific (Linux x86_64 or aarch64).
License in practice
Licensed under Apache License 2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions—suitable for most production and research contexts.
Quickstart
pip install vineyard-bdist
import vineyard
client = vineyard.connect(vineyard_ipc_socket)
data = client.get(vineyard_object_id)Verify before relying
- Whether vineyard-bdist is the correct distribution for your use case versus the main vineyard package
- Specific performance gains or latency improvements in your distributed workload
- Compatibility with your particular big data framework (Mars, GraphScope, PyTorch, etc.)
Package facts
| License | Apache License 2.0 permissive |
| Python support | Not specified |
| Install friction | Medium. Platform-specific wheel |
| Runtime dependencies | None |
| Maintenance | Aging 715 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 102,682 / month, #12,854 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 5 - Production/StableIntended Audience :: DevelopersIntended Audience :: Science/ResearchLicense :: OSI Approved :: Apache Software LicenseOperating System :: MacOS :: MacOS XOperating System :: POSIXProgramming Language :: PythonProgramming Language :: Python :: 3Programming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.6Programming Language :: Python :: 3.7Programming Language :: Python :: 3.8Programming Language :: Python :: 3.9Topic :: Software Development :: LibrariesTopic :: System :: Distributed Computing |
Evidence: vineyard_bdist-0.24.2-py3-none-manylinux2014_aarch64.whl; vineyard_bdist-0.24.2-py3-none-manylinux2014_x86_64.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “in-memory data sharing”
- vineyard-bdistVineyard is an in-memory immutable data manager that enables…
- vineyardVineyard is an in-memory immutable data manager that enables…
- pylibmcpylibmc is a Python client for memcached, the distributed…
Give your agent the search over MCP, or paste the wish link into any chat.
More Libraries packages
urllib3 is an HTTP client library that provides thread-safe connection pooling, SSL/TLS verification, multipart file uploads, request retries, compression support, and proxy handling for Python applications.
Requests is a Python HTTP library that simplifies sending HTTP/1.1 requests with automatic handling of headers, authentication, cookies, and response parsing.
Pluggy provides a plugin system that lets you define hook specifications and register implementations to be called in sequence, enabling extensible Python applications without tight coupling.
Install it if you're building an extensible application or framework.
Provides parsing, arithmetic, and recurrence rule computation for dates and times, with timezone support and iCalendar RFC compliance.
Install it if you need to parse flexible date strings, compute relative dates, handle timezones, or work with recurrence rules—it's the de facto choice for these tasks.
Six provides utility functions to write Python code that runs on both Python 2.7 and Python 3.3+, smoothing over language differences between the two versions.
pytest is a testing framework that lets you write test functions using plain assert statements and automatically discovers and runs them, with detailed failure reporting.
See also vineyard · dask · mooncake-transfer-engine-cuda13 · mooncake-transfer-engine · apache-flink-libraries · apache-tvm-ffi · pyspark-pandas · nvidia-nvshmem-cu12 · delta-sharing · litdata