lsst-daf-butler
An abstraction layer for reading and writing astronomical data to datastores.
Decision gist · record as of 2026-08-14
Yes, if you are working within the LSST/Rubin Observatory ecosystem or need to integrate with their data pipelines. The package is actively maintained, has no known security vulnerabilities, and installs cleanly. However, it is a specialized tool tightly coupled to a specific observatory's data model; it is not a general-purpose data access layer. Verify that your use case aligns with LSST's data organization before adopting it.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.11 or higher.
- If using PyArrow version 18 or later, set ARROW_DEFAULT_MEMORY_POOL=jemalloc environment variable to avoid memory leaks when reading Parquet files.
- Active maintenance with a release within the last day.
License · maintenance · safety
BSD-3-Clause OR GPL-3.0-or-later (unclear) — Dual-licensed under GPL-3.0-or-later and BSD-3-Clause with unclear license treatment in the package metadata. Users may choose which license to apply, but should review both gpl-3.0.txt and bsd_license.txt files to understand their obligations.
last release 2026-08-13 (1 days) · last repo commit 2026-08-14 · 15 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 176,601 downloads/mo, #10,239 on PyPI
Alternatives
Verify before relying
pip install lsst-daf-butler
from lsst.daf.butler import Butler
butler = Butler(config='path/to/butler.yaml')
dataset = butler.get('dataset_type', dataId={})- Whether the memory leak workaround (ARROW_DEFAULT_MEMORY_POOL=jemalloc) is still necessary for current PyArrow versions
- Whether the package is suitable for non-LSST/Rubin Observatory use cases or if it is tightly coupled to that ecosystem
What it is and what it does
lsst-daf-butler is a data access framework designed for the Vera C. Rubin Observatory's LSST project, providing a Python abstraction layer that sits between scientific applications and underlying datastores. It handles the storage and retrieval of astronomical image data and related metadata, allowing researchers to work with large datasets without needing to know the details of the underlying storage system—whether files are stored locally, in cloud buckets, or in databases.
The package is built on a stack of mature dependencies: SQLAlchemy for database operations, PyArrow for efficient columnar data handling, NumPy for numerical work, and Pydantic for data validation. It requires Python 3.11 or higher and is actively maintained by the LSST collaboration. The documentation notes a known memory leak issue in PyArrow version 18 and later when reading Parquet files with the default allocator; the workaround is to set the ARROW_DEFAULT_MEMORY_POOL environment variable to jemalloc.
Use it for
- Access and query large astronomical survey datasets from the Rubin Observatory without managing storage details directly
- Build data pipelines that read and write image metadata and provenance information in a standardized way
- Integrate with LSST processing systems that rely on the butler for consistent data organization
- Retrieve Parquet-formatted astronomical data while managing memory efficiently in long-running analysis jobs
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are working within the LSST/Rubin Observatory ecosystem or need to integrate with their data pipelines.
The package is actively maintained, has no known security vulnerabilities, and installs cleanly. However, it is a specialized tool tightly coupled to a specific observatory's data model; it is not a general-purpose data access layer. Verify that your use case aligns with LSST's data organization before adopting it.
Install
lsst-daf-butler on PyPI
Before you install
Active maintenance with a release within the last day. Low install friction with a pure-Python wheel. Requires Python 3.11 or higher and depends on a moderately sized stack of scientific and database libraries (SQLAlchemy, PyArrow, NumPy, Pydantic).
Requires Python 3.11 or higher. If using PyArrow version 18 or later, set ARROW_DEFAULT_MEMORY_POOL=jemalloc environment variable to avoid memory leaks when reading Parquet files.
License in practice
Dual-licensed under GPL-3.0-or-later and BSD-3-Clause with unclear license treatment in the package metadata. Users may choose which license to apply, but should review both gpl-3.0.txt and bsd_license.txt files to understand their obligations.
Quickstart
pip install lsst-daf-butler
from lsst.daf.butler import Butler
butler = Butler(config='path/to/butler.yaml')
dataset = butler.get('dataset_type', dataId={})Verify before relying
- Whether the memory leak workaround (ARROW_DEFAULT_MEMORY_POOL=jemalloc) is still necessary for current PyArrow versions
- Whether the package is suitable for non-LSST/Rubin Observatory use cases or if it is tightly coupled to that ecosystem
Package facts
| License | BSD-3-Clause OR GPL-3.0-or-later unclear |
| Python support | Supports the current Python release >=3.11.0 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 11 packagesastropypyyamlsqlalchemyclicklsst-sphgeomlsst-utilslsst-resourcesdeprecatedpydanticpyarrownumpy |
| Maintenance | Actively maintained 1 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 176,601 / month, #10,239 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Intended Audience :: Science/ResearchOperating System :: OS IndependentProgramming Language :: Python :: 3Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Topic :: Scientific/Engineering :: Astronomy |
Evidence: lsst_daf_butler-30.2026.3300-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “astronomical data access framework”
- lsst-daf-butlerProvides an abstraction layer for reading and writing astronomical…
- astroqueryAstroquery provides a Python interface to query and retrieve data…
- specutilsSpecutils provides Python representations of astronomical spectra and…
Give your agent the search over MCP, or paste the wish link into any chat.
More Astronomy packages
Albumentations applies image transformations to training data, supporting classification, segmentation, object detection, and pose estimation with a unified API for images, masks, bounding boxes, and keypoints.
Install it if you need a unified, production-grade augmentation API for computer vision tasks.
Converts dates between the Gregorian calendar and 15+ alternative calendar systems including Hebrew, Islamic, Julian, French Republican, Mayan, and others, plus calculates holidays across North American, Jewish, and other traditions.
Install only if you actually need multi-calendar support; the single pymeeus dependency is manageable.
Converts between Julian dates and Gregorian or Julian calendar dates, storing Julian dates as pairs of floating-point numbers for maximum precision.
However, be aware that it will not receive bug fixes or updates—verify that its behavior matches your requirements before committing to it in a production system.
PyERFA wraps the ERFA C library (Essential Routines for Fundamental Astronomy) as NumPy universal functions, exposing key astronomy algorithms for scalar or array inputs.
Install it if you need fundamental astronomy calculations or if another package depends on it.
PyMeeus implements astronomical algorithms from Jean Meeus's classical reference, providing Python classes for computing celestial positions, lunar phases, planetary phenomena, and time conversions with minimal dependencies.
However, verify Python version compatibility first and consider whether other libraries better suit production-grade use cases.
PyEphem computes high-precision positions of planets, moons, stars, comets, and asteroids in the sky for any observer location and date, using C-based numeric routines from the XEphem astronomy application.
See also astroquery · skyfield-data · asdf-standard · asdf-astropy · pyds9 · pyvo · datasets · photutils · specutils · regions