dlt
dlt is an open-source python-first scalable data loading library that does not require any backend to run.
Decision gist · record as of 2026-08-14
Yes. dlt is actively maintained, has no known vulnerabilities, low install friction, and a permissive license. It solves a real problem—automating tedious data loading—with a clean, Pythonic API and support for many sources and destinations. The declarative resource model and schema inference reduce boilerplate significantly. Start with a simple REST API or SQL database extraction to evaluate fit.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.10 or later.
- Optional extras (e.g., dlt[duckdb], dlt[bigquery]) needed for specific destinations.
- Low install friction with a pure-wheel distribution and 26 runtime dependencies already packaged.
License · maintenance · safety
Apache-2.0 (permissive) — Apache-2.0 permissive license allows commercial use, modification, and distribution with minimal restrictions. No licensing concerns for most use cases.
last release 2026-08-11 (3 days) · last repo commit 2026-08-14 · 5,738 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 8,210,929 downloads/mo, #1,652 on PyPI
Alternatives
Verify before relying
pip install dlt
import dlt
from dlt.sources.rest_api import rest_api_source
source = rest_api_source({
"client": {"base_url": "https://pokeapi.co/api/v2/"},
"resources": [{"name": "pokemon", "endpoint": {"path": "pokemon"}}],
})
pipeline = dlt.pipeline(pipeline_name="pokemon", destination="duckdb", dataset_name="pokemon_data")
print(pipeline.run(source))- Whether the 5000+ sources mentioned in the description are pre-built integrations or community-contributed templates.
- Performance characteristics and scalability limits for large datasets or high-frequency incremental loads.
- Maturity and stability of Python 3.14 experimental support in production environments.
What it is and what it does
dlt is a Python library that handles the repetitive parts of data pipelines: extracting from REST APIs, SQL databases, cloud storage, or DataFrames; inferring and normalizing schemas automatically; and loading into any of 20+ destinations. You define sources declaratively using decorators or configuration objects, then point a pipeline at a destination—dlt manages credentials, DDL, type mapping, staging, and schema drift for you. It's a library, not a platform: you pip-install it into your existing code and keep your workflow intact.
The package supports incremental loading (load only new or changed rows), merge strategies (upsert on primary key), schema contracts (freeze, evolve, or discard unexpected data), and secrets injection from environment variables or config files. It works anywhere Python runs—Colab notebooks, AWS Lambda, Airflow DAGs, local scripts, or AI coding agents. The Dataset API lets you reconnect to a pipeline by name and read tables back in the format your tool needs (DataFrame, SQL query, etc.).
Use it for
- Load REST API data into DuckDB or Snowflake with automatic pagination, filtering, and schema inference.
- Replicate tables from a MySQL or PostgreSQL database into a data warehouse with incremental updates.
- Ingest CSV or Parquet files from S3, GCS, or Azure into a destination, handling schema drift automatically.
- Build an Airflow DAG that extracts from multiple sources, normalizes nested data, and upserts into BigQuery.
- Merge pandas or Polars DataFrames into a warehouse with zero-copy Arrow support and type safety.
- Enforce data quality at the gate using schema contracts that reject or adapt unexpected columns and types.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes.
dlt is actively maintained, has no known vulnerabilities, low install friction, and a permissive license. It solves a real problem—automating tedious data loading—with a clean, Pythonic API and support for many sources and destinations. The declarative resource model and schema inference reduce boilerplate significantly. Start with a simple REST API or SQL database extraction to evaluate fit.
Install
dlt on PyPI
Before you install
Low install friction with a pure-wheel distribution and 26 runtime dependencies already packaged. Active maintenance—last release 3 days ago, repository at 5738 stars, continuous commits. Supports Python 3.10 through 3.14, though 3.14 support is noted as experimental.
Requires Python 3.10 or later. Optional extras (e.g., dlt[duckdb], dlt[bigquery]) needed for specific destinations.
License in practice
Apache-2.0 permissive license allows commercial use, modification, and distribution with minimal restrictions. No licensing concerns for most use cases.
Quickstart
pip install dlt
import dlt
from dlt.sources.rest_api import rest_api_source
source = rest_api_source({
"client": {"base_url": "https://pokeapi.co/api/v2/"},
"resources": [{"name": "pokemon", "endpoint": {"path": "pokemon"}}],
})
pipeline = dlt.pipeline(pipeline_name="pokemon", destination="duckdb", dataset_name="pokemon_data")
print(pipeline.run(source))Verify before relying
- Whether the 5000+ sources mentioned in the description are pre-built integrations or community-contributed templates.
- Performance characteristics and scalability limits for large datasets or high-frequency incremental loads.
- Maturity and stability of Python 3.14 experimental support in production environments.
Package facts
| License | Apache-2.0 permissive |
| Python support | Supports the current Python release <3.15,>=3.10 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 26 packagesclickfsspecgitpythongiturlparsehumanizejsonpath-ngorjsonpackagingpathvalidatependulumpluggypytzpywin32pyyamlrequestsrequirements-parserrich-argparsesemversetuptoolssimplejsonsqlglottenacitytomlkittyping-extensionstzdatawin-precise-time |
| Maintenance | Actively maintained 3 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 8,210,929 / month, #1,652 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 5 - Production/StableIntended Audience :: DevelopersLicense :: OSI Approved :: Apache Software LicenseOperating System :: MacOS :: MacOS XOperating System :: Microsoft :: WindowsOperating System :: POSIX :: LinuxProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Programming Language :: Python :: 3.14Topic :: Software Development :: LibrariesTyping :: Typed |
Evidence: dlt-1.30.0-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “schema inference and normalization”
- dltdlt automates extraction, schema inference, and loading of data from…
- django-log-formatter-asimFormats Django application logs in Azure Sentinel's ASIM (Advanced…
- sagemaker-schema-inference-artifactsProvides pre-built schema inference artifacts and task sample…
Give your agent the search over MCP, or paste the wish link into any chat.
More Libraries packages
urllib3 is an HTTP client library that provides thread-safe connection pooling, SSL/TLS verification, multipart file uploads, request retries, compression support, and proxy handling for Python applications.
Requests is a Python HTTP library that simplifies sending HTTP/1.1 requests with automatic handling of headers, authentication, cookies, and response parsing.
Pluggy provides a plugin system that lets you define hook specifications and register implementations to be called in sequence, enabling extensible Python applications without tight coupling.
Install it if you're building an extensible application or framework.
Provides parsing, arithmetic, and recurrence rule computation for dates and times, with timezone support and iCalendar RFC compliance.
Install it if you need to parse flexible date strings, compute relative dates, handle timezones, or work with recurrence rules—it's the de facto choice for these tasks.
Six provides utility functions to write Python code that runs on both Python 2.7 and Python 3.3+, smoothing over language differences between the two versions.
pytest is a testing framework that lets you write test functions using plain assert statements and automatically discovers and runs them, with detailed failure reporting.
See also definite-sdk · dlthub · intake · dlthub-client · dagster-dlt · substrait · databricks-dlt · dbt-duckdb · petl · bigquery-schema-generator