collate-sqllineage

What it is and what it does

collate-sqllineage is a SQL lineage analysis tool that parses SQL statements and identifies which tables are read from (sources), written to (targets), and any intermediate tables created during multi-statement execution. It abstracts away the complexity of SQL parsing by using pluggable backends—sqlfluff or sqlparse—to build an abstract syntax tree, then stores the lineage relationships in a graph using networkx. You can use it as a command-line tool to analyze SQL files or query strings, or import it as a Python library to programmatically extract lineage data.

The package supports multiple SQL dialects, column-level lineage tracking (showing which source columns flow into which target columns), and can visualize the resulting data flow as a directed acyclic graph in a browser. It identifies intermediate tables when analyzing multiple statements in sequence, making it useful for understanding complex data pipelines. This is a fork of the original sqllineage project, maintained by the OpenMetadata community with added sqlfluff support.

Use it for:

• Extract source and target tables from INSERT, SELECT, and CREATE statements to understand data dependencies in a pipeline.
• Analyze multi-statement SQL scripts to identify intermediate tables and trace end-to-end data flow from raw sources to final outputs.
• Track column-level lineage to see which source columns contribute to each target column in complex transformations.
• Visualize SQL data lineage as a directed graph to document data flow for data governance or impact analysis.
• Validate SQL syntax across different database dialects (PostgreSQL, BigQuery, Snowflake, etc.) and catch keyword conflicts.
• Integrate lineage extraction into data cataloging or metadata management systems to automatically populate table dependencies.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Analyzes SQL statements to extract source and target tables, intermediate tables, and column-level lineage, using pluggable parsers (sqlfluff or sqlparse) and networkx for graph representation.

Yes. The package is actively maintained, has low install friction, carries no known vulnerabilities, and solves a concrete problem—extracting data lineage from SQL without manual parsing. Use it if you need to understand table dependencies, build data catalogs, or visualize SQL data flow. The permissive MIT license removes licensing concerns.

Install

collate-sqllineage on PyPI

pip install collate-sqllineage

uv add collate-sqllineage

poetry add collate-sqllineage

Installing collate-sqllineage

Before you install

Low install friction with four runtime dependencies (sqlparse, networkx, collate-sqlfluff, sqlglot). Active maintenance with a release 50 days ago. Supports Python 3.9, 3.10, 3.11.

License in practice

MIT license (permissive) means you can use, modify, and distribute this package with minimal restrictions, provided you include the license notice.

Quickstart

pip install collate-sqllineage

from sqllineage.runner import LineageRunner

runner = LineageRunner("insert into db1.table1 select * from db2.table2")
print(runner.source_tables)  # {Table: db2.table2}
print(runner.target_tables)  # {Table: db1.table1}

Verify before relying

• Whether collate-sqlfluff (a fork-specific dependency) is actively maintained and compatible with current sqlfluff versions
• Performance characteristics when analyzing very large SQL files or complex multi-statement batches
• Whether the graph visualization feature requires additional system dependencies beyond the listed runtime packages

Package facts

Evidence: collate_sqllineage-2.1.4-py3-none-any.whl

Tags

More Database packages