bigquery-schema-generator
BigQuery schema generator from JSON or CSV data
Decision gist · record as of 2026-08-14
Yes, with conditions. The tool fills a genuine gap in BigQuery's schema detection workflow. However, dormant maintenance (944 days since last release) and high install friction are concerns. Install if you regularly work with BigQuery data ingestion and need reliable schema inference across complete datasets; avoid if you need active maintenance or have strict dependency constraints.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.6 or higher; the generate-schema command may be installed in different locations depending on your Python environment (e.g., /usr/bin, /usr/local/bin, or ~/.local/bin).
- High install friction: the package is distributed as a source tarball with no runtime dependencies listed, suggesting potential build or environment setup requirements.
- Maintenance is dormant—last release was 944 days ago, though the repository remains active with a recent commit.
License · maintenance · safety
Apache 2.0 (permissive) — Licensed under Apache 2.0 (permissive), allowing free use, modification, and distribution with minimal restrictions; suitable for both open-source and proprietary projects.
last release 2024-01-13 (944 days) · last repo commit 2024-01-13 · 249 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 787,914 downloads/mo, #5,059 on PyPI
Alternatives
Verify before relying
pip3 install bigquery-schema-generator
generate-schema < file.data.json > file.schema.json
# Or for CSV:
generate-schema --input_format csv < file.data.csv > file.schema.json- Whether the high install friction is due to compiled dependencies, system libraries, or simply source-only distribution.
- Current compatibility with modern BigQuery API versions and schema features.
- Performance characteristics when processing very large datasets.
What it is and what it does
BigQuery Schema Generator is a command-line tool and Python library that infers BigQuery table schemas by scanning complete datasets. Unlike BigQuery's built-in auto-detection (which examines only the first 500 records), this tool processes all input records to produce a more accurate schema, especially useful when data fields are added organically over time and may not appear in early records.
The tool accepts newline-delimited JSON or CSV format on standard input and outputs a BigQuery-compatible schema in JSON format. It can be used as a standalone command-line utility or imported as a library for programmatic schema generation. It supports various options for handling nulls, quoted values, type inference modes, and schema merging with existing schemas.
Use it for
- Generate accurate BigQuery schemas for datasets where fields appear sporadically across records, ensuring no data is lost during table creation.
- Pre-validate and generate schemas for CSV or JSON data before bulk loading into BigQuery using the bq load tool.
- Merge new schema information from incremental data exports with existing table schemas to keep them up-to-date.
- Debug and inspect inferred field types and structure from raw data files before committing to a table design.
- Automate schema generation in data pipelines where data format or structure may vary across sources.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, with conditions.
The tool fills a genuine gap in BigQuery's schema detection workflow. However, dormant maintenance (944 days since last release) and high install friction are concerns. Install if you regularly work with BigQuery data ingestion and need reliable schema inference across complete datasets; avoid if you need active maintenance or have strict dependency constraints.
Install
bigquery-schema-generator on PyPI
Before you install
High install friction: the package is distributed as a source tarball with no runtime dependencies listed, suggesting potential build or environment setup requirements. Maintenance is dormant—last release was 944 days ago, though the repository remains active with a recent commit.
Requires Python 3.6 or higher; the generate-schema command may be installed in different locations depending on your Python environment (e.g., /usr/bin, /usr/local/bin, or ~/.local/bin).
License in practice
Licensed under Apache 2.0 (permissive), allowing free use, modification, and distribution with minimal restrictions; suitable for both open-source and proprietary projects.
Quickstart
pip3 install bigquery-schema-generator
generate-schema < file.data.json > file.schema.json
# Or for CSV:
generate-schema --input_format csv < file.data.csv > file.schema.jsonVerify before relying
- Whether the high install friction is due to compiled dependencies, system libraries, or simply source-only distribution.
- Current compatibility with modern BigQuery API versions and schema features.
- Performance characteristics when processing very large datasets.
Package facts
| License | Apache 2.0 permissive |
| Python support | Supports the current Python release >=3.6 |
| Install friction | High. Source build required |
| Runtime dependencies | None |
| Maintenance | Dormant 944 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 787,914 / month, #5,059 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
Evidence: bigquery-schema-generator-1.6.1.tar.gz
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “bigquery schema generation”
- bigquery-schema-generatorGenerates BigQuery table schemas from complete JSON or CSV datasets…
- ddlparseParses CREATE TABLE DDL statements from MySQL, MariaDB, PostgreSQL,…
- db-dtypesProvides pandas extension data types for SQL systems like BigQuery…
Give your agent the search over MCP, or paste the wish link into any chat.
More Database packages
psycopg2-binary is a PostgreSQL database adapter for Python that implements the DB API 2.0 specification, enabling Python applications to connect to and query PostgreSQL databases with thread-safe concurrent operations.
Python client library for connecting to and executing commands against Redis key-value stores, supporting both synchronous and asynchronous operations.
Install it if your application needs to interact with Redis; the only prerequisite is a running Redis server instance.
YDB Python SDK is the official client library for connecting to and querying YDB databases from Python applications.
Install it if you need to connect Python applications to YDB databases.
Connects Python applications to Snowflake data warehouses using the DB API 2.0 specification, enabling SQL queries, data transfers, and warehouse operations.
sqlparse tokenizes SQL text into a tree of statements, clauses, and expressions, and provides functions to split scripts, format queries, and inspect parsed tokens without validating dialect or syntax.
Install it if you need to manipulate, format, or analyze SQL text programmatically.
Provides base adapter protocols and shared functionality that database adapters use to integrate with dbt-core, handling connections, dialect translation, relation caching, and core interface management.
See also ddlparse · bigquery-magics · google-cloud-bigquery-storage · google-cloud-bigquery · google-cloud-bigquery-connection · dlt · google-cloud-bigquery-datatransfer · bigquery · bigframes · jsonlines